Add Taxonomy to body Class

I wanted to add the taxonomy to the body class of single pages.
I added this function to functions.php.
It does add the taxonomy to single but strips away other classes in the category pages, thus eliminating the sticky header.

This is the function:

function add_taxonomy_to_single( $classes ) {
    if ( is_single() ) {
        global $post;
        $my_terms = get_the_terms( $post->ID, 'brand' );
        if ( $my_terms && ! is_wp_error( $my_terms ) ) {
            foreach ($my_terms as $term) {
                $classes[] = $term->slug;
            }
        }
        return $classes;
    }
}
add_filter( 'body_class', 'add_taxonomy_to_single' );

This is the link to the single page (sticky menu is working):
http://trm.ojs.mybluehost.me/furniture/chairs/bella-850-2/

This is the link to the category page (sticky menu stopped working):
http://trm.ojs.mybluehost.me/furniture/brand/chairs/

Any idea why the function is causing issue with the body class?

Thanks,
Dan