Both sentences are grammatically correct.
The question here is whether or not to use a singular noun or a plural
noun in this comparison.
The choice is stylistic: does the writer want to compare his subject
to one other (nonspecific) boy in the class, or to all of the boys in
the class? The Grammar Exchange would probably use either the singular
noun, as in your sentence (a):
He is taller than any other boy in his class;
or this alternative of sentence (b):
He is taller than any of the other boys in this
class.
Using any of the with a plural noun compares the subject
(he) to all the rest in a known, specific group. So, if you want to
compare the boy to all the others in the group, this
is a good way to do it.
Any other can be used with both singular and plural
nouns, as in these examples:
Singular: Any other man would have opened the door
for a lady.
Plural: I haven't met any other men who open doors
for ladies.
Sometimes the choice of whether to use a singular or plural noun with
any other is purely up to the writer and what he wants
to say:
Singular: I'm looking for any other book I can find
on this subject.
Plural: I'm looking for any other books I can find
on this subject.
Is the writer looking for one other book or any number of other books?
Only the writer can determine that. The writer can choose either sentence,
according to what he or she wants to say.