Come to think of it, even Saruman does not have much "magic".
Whatever little we saw was at Orthanc, when he and Gandalf fought
each other, and Aragorn said they only needed to be wary of his
words. Gandalf - the little fireworks at the Shire for the kids,
defending himself before Aragorn's sword, Gimli's axe and Legolas'
arrow when he first appeared as White Wizard at Fangorn forest,
transforming himself then Theoden King at Rohan - hardly the stuffs
used to fight Orcs. Guess there's a more realistic limit to what
powers a wizard possess, instead of flying around on broomsticks,
et al. Wisdom is probably the greatest prowess they have.
I think we should not mistake a servant's affection for his master
as being gay. Remember, they come from the Shire, hitherto
untouched by evil and cynicism of the world we live in today, so
they obviously speak differently.