Actually, it's because they're too-much there. Sure, some physical aspects of the character are neglected (rendered glasses in particular would be annoying enough that they would actually be far more distracting than immersive), but ultimately the control is far more direct than on most 3rd-person control models.
Really, they'd only be annoying because of the false perspective an FPS is forced to use; because most people don't own a gigantic convex television that takes up their entire range of vision, we're forced to have an artificially narrow view of the world that consists only of central vision, and sometimes not even all of that. Gordon's glasses would be no more or less annoying than real glasses if they could put them where real glasses would be in his view, but they can't.
It goes far deeper than visible physical interactions, too. The reason so many FPS player characters don't speak except in cutscenes is that the direct association of first-person view means that if they say anything, it had better match up with the mental state of the player, and it's hard to do that consistantly with procedural speech act usage.
Yeah, but the problem is if they say nothing at all, the game really isn't asking the player to immerse themselves in anything; rather, it just has a blank space in the world for them to stand in and not have their presence challenged. It's like those cardboard cutouts of strongmen at beaches; a child can stick their head through the hole and pretend they're the strongman, but in reality there's nothing there; they're just a child sticking their head through a piece of cardboard, not being asked to act like they think a strongman would while they do so. Similarly, the silent protagonist is a cop-out; rather than trying to create a convicing everyman for the player to emphasise with ("I want to be this person"), it creates a nobody who they don't have to empathise with. It's avoiding immersion entirely so it can't possibly break it.
This falls down when we're asked to care about the nobody; for example, Alyx Vance falling in love with a man who has the personality of a tree stump, or trying to make people care about Rabbit dying in Medal of Honor even though the only way you can remember which one Rabbit even was was that he hung around with one of the guys from ZZ Top. Wait, no, that was Deuce. See?
- Evil Tim