Lets see how it would affect a game...
I think the only person it'd be relevant for would be, as has been said earlier, a wizard with a really high presence. You'd only ever spend that many skill points if you plan to get in to a lot of social conflicts as otherwise it'd be a waste. Having high Discipline, high conviction and high presence (+5 presence required minimum), plus quite probably a high social attack skill too, almost guarantees you'd be weak in a physical conflict. Taking a social consequence WOULD have an affect on that PCs game.
If you're concerned that social stress doesn't matter enough in a physical conflict then simply rule that the backlash stress only hits the physical or mental stress track, but that consequences may be social if it makes narrative sense. It's not as if consequences are limited in use anyway. A broken nose or dislocated jaw might be physical but they could certainly be compelled in a social conflict; a sudden fear of heights might be a mental consequence but that could affect a physical conflict on a roof top, and a social consequence like 'deep humiliation' could be tagged in a mental conflict with the person that embarrassed them.
I say, if someone has invested heavily in social skills (which is rare anyway) they should be allowed to take advantage of them... in fact they should probably be rewarded for not making another 'let's-see-how-hard-I-can-hit' character (oh so I see you've max'ed out lore and given yourself focuses in creating items and refinements for extra items... but you don't plan on making anything. Again.)