It's doesn't fit logical consistency, and it flies in the face of multiple situations in the very narrative that provide the basis for this game. The problem is, you're looking at the combat rules purely in a vacuum. The idea that "either both are okay, or neither are" completely ignores the situations surrounding and following combat, which are part of the balancing factors, and I have brought them up several times. A character that turns everything into a physical fight doesn't get off scott free. Very often, he ends up in a much worse situation than if he had just taken his social knocks gracefully.

Of course, a lot of this comes down to GMing style. I am a firm believer that everything within a game needs to maintain an internal logical consistency. It needs to "make sense" so that the players can just make use of common sense (possibly a slightly modified version, but common sense nevertheless) to determine what they can and can't do, and what is a "good idea" and what is a "bad idea" without needing a deep, "crunchy" knowledge of rules mechanics. As far as I am concerned, if a rule has to fall back on "because it's a rule," rather than having an easily plausible in-world explanation for why X can/can't be done, it's a bad rule. It is hard to find such explanations for social situations not being allowed to descend into violence, because it happens a fair amount in the stories, and players can point to that.

Honestly, I like TheMouse's solution of combining things the best, though. Life is rarely cut and dry, in either our universe or the Dresdenverse. Excluding social combat from a fight and versa because "that's not the kind of conflict we're having right now" seems a very artificial shackling. A warlock can throw a mental attack during a fight, why not a social one? It lets a socially strong character potentially hamstring a fighter by tying up consequences in absorbing Social hits when they could otherwise be absorbing Physical damage, and vice versa.

Why should it be the other way? Just because you're good at Physical, if someone starts a social conflict, you can just punch them in the face and describe a Taken Out result where they don't have to talk to them?

That's a perfectly allowed course of action in reality. Why not in the game? It is a choice that can carry heavy consequences, but it is an allowed choice.

Violence is the ultimate negotiation. To quote Robert Heinlein, "Violence, naked force, has settled more issues in history than has any
other factor, and the contrary opinion is wishful thinking at its worst." With violence as the final potential arbitrator of any situation, what then makes social conflict a plausible option to begin with? There are no arbitrary rules of reality (the game) to force people into social conflict over physical ones so, generally, it is the conventions adopted by society as a whole.

In the Dresdenverse,society, both mortal and supernatural, has adopted various constructs of rules and regulations on when the use of violence is acceptable. Conversely, there are also punishments for individuals who defy those conventions. If someone is giving you too much lip in a bar and you decide to get physical rather than just take the social loss, then bouncers and potentially even bystanders and/or the police are going to get involved. If a wizard is losing a social fight to a WCV and lights him up, unless the vamp gave one of the strictly worded reasons for violence under the Accords, that wizard just committed an act of war to avoid losing one Social conflict. Just for the privilege of switching the mode of the fight to one of his strengths, he's not only going to have a bunch of Whites wanting his head, but probably a fair part of the White Council too.

When you're dealing with potent fae, who's deaths can unbalance nature itself, and wizards that can level city blocks with their death curse, violence outside of tight constraints is something the vast majority of entities is going to find objectionable in the extreme, and they will certainly make that displeasure felt most potently on anarchs that  regularly violate the conventions on combat. So in addition to the hazards of misjudging the combat strength of your social opponent and getting dead rather than simply humiliated, there is also all of the other fallout that comes from talking with your fists all the time. There is no need for a kludge rule fix when in-game consequences already provide heavy deterrent to abuse.

If something is important enough to a character where they are willing to risk all of that to start a fight (or if the character is just dumb or stubborn enough to do it anyway), maybe you should let them, and then let them deal with the consequences of that choice.

Happens to Harry all the time.

Well technically, being taken out in one branch does not automatically keep one from competing in the other two, as long as at it is logically plausible (being unconscious or dead isn't particularly conducive to social discourse, but being socially humiliated rarely prevents one from throwing a punch).

Since Physical conflicts tend to have the highest immediate stakes (pretty hard to be talked to death, though some of my high school teachers tried really hard), I wouldn't consider the Social conflict "ended" so much as "deferred" while the more immediately dangerous Physical conflict is resolved. Provided participants on both sides survive the fight (via concessions, or whatnot), the Social conflict could resume. Perhaps the defeated Red leader still has some intimidating "I'll get you next time, Gadget" commentary to throw out as he is being hauled off, threats of vengeance against loved ones, and such.

If you're feeling particularly ambitious, the Social conflict could even continue during the Physical conflict, with taunts and mind games and the like.

Fate rules are designed to be heavier on the narrative than the crunchy, so the best answer is probably to find whatever version works best for your group and makes for a fun scene.

There could also be unintentional targets for the effect. Perhaps the guards you're trying to ambush get veiled too. Bound to cause a stir when a thug's buddy guarding the door with him suddenly disappears, and now suddenly you have to account for an invisible enemy as well.