My impression of a lot of the 'tools' in spellcasting are there for safety, not true necessity. For example, how much of Harry's will does his fire lighting spell take? not much. Do you think he gathers that much of his will when, say, someone cuts him off in traffic?
IMHO, The spoken words and different language are there to keep you from accidentally doing something when you are irritated or surprised. Even Harry, who is a full white council wizard, sometimes has trouble with his gathered magic. Take a look as some of the early scenes with Molly, where Harry dissipates her gathered power before something accidental and unfortunate can happen.