I would allow the immunity to work against all spells that weren't backed by the invocation of an appropriate aspect for effect.
On the one hand, I think that makes a lot of sense. On the other hand, I feel like it makes it too easy to bypass the immunity.
Air Wizard: I'm making a free Declaration that I am "Surrounded By Air", then I'm going to tag that Aspect to magically crush that Ogre with massive air pressure.
Earth/Spirit Wizard A: I'm making a free Declaration that there is "Stuff Around", then I'm going to tag that Aspect to magically hurl stuff into the Ogre.
Earth/Spirit Wizard A: I'm making a free Declaration that "Ogres have Mass", then I'm going to tag that Aspect to magically hurl the Ogre into the wall/ground/whatever.
Water Wizard: I'm making a free Declaration that "I have saliva", then I'm going to tag that Aspect to magically spit a high pressure volley into the Ogre's eye. (Okay, that one's a stretch, but you get the idea.)
Fire Wizard: This is arguably a bit tougher, but you could just carry a lighter around with you, or declare that the air contains "Ambient Heat" that you could magically drain for a Cold Attack.