I agree with the principle stated above that any direct magical effects are negated (ie falls at his feet, disintegrate, etc.) So an avalanche or direct boulder would not effect/harm him.
However, secondary effects would still apply. For example, if that avalanche collapsed the ground beneath his feet, he would fall and be damaged that way (still by the fall, not by the avalanche). If I throw a rock, it misses you but causes an avalanche (one I did not evocate), the Immunity does not apply.
Regardless, though, don't make the GM arbitrarily decide each time. Come up with a definitive rule.