I think PI works against things that aren't attacks, too. If there are any such things. It's basically Infinite Toughness.
Things that result in physical stress or consequences.
Which, 99.9...% of the time, are (at least adjudicated as) attacks of one sort or another.
If it doesn't result in physical stress or consequences, then PI doesn't interact with it, though there might sometimes be a justification to invoke-for-effect the aspect related to PI to render the character immune to a non-stress/consequence-inducing effect of another sort (a block created by a barrage of bullets, or zone border created by a wall of fire, etc.).