You don't roll for initiative normally, you take your flat Alertness skill and that's your initiative. In the case of Inhuman Speed, it adds 4 to that number. So if your Alertness is a 2, then w/ Inhuman Speed your Initiative is considered a 6. If your group prefers rolling, then you'd get to roll at 6 instead of 2.
When you're actually making Athletics rolls, though, of course you get the bonus from Inhuman Speed. The proof is in the trapping description of "Athetlic Ability" on YS 178. If your GM can't understand that, find a new GM, I'm sorry.