It's mentioned a few times that even when he isn't actively projecting energy with the staff, the energy held at the ready makes it... metaphysically more than just a piece of wood, I guess? It definitely came up in Dead Beat where the energy he's holding ready in the staff lets him bludgeon corpsetaker's specters with melee strikes. In Proven Guilty he also crushes the skull of one of the fetches with the staff, and describes the blow as "powered by muscle and magic alike".
The understanding I had was that even when he's 'just' using his staff as a beatstick, it's an instrument of his Will and reinforced by his Will, and the physical strikes are empowered by that. edit: it's also why he can regularly use it as a prybar without snapping the wood.