I've had a similar idea only with a demon, but the concept might help you as well.
The character and a few friends had - by accident - summoned a demon. The character is a roleplaying geek, so he "knew" a bit about demons. He was able to bind the demon to himself, before it was able to do any harm. From that moment on, the demon and the character have become one, fighting over control of the body. The demon's agenda is destruction, and the longer he is not able to destroy or kill, the angrier and stronger he becomes. The character on the other hand is trying to keep the demon at bay, so he is directing the destructive forces of the demon in a better direction. The demon does not care what it destroys, as long as there is destruction.
I treat this as an Emissary of Power / White Court Hunger combination. Every scene that does not satisfy the demons agenda strengthens it, making it more likely to take over.
On the other hand, just killing the character would not help, because it would be a blood sacrifice to the demon, making it even stronger and helping it cross into our world entirely.
Maybe you can apply something similar to your character. The warden would not dare kill the character, because it would open the doorway for the outsider (and maybe even some of its friends) but he would constantly have to watch over him, and help him not lose control. Of course on the long run, the warden might try to get rid of the outsider for good, and if he does, he would change the characters high concept, which might not be wanted.