Seb, you're vastly oversimplifying it (as well as being overly snarky and hand-waving). It's not a case of "just thinking" about it. Read the passages where Harry talks about the props' use - they are a stand-in for the mental process of a detailed, intimate conceptualization of precisely what the object IS. Think about how hair is used in tracking spells; it's simply a link to the person the wizard wishes to track... i.e., a shortcut (no pun intended) to THAT identity. In fact, as Harry points out more than once, using a physical link can be compromised, while a clear, exhaustive, intimate conceptualization of the target's identity (assuming, as with the hair, that the link is still accurate) removes an extra layer of linked identification.
And yes, you can change the phone's number, switch phones, etc. - just like you can shave your hair off, jump into running water, etc. No connection short of a BEING'S true name is 100% reliable, so that's not an argument against using a phone number to track a phone that is associated with it, no more than it's an argument against using hair to track the person from whom it was cut.