That's why I think it is unreliable. Thomas was a small child, so his understanding of what he remembers is unreliable. Lord Raith isn't going to admit that he wasn't able to enthrall Margaret completely or at all. Lara who might know and would tell Thomas has her own agenda so I wouldn't fully trust what she said either. I do believe at some point this will all be revisited and hopefully Jim will have made up his mind by then as to how he wants that to resolve itself.
The thing is, people put together details after the fact; little kids don't STAY little. There's family gossip, the "odd silence" on some topics (or some aspects of some topics), the "We Don't Talk About Bruno" (proceeds to talk about Bruno), etc. There's also the stuff learned from outside sources... non-Raith whampires, non-whamp supernaturals, etc. (I know people in RL who have learned things (as adults) about their parents, that they had lived through without noticing "what was really going on").
I presume virtually all the supernaturals include (at least some) learning-about and direct-experience-with their peer-groups. We first meet Irwin in just such an environment.
It was never a secret that Thomas was the son of Margaret LeFay, and I'm quite sure Thomas had met (at least some) fae before he started hanging out with Harry; and they would likely have commented on his mother (much as many of the Fae have commented to Harry). Other Supernaturals, too (such as Goodman Grey).
We don't know
who Thomas may have spoken to, but I think it highly counter-intuitive to think Thomas would
not have had experiences similar to what we know Harry had.
So -- in addition to his childhood memories -- Thomas will have a lot of other inputs to assemble a broader and more-nuanced view of his mother.
I suspect Thomas using the phrase "...there was some sort of business between them..." is both Watsonian-Thomas acknowledging that he
didn't know the gist of the "business," and Doylist-Thomas cueing us that the "business" is an important matter still to be revealed.