As an American, I can definitively state not even all Americans would get it. I know the phrase, but I've never really been clear on whether it's a good or a bad thing to be built like a brick ****house. I mean, on one hand, I'd rather that then something easily tipped or otherwise destroyed, but on the other, it could totally be interpreted to mean someone who is built blockily and sturdily and not particularly conventionally prettily. (Someone saying this in admiration could simply be commenting on strength, not hotness.) I never interpreted it as meaning "constructed of superior quality than necessary".