Yep, that's the reason I don't go professional.
How do you define "huge"? And where do you draw the line between loving your job but not hauling down monster bucks and GETTING the big paychecks but loathing going in every morning... and everything in between? I started out down a career path that would've ended up in the category of "wealthy", but after even just a little while in the field, I realized that the environment in which I was going to work was not something I could be happy with in the long run, no matter the remuneration. I've since gone into two fields that were each very low-paying by comparison, considerably more difficult work-wise and generally ignored or even frowned upon by the "right people" as being without worth (i.e., "it doesn't mean big pay"), but which were both work that I do not dread every day, that is genuinely worthwhile instead of upholstered parasitism (in my book, anyway) and that gives me some sense of doing what I actually enjoy.
In other words, if it's something you enjoy, then perhaps the "huge" money (I'd still like to see the numbers defined) won't end up mattering as much to you in the end. And if you play your cards right, you CAN make a reasonable income doing something you like; freelancers in my field can make $0.15 per word if they're specialized. Me, if I could make freelancing a steady income (i.e., a regular client base), I'd actually be making MORE, and while I'm not upper-class, I'm definitely well into middle-class. A comfortable income if it's handled right. And I don't resent the work, which is more than most people I've met can say.