I think a well defined set of rules is very important.
It makes it feel more real, more thought out.
I'm inclined to disagree with that...
Random spell casting without a good system comes off as cheesey to me.
..though mostly to agree with this, depending on the feel you're aiming for. (Traditional fairy-tale logic, or magic that works by dream-logic, does not benefit from the feel of an underlying system.)
The problems with working out the rules for your setting as if you were playing D&D or Magic the Gathering is that it has the failure mode of your book reading like a write-up of a D&D game or a Magic game; and while there's any amount of D&D or MtG spin-off fiction out there, and certainly there's a market for books with neatly designed magical systems that work on about that level of complexity - I would think of Brandon Sanderson as an example of an author of original worlds that have that sort of feel - I can't say I find them appealing or convincing.
What gives a world like the DF the feeling of realism is that how magic works is
complex, and the more we find out about it, the more complex it is; it's not a well-defined set of rules you can sketch on one page, so therefore it plausibly comes across as something that you can spend years of study getting better at and understanding better without by any means having all the answers, as I think the volume of discussion about it in the on-topic parts of the forum demonstrates. Harry doesn't know everything; he understands the basics of his form of magic, but much of the rest of how the universe works he is picking up as he goes along, and sometimes he is out and out wrong; at the start of the series he believes that the magic he knows how to use is inherently a positive life-generated force and black magic only happens when that force is abused for evil ends; he later bumps into several examples of magic that appears to be inherently evil or otherwise hostile to life, such as Mavra's black barbed-wire spells in GP, the mordite in DM or the curse in BR. And, indeed, he has his assumptions about how inherently negative necromancy is shaken by Kumori's using it to keep the random gangster alive in DB.
I have no idea whether Jim has notes somewhere as to exactly how the black barbed-wire spell stuff relates to necromancy; there's plausible grounds in the text for arguing that they are the same thing, or unrelated forces. But that sort of uncertainty, on the edge of what a viewpoint character knows and where they have not had the time or the motivation to dig into it in depth, reads to me as more plausible for a realistic character's understanding of a realistic world than a neat system where everything is understood to the last detail.
In a similar vein, think through the consequences of what your magic actually does to your world. The number of fantasy universes that potter along with medieval-type populations despite having magical healing that clearly implies drastic reduction in infant mortality compared to a medieval setting is kind of ridiculous; and far too few people pay attention to what magic does to economics. (I highly recommend Daniel Abraham's Long Price Quartet as a good example of interestingly different magic being really integrated into the workings of a society.)
