I've written two complete murder mysteries and am about 2/3 of the way through a third--while that doesn't make me an expert by any means, I've learned a few things along the way:
1. Outlining really helps. I know some writers are dead set against having any kind of plan, but I personally need to outline each chapter just to help me remember who knows what at any given time and what everyone's motivations are. As the omnicient author of course I know everything that's going on, and one of the hardest things for me is to remember that every character is in the dark to some extent.
2. What I call the Alpha Story is the main plot line, e.g. The boss everyone hates has been murdered. Who did it and why? This can have a pretty straightforward answer, but since you want your novel to go on for more than half a dozen pages, you need to introduce a bunch of Beta Stories into the mix. Maybe everyone in the office is a suspect. What are their backstories, and how have they interacted with the hated boss in the past? Then there's the guy in the pinstriped suit who shows up every day at the water cooler at 10:00 AM precisely...but nobody seems to know who he is.
I'm a big fan of novels like Dirk Gently's Holistic Detective Agency, in which a number of seemingly random and unrelated events all turn out to fit together perfectly, like the pieces of a jigsaw puzzle. Once your Alpha Story is in place, you can start to fill in more pieces of the puzzle: Your protagonist has a chance encounter with a woman in a red dress...but it's not so much by chance, as she turns out three chapters later to be the assassin hired to kill the tabloid journalist who's been blackmailing the cheating wife of a corrupt businessman...who is none other than the guy in the pinstriped suit. Or something. You get the idea.
3. Know your protagonist. Is he/she a professional cop, a P.I., a true-crime writer, or a curious bystander? Maybe she's the sister of the boss, determined to find justice for her sibling. Who and what your protagonist is will determine much of what he/she can and can't do, level of skills such as marksmanship and martial arts, and how much access he/she has to the information needed to solve the case.
4. The setting, be it a major metropolis, an isolated village, or a fashionable resort, will determine the characters' behavior to a large extent. Your setting, if you're really good, can also be as much a character as the people who inhabit it.
5. Do the research. At the very least, arm yourself with an up-to-date map of your location and some good photographs. If possible, visit the place and spend some time walking around. Talk to someone who actually lives there. They will know that it takes more than five minutes to walk from the library to the Post Office and if there's an ordinance against keeping poultry in your backyard. If your protag. carries a gun, make sure that it's appropriate to the time period and place, that the character would realistically own such a firearm, and that the damage it does to the victim, or whomever, is accurate. (You don't need to go on for a whole chapter about it, though, just to prove you did the research. That will bore the hell out of most readers.)
6. As with all writing, remember: it's okay for your first draft to suck! Don't let it hold you back, just keep going. It doesn't get any easier, but you WILL improve.
