I would put the Nevernever location associated with the source of the bugs close to Summer. Strong life force and unregulated growth seems like a twisted summer thing.
I vaguely recall some Lovecraftian horror that involved a swarm of bugs.
A couple of ideas:
If you go with a large number of weaker swarms, then you can have a siege scenario, where the party forts up, perhaps with magic walls, or improvised defenses, dealing with the bugs poring slowly through, and having to figure a way out or to reverse the siege. Make it more of an endurance fight, as the total number of bugs seems endless. If the bugs do things like suddenly withdraw into the roof or floor, or other tactical maneuvers, the party will wonder after the battle if they actually got all of the swarms.
Options for fighting the swarm:
Form a Raid(tm). Get lots of insecticide, lots of anti-bug armor. Perhaps fireproof suits (thaumaturgical prep?) and just burn everything.
Find the local bat cave, and bring a few million allies to the fight.
Go after the source, in the Nevernever
Scholarly: use pheromones to lure the bugs, talk to a beekeeper, find out what preys on various bugs. Maybe get a type of tiny parasitic wasp that lays ITS eggs in that kind of bug. A human controlled swarm may not engage in the normal self-cleaning anti-parasite mode that an uncontrolled swarm would.
Massive quantities of flypaper
Thaumaturgy: Use the swarm against itself. Magic directed at one example of the tightly linked swarm could affect all, provided the party can pump enough power into their thaumaturgy before the link burns out.