THe Holy Roman Empire surviving in the form it had in, say, 1650 would seem extremely unlikely. Even if the Napoleonic wars with their reorganisatzion didn't happen I would expect the greater states to sort of accrete smaller ones until some kind of balance is reached between a few major players.
If we're talking about the 19th century states, OTOH, I think that has potential. Definite potential. You'd need to blunt the thrust of nationalism somehow - maybe it's cultural (no Sturm und Drang), maybe historical (no 1848), maybe both... The competing force of local loyalty and distrust of the 'others' would have been able to hold its own, and if you posit a German customs union and maybe more freedom of movement and trade, that would allow you an outlet for idealistic identification with the 'whole'
A successful non-unified Germany might serve as an example for other European nations, making the desire for monolithic national-ethnic states less overwhelming. German would also likely stay more fashionable without a war of 1870. An image like OTL Scandinavia might develop - you know, nice, enviably peaceful people who are scarily good at all sorts of things, but ultimately need not be taken seriously because they're 'too different'.
Without the wars of 1866 and 1870-71, Prussia will have a harder time being taken seriously as a first-rate power. France remains a leading power with an unbroken record of victory and without an unhealthy obsession with 'revanche'. Russia and France would not find the unifying interest in keeping Germany down. They might seek to play diplomatic chess with German states, with Britain balancing the board...
No WWI, of course. If there is a great-power war, it might involve France and Russia, with Germany as a battlefield, or France and Britain, with various German states as British 'continental swords'. They had better get something out of it, otherwise there'd be clamoring for national unification (you know, 'never again be other nations' footmat!'). A Germany with the attitude of a 20th century neo-ethnic nation-state with something to prove is scary...
I could see it. Today, Germany is the heart of a European free-trade zone, its small states using a century of practice at the art of getting along in a compromise-based system. It is a prime tourist destination for those seeking to dream among romantic forests, walk in the footsteps of the brothers Grimm or Goethe, and experience first-hand the painted fairy-tale villages of children's-book fame. Its industries turn out high-tech goods, and many of its cities offer a haven to Jews fleeing persecution in Poland and Russia...
Adjust hue control if you'd like it darker