I think, realistically, you need quite a remarkable, and early, POD to really get to the roots of this issue. Basically what happened in France in the interwar period was a division on the left over what the "acceptable" limits of socialism/radicalism were. Most parties were agreed that any Cartel des Gauches would not assume a Communist flavour, but beyond that there was deep division over economic policy.
In many ways the division of the French left in the period is about the aftershocks of the Paris Commune - one of the reasons the Radicals bumbled around the issue of Female Suffrage, for instance, was that some moderates in their ranks were concerned that coming out in favour of it would paint them as radical Communards (given the Commune's support of female rights). Likewise, anything that looked too militant or radical made the moderate elements of any left-wing coalition very jittery indeed.
I don't think this is an impossible AHC though - just difficult. Unless you want a 19th century POD (which probably butterflies the situation you are talking about completely) you need, I think, a rallying point around which the left can remind itself what unites it rather than what divides it.
For the French left in the early 1930s this is a couple of issues: a developing notion of anti-fascism, a vague desire to promote industrial harmony and 'protect workers' but without too much specificity, anti-clericalism, and above all an external threat that it can rally around. Just look at Blum's Popular Front in 1936 - although it collapsed in 1938 over the Spanish Civil War it managed an electoral breakthrough as a result of the left fearing the chaos of 1932-4 had been a fascist conspiracy to overthrow the Republic.
A couple of ideas:
*Have a small neighboring nation descend into chaos. Spain is a possibility, but also Belgium/Netherlands or Switzerland. 1932 is very different to 1936 and standing up for a Republican Government abroad is an ideological issue the left in France can broadly get behind (particularly if, like in SCW, said Government is endorsed by Moscow) but also the idea of regional security is a boon and distraction when dealing with the right in France.
*Alternately have a serious war scare happen before the 1932 election and the right-wing fumble the response. Pacifism was a major force on the French left. Note, though, that this cannot be a war scare which might pit French interests against Soviet ones as this would fracture, rather than unite, the left.
*Have some of the various ultra-right groups actually attempt an abortive coup sometime between 1932 and 1933 after the Cartel wins the election. OTL 1934 came close, but that was a spontaneous affair not a planned coup involving disgruntled soldiers. If something like the SCW general's plot or the Japanese 2-26 incident happens in France (eminently possible) then the left will be able to rally in indignation and support of the Government and the Republic. A bloody and unsuccessful coup that is quickly stomped out is a very different experience to the February riots of 1934 that brought Dalladier and the Cartel down OTL.
Just to spark some discussion (and bump the thread), I think I'd also like to ask the following question: what exactly caused France to go through five prime ministers in a little more than a year?
Just a note - this wasn't actually that unusual. Its to do with the unstable "Hollow Classes" cabinets of the interwar era. It plagued the right too.