I don't think 1980s are possible, unless WW2 finishes earlier and with a wildly different result. Perhaps a smaller scale Project Manhattan that would go slower and try concept one by one, instead of in parallel.
One you have a total war situation and a determined foe such as Japanese and US in the mix, I think nuclear weapons are near certainty. It cannot be derailed without significantly altering the knowledge and development of physics.
If you give the starting date as post WW1 you could have both A. J. Dempster and F. W. Aston dying in the Spanish Flu epidemic, thus setting back the invention of the mass spectrometer by some years, maybe into the '30s, and from there you could easily, through work on other projects, see the discovery of U-235 pushed back into say the '50s.
You have to kill the Manhattan Project, perhaps a 5th Columnist blows up critical personnel/documentation or perhaps that Japanese Fire Balloon in 45 does more than just shuts down 3 generators.
That still leaves a problem the Soviets are just 2 years really behind the Manhattan Project. The problem for the Soviets was primarily uranium shortage. Disrupting their supply of uranium or cutting it off would effectively prevent them from building a bomb anytime soon.
To ensure that the Soviets don't get the German Uranium piles the British or Americans ensure the destruction of those facilities before Berlin falls.
However given even these it would only push back probably 10-15 years their development so your talking maybe early 1960s.
It is possible in those years that the nations had come to terms with it and banned the research of such weapons. This would probably also end all research into nuclear energy, as all sides would be suspicious the other side was making a bomb. Even given that some of the governments would probably black book development, it would be slower you might could stretch it another decade or two and none would admit they had it without the first showing they had it.
The lack of nuclear deterrents will probably make the world much messier than we see it today. It means Korea or Vietnam may have become a full fledged WWIII or WWIV.