I was thinking an alliance would be beneficial for the Russians because they don't have to worry about an enemy blocking them in the Bospherous, and overall is just an alliance that is kind of untouchable.
It would be quite a useful alliance for both sides. Russia gets free access through the straights which they perused for centuries and the Ottomans have one of the most dangerous of their numerous fronts guarded and both get a helping hand in Europe, especially when dealing with the Habsburgs in eastern Europe and the Balkans.
Still there are several political, ideological and spiritual roadblocks for this alliance to work In the long term. The biggest of these being Constantiople/Instanbul and the legacy of the Roman Empire. Both Russia and the Ottomans(the Orthodox and Islamic respectively) claim to be the successors of the Byzantines and Constantiople is a big part of that claim.
Always gonna be a sore spot cause even with an alliance Russia is going to want it in case the alliance fails, for prestige and legitimacy and for religious reasons.
As for the matter of the 'Invincible Alliance' I would deny that it's very formidable and no other duo or even trio of continental powers is going to challenge it but between them they have a loooong border to cover and due to logistics they can't fully support one another everywhere(Ottomans can't help near the Baltic region and Russia can't help in Africa and Arabia.
Yet the strength of this alliance is what makes it fragile. Pretty much every European nation, and depending on the time period a few Non-European nations are going to be whispering and making deals with eachother to handle this Dual Alliance of the East&South.
The UK especially doesn't want this alliance disrupting the balance of power they covet. So British naval power and threat across the Ottoman coastline, with the Russian navy kept bottled up. The Habsburgs, along with other German states and likely Sweden providing the land armies with the possibility of France joining if this alliance threatens their interests in North Africa and the Middle East.
Even if they can't knock out Russia the allies against them are going to stay on the defensive with them and pick of the smaller and weaker of the two who was more hospitable terrain and weather to invade and occupy. Knock out the Turks and Russia starts finding itself in trouble cause the Straits are blocked and they are still stuck in a land war that' either going to end in them winning with an even larger alliance that feels more threatened, status quo or Russia suffering losses.