Recall everyone that in 1813, the was virtually no British presence in NA. Canada was saved mainly by lack of American preparation for war and the great distances involved.
Thomas Jefferson famously boasted that taking Canada was just a matter of marching. Well, it was marching through about 500 miles of wilderness. If there had been a paved road between Boston and Quebec at this point, most of Canada would have been conquered within months.
I give the few British regulars and the Canadian fencibles reasonable credit, they fought well on the defensive, while outnumbered, and for one thing captured Winfield Scott. The US Army in 1812-13 was too small, only partially trained compared to European standards and relied entirely too much on militia (large numbers of whom flat out refused to cross the border) to be an effective instrument of war even against the relatively small force available to defend Canada. Better results occurred later in 1813 (at York) but that was really a glorified raid. It wasn't until 1814 that the US Army fielded regulars equal in staying power, discipline and willingness to take casualties as the British, which is shown very clearly at Lundys Lane.
By then the window of opportunity was closed and both sides had essentially parity in numbers as far as forces that could invade the other along the Canadian border. The US never did drive the British out of Maine either, getting it back with the peace treaty.
But that works both ways. The British lacked the numbers, and more importantly, the logistics to mount a successful invasion of the US. The operation aimed at Washington and Baltimore was never meant to be more than a raid because the British lacked those two essential ingredients and also lacked cavalry and artillery in any strength because they lacked the needed horses, mules and oxen needed to haul wagons, guns and mount dragoons or lancers. Nor were the Americans likely to supply them, because after a year of British raids along the Chesapeake the locals were well aware of the importance off keeping the British afoot and limited to as shallow penetration as possible.
Which is the whole problem with the idea of landing 20,000 men in Maryland proposal.
Also, going over a few quick sources (all my decent books on 1812 are in storage) I cannot find any British cavalry regiments assigned to North America at all during this time period. I might have missed something, but even at Lundys Lane, if Drummond had cavalry available to use in its traditional scouting role, he would have found the weakness in the American flank and turned a bloody draw into a possible victory. Since he didn't, I feel safe in assuming that cavalry wasn't available.
So in effect this army proposed by EnglishCanuck would be blind aside from a few mounted officers and lacking in artillery. Without it, assaulting the entrenchments at Baltimore has the excellent chance of being a costly victory like Bunker Hill or just as possibly, an army wrecking defeat as at New Orleans.
The Americans had cavalry. It wasn't of the quality to charge squares or even really function as a battlefield unit, but it was just fine for scouting and skirmishing and raiding. There wasn't much of it, but even a couple of regiments means that along with local civilians, the Americans are going to know what the British are doing on a routine basis. They had no trouble keeping track of the British after Bladensburg for that matter.
If you are going to risk potentially devastating casualties on the relatively elite but oh so limited in numbers British Army, you would be better off attacking something more vital. But the trouble is, the more vital places, like New York, Boston or Philadelphia are far more strongly defended, while less well defended cities like Charleston could be taken, but leave the British facing a hinterland not much more developed than when Cornwallis marched through it in the Revolution.
Although from a ATL standpoint, the a timeline where the British do that, and free slaves as they go, would have potentially dramatic consequences on the South and slavery in America.
But it still wouldn't conquer the US or even force the US to accept a peace that indicated defeat.
Now if the British had attacked New Orleans earlier in 1814, much earlier, while Jackson was still poorly supplied and with a far weaker force, taking that would indeed be an American disaster
But it probably simply would have meant that within a couple of decades the British and Americans fought a third war.