I don't think that defections would be a big problem, Romania participated at the 1984 Olympics and did not suffer from defections during the games.
Maybe, but the key is the
Soviet Government,
leadership and KGB did believe (all the time!), and whether justified or just Soviet p
aranoia or an actual risk, it was a major part of the decision whenever the Soviet leadership allowed anyone to travel to the west.
OTL, the Soviet's were looking at using cruise ships anchored out for their (and Warsaw Pact) teams, rather than the Olympic village (better access/egress control). As well as an army of KGB (very likely more than the US would allow), another possible excuse , "
the evil capitalists will not allow the peacefull Soviet people to provide 10 cultural advisors to assist each glorius Soviet Team member..."