And actually it seems they have moved south in Tanzania as recently as 1300 BCE, so what would stop them from going on?
They were neolithic pastoralists, while the Bantu had West African farming practices and used iron tools. Outside of the lands not suited for farming, they were assimilated.
The Chalbi Desert and the Sudd impeded movement southward, while there wasn't as much of a boundary between West and Central Africa. Maybe some sort of conquest or other disaster in Ethiopia or Nubia could cause a migration away and, if they cross the natural boundaries present, reach the East African Highlands that were suited for the crops grown in Ethiopia?