Sadly, I think it often does play out this way.
I remember watching the Star Trek movie Insurrection and thinking, at first they try to relocate a people in a "nice" way or at least a straightforward way. The people resist, and then the more powerful side takes the view, we'll kill as many as necessary. How many times as thus played out in human history? Probably a bunch, but not every time. It's not a hundred percent deal.
Maybe if the colonists had focused more on fishing (which is a more stable food source anyway!) Well, this is both good and bad in that the colonists' population is likely to increase more rapidly, but they might hug the Eastern seaboard. And you might get the demographic factor in play that urban families tend to be smaller.
Maybe if getting a high school education became a big social norm for both boys and girls. Then you might get the demographic factors of delayed families snd smaller families.