The mutual traversing of individuals between two parallel worlds isn't a huge issue, nor is it a small one.
If you look at it from a minor perspective, the number of people who die every year from various natural disasters and man-made calamities is not less than this; whether it rains more or less, both can potentially cause tragic devastations, turning the land into a desolate state.
In some countries where human rights are not a priority, this isn't even considered an issue.
But if we examine the largeness of the matter, looking at the long-term perspective, the problem is actually very serious.
Let's say for instance, your entire extended family, tens of individuals, are all genuine Descendants of Yanhuang. Then one day, your niece brings home a student from Ethiopia, telling you he's the father of the child she's carrying.
You think it doesn't matter since she's not your own daughter.