Justify?
Why does it need to be justified. Just have them be dragged into the nexus like Kiyoshi and Rider were due to the same anomaly happening again.
Just leave Shirou, cause really it would be interesting to see Kiyoshi trying to find another male role model while he is separated from his father. And honestly? Happy families do not need to be together for everything.
The fact that Sakura, Aoi, and Kiyoshi, despite being separated from Shirou, want nothing more to see him again. This unwavering conviction could bring them together to be closer then they ever would have been before, in ways that are not possible for them to in the world where they are all together and so might take each others presence for granted (more in the case of the kids, cause Sakura most definitely does not after HF happened).
Cause trust me, kids take their parents for granted if they group up in a happy family, and when they are separated due to something outside their power to control it makes them realize just what they have.. Trust me, I talk from personal experience in this matter, and it will make them nothing if not stronger.
It would make the family bonds stronger and ever the more potent.