There could be a rather simple explanation. I have discovered this while researching my own family. There were times when financially, the parents couldn't afford to feed all of their children, along with themselves. Also, perhaps both parents were working, and there was no "childcare" available. There were facilities that essentially took these children in, and fed and clothed them. You can sometimes discover the children listed as "inmates" in these homes. Then, when the financial situation became better, the children were returned to their parents - sometimes, just in time for the census.
