I can tell you that not all of the Waite family disappeared after the 1850 census. I found Sarah and two of the boys on the 1860 census for Morehouse Parish. The name was "Wait". Sarah, LN [1817 SC] and JN [1836 AL] were in the household of JG Harrison [1828 AL]. Also in the household were BA Harrison [1830 AL] and JF Harrison [1837 AL]. I found the three Harrison boys on the 1850 census with their parents. The Harrison family was in dwelling #89 while the Waite family was in dwelling #92. I feel sure that the two families were related. Harrison and Thomas were of about the same age. It may be that they were first cousins. Harrison's mother could have been a Harrison before her marriage, thus she would have given her son the name of Harrison. Just a thought...
JG was James G, BA was Benjamin A and JF was John F. I also found that the Thomas Harrison family was in Wilcox Co., AL. It is likely that both families moved from SC together and then moved from AL together.
I was able to find James G [JG] Harrison on the 1870 and 1880 census.
Harrison Waite likely died between 1850 and 1860. Perhaps some of the kids did, too.
The only Waite in SC on the 1820 census was a John Waite who was likely the father of Harrison.
The "extra people" you asked about were likely children who died before the next census, so you would not know their names.
Hope this info helps in your research.