Thank you very much Darlene!
That has to be him, can't find anything else even remotely close. So Mary F. Ellis must be his mother, and 2 brothers and 2 sisters. I was thinking maybe his father had died, but he wasn't in any of the census. He wasn't even in a 1880 census that I could find, and the children were being born around that time. And I thought they were in the 1910 census but they aren't in a census until 1930 Fulton Co. What a puzzle!! Thanks again