I found a census from 1930 (minnesota) where a George Elmer matched the age of my ancestor. He is listed as an inmate with a long list of other men. He was born in 1863. He had 2 sons, Willard born 1890 (who was 9 in 1900, living w/ his maternal grandparents, and a younger brother Asa born 1893. His wife Cora and son Asa were not living with Coras parents, so I found that odd. I also know from my parents that Asa grew up and had a family (and it is backed up on ancestry.com) so it may possibly not be my "George" but the circumstances point to something strange going on between 1895 (where George, Cora, Willard and Asa were living together) to 1900 where Willard was living with Cora's parents.