Ah ha! I mixed up two John WILLIAMS family's. I should read records when I'm under the weather, but I'm not up to much else. lol

In the 1871 census there is a family that fits AND they have a child of William HALL and Mary SMITH living with them.
Still find it a bit odd that in the 1861 census Mary is living beside the GERMAN John WILLIAMS and wife Mary and in 1871 two of her kids is with the ENGLISH John WILLIAMS.

1871 census for Artemesia, Grey Co.
John Williams, male, age 31, born in England, married
Eliza Williams, female, age 28, born in Ireland, married
Anna Williams, female, age 4, born in Ontario
Harriet Williams, female, age 11/12
William Hall, male, age 19, born in Ontario, Irish, single
Maria Hall, female, age 16, born in Ontario, Irish, single

By the age of Eliza in the census she is more likely a daughter of one of the Smith children, then a sibling.