The 1861 census of vienna , Bayham Twp has
Joseph born Upper Can 1812
Jane C wife us 1828 , some places say 1826
Wallis E. 1853, they must mean Walter
and Wilmot 1856
This is obviously the right Joseph.
Now in 1870 , he is living back in Middleton . not head of the household. It appears that Jane died beween 1861 and 1870 . I am guessing in Vienna. I cannot locate her place of burial , it is possible she is buried in Courtland
without a stone or one that is legible . It seems that Joseph moved back to Middleton. I will look some more.