Wow, thanks for this. It confirms a lot I had already discovered.
Richard Henry Sandoe married Evelina E Stanaway on the 26th February 1885 ( I have the marriage certificate here ) so yes, Joseph Henry was born out of wedlock in 1894. Evelina was in service to a William and Sarah Warmack by the time of the 1891 census, this could have been when she met Richard?
So she married Richard on 26/02/1895 and by October 1895 was in NSW, Australia, not sure what went wrong there, seems strange to marry and be pregnant and then make a trip to the other side of the world without your husband..........and she would have been pregnant with Elsie Sandoe at this time.

Richard went on to marry Elizabeth Emily Burlace in 1907 and had 3 children - Evelina Annie in 1900, Violet Emily in 1901 and Lilian in 1903, who died at about a month old. So these 3 girls were born out of marriage too. Violet Emily was my grandmother.