Thank you again ! I’ve dug out my old notes on Milnes in Moray and found some of the names mentioned. This does look like a promising lead. Sometimes a fresh pair of eyes can help you see better.
A quick scan on IGI finds the offspring of James Milne and Charlotte Masson. 8 are listed. The 1851 Census for Aboyne has an Ann, but only 5 years old. That seems just a few years too many out. There is no Eliza or Ann in the list of offspring from IGI either but that is not uncommon. 1861 sees the family in Rothes with James staying with his son Andrew in Old Street perhaps helping out with the new born child and Charlotte in Breich Street. There is an Elspet Milne in Elgin in 1861 working as a servant for a Masson family ! In 1871 James is by himself in Breich Street and Charlotte appears to be visiting her son, Alexander, in Birse in Aberdeenshire.
These will help my searches but I fear Elspeth is still going to be hard to track down. Here is the extract from the Kirk Session records
Compeared Elspet Milne residing presently with her brother in Breich Street confessing that about 3 weeks ago she had brought forth a child in fornication and declaring the father of it to be George Inverarity, Muirhead, Leochel Cushnie. Being asked she stated that the guilt was committed in Leochel Cushnie when she was a servant at Muirhead. An extract of this minute was appointed to be sent to the Kirk Session of Leochel Cushnie that they may take such steps in the discipline of this chiel as they may judge best and the said Elspet Milne was dismissed for the present with a suitable exhortation.
Unfortunately the Kirk Session records for Leochel Cushnie for the period have not survived so I can’t follow up that lead.
Her brothers seem to have been James, Andrew, Alexander and John. I think that will be my next line of attack. I do have a possible age now and perhaps her parents will narrow down the number of marriages I have to examine.
Thanks again for your help !