There is a handwritten note on the back of the painting. It does not quite synch with some other details I have read, but here it is:
"Mrs. Hester Middleton
Painted in 1840
She was daughter of the Earl of Swift. She eloped with the coachman Wm. Middleton, and came to America about 85 years ago.
This was information given me by my mother shortly before her death about 1940.
Hazel Larkin Meissner"