The reason I think there is a possibility you may have some knowledge of my ancestor is because where my great-grandmother was born is not far from the Canadian border (Skowhegan, Maine).
My great-great-great grandmother, Betsey C. Joy, married another man, Israel Folsom, but was somehow involved with if not married to a James Hughes before that and had two sons by him. One of these sons was named James Hughes and was born in 1820.
If you see the Lathrop Family tree that we started you'll see how it all plays out. We could be barking up the wrong tree too tho. Finding records from that day is a problem with some regions.