I am still working on parts of the same family. Eli's brother Henry E born in 1850 (or is Henry E really Eli?) was married to a gggaunt in the Justice family tree, she was named Olive Louise Justice Gardner (part of the mystery - her father appears to be a Gardner but she was raised by a Justice). Henry would have died before 1890, he was listed as the lighthouse tender in Ashland in 1888-1889. Thereafter she was listed as a widow. His family took the name Genery from that point forward it seems.
Here is my tree linkhttp://trees.ancestry.com/tree/22610099/person/1278075939
Will keep my eyes open for Eli if you can point me to your tree