Could your Asa J Scott from Franklin County, GA be the following?
1850 - District 32, Franklin, GA; Asa J is living in home of Robert (b 1783, VA) and Mary Scott (b. 1789, GA), presumably his parents.
1860 - Franklin, GA Asa J is living with wife Sarah and Children Mary and Benjamin
1870 - Franlin , GA Asa J is living with Sarah, Mary, Benjamin, Jeptha, and John. Next door to Benjamin and Elizabeth Stonecypher,(both 85)
1900 - Shoal Creek, Hart, GA Asa J is a widower living with daughter Mary A Smith
I show Asa J Scott and Sarah Stonecypher being married on 29 Jan 1851 in Franlkin Co., GA.
Robert Scott, Father of Asa J Scott (b 1829) is the brother of my Thomas Scott who died in 1849 and lived next door to Robert for many years.