I'm not related to the Sawyers but I can shed some light on the family.
Camden County Deed Book Z pages 380 and 381 have a deed division in 1852. The heirs of Abner D. Sawyer are listed as James (adult), Penelope (adult), William, Evin and Fany (all infants). I know this disagrees with the census which lists William as an adult, but it could just be a mistake on the part of the transcriber. I got the data from "Kinsfolk of Camden County Deed Books Vol 1" by Elizabeth Jones