I think that the Robert Mann who ended up in Mannboro originated in Chesterfield County, moving to Amelia due either a land grant or a large, cheap land purchase. Possibly the same generation as your 1732 guy, or possibly a generation later.
You might try looking for him on www.virginians.com
My ancestor Watkins Turner (1800-1863) married Judith Mann in 1830, and I think that her farther was a Robert Mann. She last appears on a deed in 1857, and Turner was remarried by the time of the 1860 Census.
I don't have much info on Judith A. Mann, because there were no children from this marriage and she is not actually my ancestor. The marriage appears via a bond posted in Amelia County in 1830.
Hope this helps,