Finding very little, but finding some records.
Seems you are a little off of the dates, but his is common for years in the early 1800's.
So far am not finding any VA records of either Charles, Henry or other family members.
The earliest record is an 1860 Groton, New London CT census.
All census are of this town.

1860
Henry S. 40 1820 ct
Jane 32 (1st wife)
Charles 12 1848-va
Sarah E 10
john 9
William 6
Ella 4
Fanny 2

1870
Henry S 50
Virginia 42
Charles A 22 - va
John D 19
Wm A 16
Ella 14
Fanny 12
Frederick 10
Mary 7

1880
Henry S 60 (stone cutter)
Va 51
John D 29
Ella 24
Fanny M 22
Frederick 20
Mary E 17

I did find a family tree with some info. on Henry with no documentation to back up, but will pass along.
Henry S. Gilbert b.feb 5, 1820
married 1845 petersburg va
died july 18099
parents John Cooley Gilbert and Fanny Smith
wife- Va Ann Wells

Now, the other record I found is a mortailty record for your Charles Gilbert in New Orleans LSA showing he died at the age of 32 of cancer. His occupation is listed as a laboer.

Sorry I could not find more for you so far.