1881 cneus - you may be missing at least one child.
Charles DOWNING M Male English 55 England Labourer Church of England
Sarah DOWNING M Female English 50 England Church of England
Thomas H. DOWNING Male English 21 Ontario Labourer Church of England
Mary Ann DOWNING Female English 17 Ontario Church of England
Elizabeth DOWNING Female English 14 Ontario Church of England
Charles DOWNING Male English 14 Ontario Church of England
George DOWNING Male English 10 Ontario Church of England
Source Information:
Census Place Pickering, Ontario South, Ontario
Family History Library Film 1375880
NA Film Number C-13244
District 132
Sub-district A
Division 2
Page Number 34
Household Number 170

Here's the marriage for Eliz. Downing;
12717-95 Frederick Joseph BOWMAN, 26, clerk, Berlin, same, s/o William Henry & Alice, married Elizabeth DOWNING, 27, Pickering, same, d/o Charles & Sarah, witn: Clara A. COX of Toronto & E. S. HOUSTON of Niagara Falls, 21 Aug 1895 at Niagara Falls

This may be them in the 1901 census
2 23 13 Rasberry Wm W M Head M Sep 12 1869 32
2 24 13 Rasberry Maria M F Wife M Oct 28 1869 32
2 25 13 Rasberry Winnifred F Daughter S Sep 26 1897 4
2 26 13 BOWMAN Fred M Lodger M Jun 6 1870 31
2 27 13 BOWMAN Elizabeth F Lodger M Feb 2 1870 30
2 28 13 BOWMAN Winifred F Lodger S Jan 5 1898 3
2 29 13 Simpson J W M Lodger S Dec 25 1876 24
1901 Census of Canada
Subdistrict: Ward/Quartier No. 5, HAMILTON (City/Cité), ONTARIO
District Number: 69
Subdistrict Number: e-5
Archives Microfilm: T-6471