Birth registration index shows:
Austin Bernard G Fane born in the March Quarter of 1883 in Woburn district Volume 3b Page 411 This is the info needed to obtain the birth certificate, but narrows down within about 4 months when he was born. Woburn is in the county of Befordshire.
I didn't find William Walter, as the Free BMD isn't complete. I didn't look at the Ancestry's complete BMD index, but it's free to all after registeringhttp://www.ancestry.co.uk/search/rectype/vital/freebmd/bmd.a...
Hopefully you'll have some luck there looking for his birth.
The 1891 census finds the following family in Kent with Austin's mother re-married.
Charles Fishburne 30 Farmer
Mary Fishburne 31
Margaretta Fishburne 13
Frederic J Fishburne 10 ½
Christina Fane 10
Austin B G Fane 8 born Toddington, Bedfordshire
Louis R Fane 6
Morgan W E Fane 5
Wilfred G Fishburne 1
If you go to the LDS website you'll find Mary with husband William Fane and the first two daughters on the 1881 census in Toddington, Bedfordshire.
Back to the Free BMD shows the following marriage
1888 Sept. Qtr. Woburn 3b 633
Charles James Fishburne
That should get you started with lots to follow up with.