Although it hasn't been updated for a few years, you'll find some information about Queen Victoria's baptism at "Royal Christenings" at
http://mypage.uniserve.ca/~canyon/christenings.htm British royal christenings (like most christenings) are recorded in sacramental registers, which I don't believe have been digitized for online viewing. But if you could see Queen Victoria's original christening record, you'd find it in the "Chapel Royal Register Book", sometimes known as the "Royal Register". As other posters have noted, QV was christened at Kensington Palace, but the event was recorded in the register book of the Chapel Royal at St James's Palace, London.
The best you can probably do is to read about QV's christening in *The Times* of June 25, 1819, and in various biographies dedicated to the Queen.