Before 1 Jan 1970 you had to be 21 to vote, so the short answer is it would be the father, and the long answer would be it depends.
The Great Reform Act of 1832 extended the right to vote, but not by much. The Act gave the vote in towns only to men who occupied property with an annual value of £10, which excluded six adult males out of seven from the voting process.