This drowning of September 1856 seems to fit the scenario of the folk tale better than the September 1857 drowning mentioned in the posting.

Middlebury Register
September 24 1856
DROWNED. Four men in the employ of the West Castleton Slate Company attempted to cross Castleton Pond in a boat and when within a hundred yards of the shore they sought to reach, the boat capsized and two of the number, an Englishman and an Irishman were drowned.