OK, so that's a lead! Wikipedia indicates, "Built originally as a tavern and inn, it was converted for use as an academy in 1846. It was used as a school from 1847 to 1861, at which time it became a private residence." So, the years of operating as a school fit my theory. I have driven by the Academy many times, it is very close to my hometown of Bergen, NY.
Can you tell me anything further about the Riga Academy, or a resource for more information? Googling around suggests that it enrolled both males and females. Thomas would have needed to meet his future wife somewhere, Minerva Collins, who is found in the 1850 US Census for Gates. Is there a source with enrollment records?