Any office that is open to the public can be visited in person. Call for their summer hours, as they may change from the winter. And the main library is literally just across the river (a 2 minute walk) from all those court records.
I used to work there (in the records building) but that was years and years ago. We had to search things by hand. Now I am told that they are computer searchable but NOT from the outside. The Roch Municipal Archives has a great collection of old police books (and the library has the police scrapbooks in the basement, which can be brought up for examination.)
So, yes, all the offices are available for a public visit. It's worth a call to find out the hours and fees, e..g. the Surrogate's Court records are free to search if you do it, and 90 (yes, ninety) dollars if they do it.