Just happened on your message. I was there over the same time period you were. I came there in August of 1948 and was adopted in September of 1951. I stopped by there two years ago. They told me that the original building is gone. They also told me about how to contact the State of Wisconsin about adoption information and I was actually able to track down my birth mother although I was a bit too late.
Good luck with your search. It was well worth it in my case. I was able to answer a lot of questions and I'm glad I did it.