When you post, post the person(s) full name and DOB, that helps volunteers to find records on someone. Someone might recognize his name and being a pastor, but not having known him in the area you describe. He might have served at many other church locations. I tried to find an O.S. Brewer, without success. Just looking for a "Brewer," in Texas resulted in thousands of names and news articles.
You said you found a picture of a building...was it in your grandfather's possessions or where did you find it?
The First Baptist Church in Quanah:http://fbcquanah.org/AboutUs
Call them and ask. They "might" have a copy of the reunion records.
There is also a First Baptist in Mangum.