Without knowing where in the world you and your family are from, it's difficult to give specific advice, but this is the first thing which springs to mind - is your grandmother still alive, because she might be able to give you very useful pointers.
For example, it would be helpful to know when your grandfather changed his name, i.e. was it before or after he married your grandmother? Probably stating the obvious, but does your grandmother know what his name was before he changed it, and why he changed it?
I apologise for any upset if your grandmother has since passed on, and in this case, it might be worth asking her children, or any of her remaining brothers and sisters if she had any, so that you have more of a frame of reference with which to work backwards.
I hope this helps a little. :-)