It would be helpful to know what information you currently have.
When did your ggrandfather die and what name was he buried under?
Which censuses have you found him in and what name/age was he listed as?
Did he ever become a U.S. citizen?
Where did the second marriage take place?
The best approach is to start from what you know and work your way back. Right now, I can't tell what information you have and the source of that information.