Are your years accurate?
If he was born in 1854 and you last found him in 1861, he would have been 7 years old. Is this correct?
Where did you get the information on his wife and children?
If you can narrow down the "at some point" emigration, it might help someone find the records for you.