Most likely the family left Trego County in the late 1880's or early 1890's. These were tough economic times in Trego, Western Kansas, and Kansas in general. The population may have dropped by up to 50% in the county in this era.
I did a quick check of the Trego and Kansas 1895 census and did not find the Madison's that you named. You can also check this by going online to www.kshs.org
. Go to the Genealogist link and then click on census. There you will find an index of the 1895 KS census. Look for your Madisons.
As may already know, the bulk of the 1890 Federal census burned in Washington. As for the Trego 1885 Kansas census, I've looked at that for my own research purposes and only the Collyer Township records remain. The Collyer Township was in the western 3rd of the county. Apparently the Glencoe records are lost.
Check out that 1895 online record to see if you can find them through the Kansas State Historical Society database.