On the 1870 census for Lee co. MS. there is a Joel Allen who was married to a Sarah with children William and Joel 1 year old. It says he was born in SC. and right below Joel is a John and Martha Allen around 60 years of age. John was also born in SC. Could this be your Allens.
in 1850 Joel Allen is living with Parents John and Martha Allen in Pontotoc MS. the age is off by a few years which is not that strange for the census. it give John and Martha and Joels place of birth as SC with the rest of his siblings in ALABAMA and MS.

Had one daughter died in Ellis Texas in 1919 and it also gave his place of birth as SC.

had two sons named Dock allen who died in 1973 Ellis Texas
and Lester Earnest who died in 1969 Ellis Texas.

Hope this helps.
Sue