One branch of my family dropped the Von. Carl VonBergen was born in 1875 in Switzerland. At least one of his sons, also named Carl (b. 1913), dropped the Von and moved to the Detroit area in the 1930's. Another son was Ernie. I don't know much about him.
The family were members of the Apostolic Christian Church. As anabaptists (like the Amish and Mennonites), they suffered persecution in Switzerland. I don't believe they were "thrown out", but religious freedom was part of the reason they left. In the early part of the 20th century this church had several schisms in America. There was much bitterness and a lot of "shunning" going on.
I would love to hear from you if you think your family is related to mine.