Very interesting rumor. There is a similar rumor in my Raleigh family (from Maine as far back as I can trace) that there were 4 Raleigh brothers who left Maine about the time of the Civil War and acted as privateers, mostly hiring out to the South as blocade runners.
I'll give you my tree as far as I know it, maybe some names will ring a bell with you. I am Harry S. Raleigh, Jr. My father born1915 in Cape Elizabeth Me. His father Alonzo L. Raleigh born 1893, Lubec, Maine. His father Richard L. Raleigh born St. George, Me., married Lena Morong and had 8 children, Earl Le Roy, Richard, Clarles, Laura, Alice, Mary, Etta and Alonzo.
If the rumor has any substance those 4 Raleigh brothers were probably the generation before my g-g'father, Richard L. Raleigh.
This common rumor is very interesting. Almost too much for coincidence.
Good luck. Let me know if you find any connection.