Thanks for the reply back.
With help from others, I found a book "A Genealogy of the Descendants of John May who came from England to Roxbury in America 1640". On page 89, the book shows a Nancy and Clarissa (sisters and daughters of Edward Selden and Sibyl May) both married to Thayers that lived in Palmyra, NY. This has to be the twins, Joel and Levi. That same page also shows Levi Thayer from Palmyra, married to Elizabeth Selden daughter of David Selden and Cynthia May (cousins of Nancy and Clarissa). Cynthia and Sibyl May were sisters and daughters of Eleazer May and Sibyl Huntington.
The Memorial of the Thayer name: from the Massachusetts colony of Weymouth and ... By Bezaleel Thayer, books shows Levi Thayer married to both Clarissa and Elizabeth (after Clarissa's death) and Joel Thayer married to Nancy Fuller Selden.
So I think I finally knocked down my brick wall, with lots of help.