I had read many postings and the project discussion page on this, then I wrote support for clarification. This is what they wrote back -
"Thank you for contacting Ancestry.com regarding who Self is on the Boston, Missing Immigrants project.
On the Boston, Missing Immigrants project, the Self is the missing immigrant that is listed. Everyone in each of these should be showing their relationship to the missing immigrant. On the image that you sent of John, William and Peter Buckely, the three of them will be Self as the record is about all three of them and then Cornelius will be Father and Ellen will be Mother and Mary and Honora are the sisters.
On this project, anyone that the record is for, so those that it says Of... you will want to key those in as Self and the rest, simply key in with their relationship to the person or people the record is for."
It makes this part of the project more clear for me and hope it helps others.