The phrase "extended family" in the title of this report is descriptive of who will be included in the report, ie the "extended family" (eg peripheral relatives), of the selected person.
1) The first thing to take a look at is what is your choice to the option in the right panel for: "Include everybody in this file?".
You want to uncheck that.
2) Once that is unchecked, the report is designed to include everyone related to the selected person, by blood or marriage. In-law families are shown in separate tree arrangements and "connected" back to the "blood" tree by a number in the upper left of the connecting spouse.
So, the second step is to detach your spouse's family. I would suggest you select her, then Main Menu > Attach/Detach > Detach her from her family where she is showing as a child with her parents.
3) Now select the nephew and run the report. Review it.
If there are cousin marriages connecting to the mother's side at earlier generations, that group of people may still be included in the report. I have one person in my file who is a multi-generational Long Islander. He is a descendant by nine different lines of descent from a single Long Island immigrant. If he marries another native Long Islander with roots back to the same small group of Long Island immigrants, this technique may not work to separate the two sides.)
4) Now, don't forget to re-attach your wife back to her parents!
It sounds like you are not reflecting step-children as such in FTM. I agree with you on that approach. I suggest to not include step-children as children of step parents in FTM because it creates all kinds of reporting problems in terms of reflecting step ancestors (if a step-parent is selected as preferred), step-cousins, step-descendants, and all kinds of non-blood relationships popping up that are not relevant genealogical connections. I prefer to give step-kids their own separate reports of their own BLOOD lines.