It is NOT just the number of people in your file but also all of the sources, facts, and etc. that are included.
I have been running 2012 since it came out and have NOT had issues with my larger files, but I also do NOT add a lot of linked sources, media. I also have better than mid range systems, with lots of RAM and disk space. I use a high speed broad brand internet connection, which all helps.
I also do NOT use any of the REAL heavy AV programs and do not do a lot of IM type of programs.
I have been know to have up to 6 IE windows open do various search sites while running FTM.
When I get a new copy of FTM, I always play with for a while before doing "production" work. I have found that the more "merging" I do the slower the program gets. In the real (production) world I do NOT merge as I want to control what is being added to my data.