The Merc prop selector/calculator might be a good place to start:
Another way to look at it would be the Max RPM range of the motor. I believe the Max RPM range for a 3.0L (125-140HP) would be between 4400-4800 RPM's at Wide Open Throttle (WOT).
First i'd look at where my WOT actually is, when properly trimmed. You might want to do this with 1-2 people, 1/2 fuel, limited gear. Then compare to when the boat is loaded. 5 adults will be roughly 1,000lbs +/- It might not be teh prop, but rather the weight of people and gear. 5 people rarely go out without loading down coolers, bags, etc. You may simply not be able to operate at high speeds with this much load. You could go down in pitch, which would affect top end, but would get you and the load up on plane. The question is, how many times will I need to run with that much weight, because you'd risk being under-propped with less weight and it simply may not be worth it.
Think of it this way. A Honda Accord will cruise nicely along the highway, but not if you try to tow a boat with it.