I lost both sets of exhaust shutters last year. I found two of the four pieces lodged in the exhaust bellows behind the shift shaft. I had to remove the lower unit and look up the exhaust to find them. I never found the other two.
You may want to tap on your exhaust bellows, or look down your prop.
The only way they would cause a problem is if they restrict exhaust. You would notice this at higher rpms. If your engine is less responsive over 3000 rpm or so, it could be your exhaust shutters.
2006 Bayliner BR205
A1 Gen 2 1.47
5.0 carb engine