Newport, Block Island, Montauk, Watch Hill are all great places. I anchor most weekends in Watch Hill, RI. (Napatree) right on the CT, RI line. We did two weeks this summer (Napatree, Block Island, Newport, Edgartown (Martha's Vineyard), Hyannis). It was great.
You can head further west and go to Sag Harbor and Greenport as well.
Let me know if you decide to head west, i'll met up with you.