FWIW - On the North shore you have a number of family and fun places to go including but not limited to: Pt Washington, Glen Cove, Oyster Bay, Huntington, Northport, Pt Jeff, Mt Sinai, Mattituck (now Strongs marine), Greenport. And on the Ct side there are even more stops with both family and fun places including bit not limited to: Norwalk, Black Rock, Cedar Island, Westbrook, Ct River, Thames, Mystic, Watch hill and then spots in RI. Some areas like the Ct. River and Mystic easily have enough spots and things to do to take up a full week without becoming old. Certainly worth looking at all the destinations and possibilities before weighing in on the choice. I have boated on each shore and this is just one opinion.
Hope this helps