One thing to keep in mind, is that in a 20 foot bowrider, it won't take much to make you wish you had a place to ditch off.
Plan for the worst, and hope for the best. I have seen steep 4 footers, and nothing more than a ripple in the course of less than an hour on the upper bay.
If it were me, and time allowed, I would plan to make it a two day trip, stopping off somewhere along the way to relax and enjoy your surroundings. There is likely, not a more beautiful place to boat on the East coast, than the upper Chesapeake Bay (But then again, I'm a bit biased.)