We go there on day trips from Kingston. Its a nice little place to hang out, do some fishing, check out the scenery, BBQ overlooking the River and watch the sunsets, and yes, camp in a cabin if you are so inclined. My son learned how to ride his bike there last year - we carried it onboard the boat and the two of us camped out on the boat at night and rode bikes during the day while he got the hang of it. We left the wife home for that one - I didn't want her to see his bloody shins!!!
The town of Staatsburg is near the park and is a nice bike ride away, as is the Mills Mansion. One word of caution - the whole damn town is CLOSED on Mondays (yes we found out the hard way)...
As for Norrie itself - its a rather no-frills marina. There are restrooms, showers, and ice. Think campground as opposed to a true marina and you'll know what you're in for. But if you're used to all the hustle and bustle of places like Kingston or Haverstraw, Norrie is a refreshing change of pace...