The entrance is the tricky part. There is a breakwater that is submerged at high tide and there is at least one shallow spot after you swing around and enter. Inaddition, the water literally rushes in or out depending on how the tide is running. To get in, you need to run straight at the beach (there was a private marker there) and make a sharp right running close to the beach. Once inside, there is plently of water to anchor and there are many big boats that go in there.
I personally don't go in there anymore due to the bent prop stories and toll on my nerves. An alternative would be to go around Centre Island into Oyster Bay. There is an anchorage off Bayville that is very nice and suitable for swimming. That is where we go now.
My advice if you really want to go there would be to follow someone else in and be mindful of the breakwater and the tide.