If you are really set on making a day trip to NYC you could always stay somewhere in Connecticut. That way you are central to the three places you want to go. Just be prepared to do some driving. I went to school in Newport and it is one of my favorite places in the world Brick Alley is a fun restaurant to go to as a previous poster said - but the Black Pearl is one of my favorites.

If you are going to stay in Newport, there are a million (well, maybe not a million...) bed and breakfasts to choose from but one doesn't really jump out at me.