Bannerman's Island 

In the Hudson River, between Newburgh and Beacon, Bannerman's Island, or Pollepel Island, sits as skeletal remains of a once impressive Scottish-style castle.  The Bannerman Castle Trust been trying to get it rebuilt, retrofitted, get it ready for people, etc, but it's a fairly costly endeavor.  They do have guided tours these days, so you can check it out.  It's definitely a nice place to visit, something interesting and unique to do.  Here are some photos I was able to snap while visiting there a few years ago.

It's definitely not the safest visit. Here is the same arch, looking from the river (left) and looking at the river (right). What you don't see, is the 10 foot drop resting right in front of the archway, easily fallen into.

Here's the main warehouse building from the inside. Yep, what you see from the outside (below) is simply a shell, the inside holds nothing.  Well, piles of brick and steel, littered with beer cans and McDonald's wrappers.

Up on a hill, away from the "castle", there's a smaller house where people lived (below)