One the west side of the river route 9W is pretty scenic. Places of interest include Bear Mountain, WestPoint Military Academy
The bit of road on the east side from the Bear Mountain Bridge south to Peekskill is quite scenic.
The Hyde Park area has the FDR home and a Vanderbilt mansion.
The town of Beacon is cute and has the DIA art museum.
Closer to Tarrytown there's Kykuit (a Rockefeller estate with a great art collection) and a collection of other historic homes.
Saugerties and New Paltz are other nice towns on the west side of the river.
New Paltz has the oldest street in the country (Huguenot Street) and you can tour some of the buildings as well as do a self-guided walk.Edited: 23 February 2017, 11:00