The best place to orient yourself is Water Sq, at the east end of Duke St. Named for an old circular stone reservoir dating from 1798, the square (actually a triangle) has a fountain topped by an old waterwheel. Today it forms a traffic roundabout, but back in the day this fountain pumped fresh water before New York City had any such luxury.
Many of the wooden shop-fronts in this area are attractively disheveled relics.