I have personally experienced harassment first hand when it comes to urban photography. In DC, one routinely gets threatened or chased off in the spirit of 'national security.' So it was with great joy that I came across this entry over at Stu Maschwitz blog. I also found another helpful post at John Nack's prolific blog on the same topic.
You can find a great PDF here that describes the rights of photographers (and videographers) within the U.S. Print this out and place it into your gear bag for the next time you get harassed.