You can carry certain knives on you. A folding pocket knife, for instance.

Also, a gun (though I don't know if people can just carry guns on them in NYC).

Though, someone correct me if I'm wrong on this, but I've heard that New York isn't a "stand your ground" state. Meaning if you seriously injure someone while defending yourself you could possibly be charged or sued?