Police said the suspect was Mr Williams, who is believed to be homeless.
NYPD did not say on Friday what charges the suspect could face.
In a statement after the incident, Buscemi’s publicist said the actor was “another victim” of a random act of violence in the city.
“He is OK and appreciates everyone’s well wishes, though incredibly sad for everyone that this has happened to while also walking the streets of New York,” the publicist said.
There have been reports of random attacks in the city and hundreds of National Guard troops were deployed in March to the subway to address high-profile incidents of violence.
The NYPD said this week that felony assaults in the city were up 15% year-to-date, and misdemeanour assaults had increased by 7%.
However, official figures suggest overall crime, external in New York City has fallen year on year.
Buscemi is a New Yorker who once worked as a firefighter. His portrayal of a corrupt Atlantic City politician leading a double life during Prohibition in the hit HBO series Boardwalk Empire won him a Golden Globe award.