Shultz was arrested Thursday night at a Lower Manhattan hotel with two loaded firearms.
Police were called when someone saw him take a gun from his pocket.
The 39-year-old appeared to be intoxicated, police said.
Matt Shultz of Cage the ElephantTim Mosenfelder/Getty Images
Watch on Deadline
Cage the Elephant is known for the 2015 albumTell Me Im Pretty.
Shultzs attorney said in court the guns were purchased legally and registered, but not in New York.
The singer is scheduled to appear in court on Feb. 22.
