The laws regarding that probably vary by municipality. Local ordinances might be a good reference, or ask law enforcement if possible, they should know what they would do if they come across someone doing such things.
A place to go to the bathroom? (clean, not at a gas station )
Getting permission from the groundskeeper or caretaker could smooth things out. I think cemeteries are a weird cross between public and private (unless explicitly marked private), so I don’t know how much they can tell you that you can’t do (unless sign are posted), just what the law says and what their preferences are for “good taste”. Some people might consider a Rock video in a cemetery disrespectful or offensive, especially if it’s visiting relatives and they see/hear you. Certainly don’t prance around on top of the gravestones or otherwise disturb the premises, just get good shots from a respectful distance. I would guess that it’s not good form to feature the names on the gravestones either (unless it were someone you knew personally?). While there’s probably no legal issue, again it could be a respect thing. I have taken personal pictures in a cemetery, including unrelated headstones around family members’ headstones (scenery was nice). I didn’t think about it at the time, but I think that’s different than putting a video of it on YouTube.
You might consider a small old cemetery out in the country, if you can find one. Likely nobody would be around (unless the caretaker lives next door), and the vibe could be cooler and creepier than one that is kept up, popular/busy, and with well-manicured lawn and landscape. I don’t know what your song subject is, but it sounds like old, isolated, cobwebby, creepy, scary, menacing is what you’re after. Something to do with Halloween coming up?
That’s one way to get around it. You could mention that if asked, too.