Is it rude to not answer someone's question when they're intentionally trying to upset you with what they ask?

Turn your head and leave. It's more of a crime to put political correctness above your own internal wisdom. Tell this person to f-off and then say your going to grab them by the ear if they speak to you that way again.