What do I mean?

Here is one of my all-time favorite regional differences in meaning. I pose this question to you, my friends. If you ask me to go somewhere with you, and I say, "I don't care to." Do you think I'm going or not. The answer and action that followed really stumped/confused/puzzled my in-laws when they moved here from New Jersey about 17 years ago.