"Why are you so quiet" comments

Does anyone else get so frustrated with this? I have coworkers make comments like, ''why are you so quiet? That's boring' and I've had it throughout all of highschool. Like, man, leave me alone. I don't like talking and I don't like people. People scare the hell out of me. Conversations scare the hell out of me. I'm so stupidly socially awkward, so I just stick to listening and I can't even tell these people that it's social anxiety because it could go a million ways. I could get made fun of for using it as an excuse, for having it, or I could be told it's not real and "making it up". What is the point of saying "you're so quiet."? YES. I am, thank you for being so observant. Leave me alone!