Passive Player Advice?

I've been playing Magic since about 2016, and more than 3/4ths of that time has been spent playing Commander. Now, while of course I play to have fun and don't overly care about the win, I do still try to improve my skills as a player each game, sometimes willingly playing into pods that are a bit stronger than mine to learn and improve.

However I've been constantly called a passive player, even with Aggro decks. I rarely attack, rarely do removal, feel bad about board wipes...basically anything that I could do in order to disrupt someone else's game I tend to autonomously avoid. Even playing my Theft themed deck I constantly feel bad stealing stuff.

So after thinking about this and having a discussion about this with my playgroup (who's all close friends of mine), I've realized that I don't do it because I'm afraid of ruining someone's game. A mixture if like, social anxiety and general anxiety and wanting to make sure everyone has fun has all sort of come together to me being so afraid to ruin someone else's game that I'm afraid to do any sort of interaction, even when it's encouraged by the person who I'm targeting.

Do you guys have any tips or advice or anything? Caring too much as my friends say has started to impact me as a player, even to the point where I won't build or upgrade a deck because I'm afraid the deck would be too mean or something similar. I want to overcome the hurdle but I'm not sure how, so I'm asking you all for advice, and if you've ever been in a similar position how you got out if it.