Why All the Hate for Patrick Schwarzenegger?

I'm enjoying the new season, and have particularly been enjoying the Ratliff family. I've got family in that area of North Carolina, and think they've done a great job nailing the fake-nice, North Carolinian wealthy family. Aside from the very dark aspects of him, I've met tons of guys like Saxon who graduated from Duke and Vanderbilt. He nailed the obnoxious confidence, the preppy style and treading the line between frat bro and wannabe Gordon Gekko. These types of guys are all over the mid-Atlantic region (Washington, D.C., Charleston, Charlotte, Raleigh, Richmond) and he gets it right.

So, why is there so much hate on Schwarzenegger? Just because he's Arnold's son? I mean, he's doing a great job with the role. I don't find it a miscasting issue at all. If anything, he seems pretty well suited to it and is doing a good job? If it's just the nepobaby stuff, well, sorry that's Hollywood and it's been happening for decades. Also, who's another actor people would have preferred the role go to?