I’ve heard some complaints about california drivers going “slow” (i.e. the speed limit) in the fast lane because its a “passing lane”. That is the case in some states, but not in California. Here’s the handbook on use of lanes:
It recommends not using the left lane, but that’s a suggestion for smoother driving, not a requirement.
Passing lanes are short lanes that open up on one lane roads to allow you to get past someone. Either a broken yellow, or a second lane that appears for a short distance and then goes away. In what California calls a passing lane, sure, I think we should string people up for not staying to the right and taking turnouts. But otherwise, suck it up. Go around them on the right if you can.