The hard answer is yes, let go and settle if your values won't lead to action. I value fairness highly but often it leads to conflict, and I'm often disappointed with the state of the world and how people treat each other, where there's little to nothing I can do to change it.
If you want to avoid those negative feelings, you'll need to acknowledge what's outside of your control, and when it's coming from a single person, try not to take things personally. Everyone is coming from a different set of life experiences and values, but we each see the world only through our own. Some people make very selfish decisions, but it doesn't make them bad or wrong from their perspective. If we judge them only by our values then that's not really fair either.