I finally bit the bullet and upgraded my graphics card from a RX 590 to a RX 6750 XT, and holy shit is there a big noticeable difference. I've been replaying some games and its almost like playing it for the first time. It just feels so different and so much more details to notice. I thought I was fine on the 590 and that I really wouldn't see much difference, but the 6750 xt is running Red Dead Redemption 2 on ultra and still has plenty to spare. My room is also not increasing in temperature due to the 590 maxing out the entire time trying to play RDR2 on medium/low settings.
Now I just need to upgrade my monitor next to get even more out of this.