So... no idea what happened but it works again.
Just in case anyone stumbles upon this thread at some point here are my current settings:
- The compatibility layer is proton-ge-custom (from AUR)
- Steam Input Translation in the Forza Horizon 4 Controller Settings is enabled using the "Official Layout for Forza Horizon 4 - Gamepad"
- Regular mode (not big picture mode)
- Steam Overlay is enabled
I still have the issue that on some screens it shows the keyboard buttons and not the controller buttons but since the controller works anyway I don't really care (it's just the optics).
Thanks everyone for your suggestions! :)