So I did a few more tests. Removed and re-added my SwissBankers Card to Apple Pay. Same result.
I also paid online and in a store with Apple Pay with no issues.
Topping up TransferWise with ApplePay worked as well.
All signs point to the Revolut App being at fault. Maybe the last upgrade introduced a bug?
My account is fully verified and I have topped up via Apple Pay with the same card. The only thing that changed between the last top up and today is that yesterday I added my TransferWise Card to the Revolut app (and verified it immediately). Maybe that confused it?