With NAT on IPv4 I set up port forwarding at my router. Where would I set up the IPv6 equivalent?

The same thing, except for the router translating 123.123.123.123 to 192.168.0.250 it will directly route abcd:abcd::beef to abcd:abcd::beef.

Assuming you have multiple hosts in your IPv6 network you can simply add "port forwardings" for each of them. Which is another advantage for IPv6, you can port forward the same port multiple times for each of your hosts.

I guess assumptions I have at the moment are that my router is a designated appliance for networking concerns and doing all the config there makes sense, and secondly any client device to be possibly misconfigured. Or worse, it was properly configured by me but then the OS vendor pushed an update and now it’s misconfigured again.

That still holds true, the router/firewall has absolute control over what goes in and out of the network on which ports and for which hosts. I would never expose a client directly to the internet, doesn't matter if IPv4 or IPv6. Even servers are not directly exposed, they still go through firewalls.