Or you can do the dns challenge for letsencrypt

This seems super overcomplicated. What I would do is put all the subdomains on the public DNS, let HTTP(S) through the firewall for the respective hosts, deny everything from outside of your local network on the http server that isn’t under the HTTP challenge path and then run the HTTP challenge as you would for a public site.

Then you can get certs, everyone outside trying to access will get 403, and inside the network you can access as normal.

Of course you’ll have to trust your http server’s ACL for that, but I’m just going to assume servers like nginx (which I use) have a reliable implementation.

Proxy host out on the public internet? Usually I just use a local private CA for this, and install the CA root in my browser.

I would recommend just using caddy. It removes the complicated part of ssl management. For a local network it'll setup a local self signed certificate authority and you can just install those certificates to any devices on your LAN that you want to have access. For a public setup it'll use letsencrypt. You will still need to setup dns if you want wildcard routing.

I'm running cloudflare and NPM, I did a DNS challenge to get my wildcard cert, then put in access lists so for my internal hosts only private IP address subnet can access them. I have my OPNsense firewall also redirect any of those internal hosts request back to my NPM host. I have everything internal with a valid https cert.