So I’ve got a couple of services that I access using mydomain.org:1234 for example, but since Nextcloud runs its own web server it’s hosted on mydomain.org/nextcloud.
How do I set up my services so I can access them from a friendly URI like Nextcloud instead of ports?
Yep this is the way. What OP wants can’t be done with DNS.