I started self-hosting a bit prior to when Docker took off, and getting multiple services running was much harder. Service A wants a certain version of PHP installed with certain plugins while Service B wants a different version. You’d follow a tutorial for installing Service C and desperately hope that it wouldn’t somehow break Service A or B. You installed Service D for a bit despite all the installation pain and now want to uninstall it - I hope you tracked exactly what config changes you made throughout the system so you can undo it.

Docker fixed all of this by making each service independent through containers which made self-hosting 10x easier. I’d also add that I love how easy it is to transfer my setup to a new server - I keep all of my container volumes in a specific directory and my docker-compose files in another and that’s all I need to backup / transfer. Without Docker you’d have to specifically handle each & every configuration file and database location, and if you later upgrade to a newer version of the OS or a different distro you’d have to handle possible conflicts between your versions and what the distro expects.

FYI I’ve had a really good experience with using Headscale for a true open-source Tailscale experience. It helps that the Tailscale clients work with it too and that Tailscale (very unofficially) help support it.

I think you should take baby-steps and focus first on just getting something running for you to use. Maybe first experiment with configuring an application you’d like in a virtual machine before you spend money on hardware too.

I have a Wireguard network setup for my devices that routes through my somewhat distant server. I find when I have both it and Tailscale open, Tailscale tries routing through Wireguard even though both devices might be on the same LAN. Unfortunately I don’t believe Tailscale has a way to forbid it from routing over other VPNs or networks.

I can’t speak for those two services in particular, but I know that Matrix will check https://domain.com/.well-known/matrix/server to see what (sub)domain is responsible for domain.com. I suspect other services also use .well-known too.