Yeah na, put your home services in Tailscale, and for your VPS services set up the firewall for HTTP, HTTPS and SSH only, no root login, use keys, and run fail2ban to make hacking your SSH expensive. You’re a much smaller target than you think - really it’s just bots knocking on your door and they don’t have a profit motive for a DDOS.
From your description, I’d have the website on a VPS, and Immich at home behind TailScale. Job’s a goodun.
+1 for the main risk to my service reliability being me getting distracted by some other shiny thing and getting behind on maintenance.
I started as more “homelab” than “selfhosted” as first - so I was just stuffing around playing with things, but then that seemed sort of pointless and I wanted to run real workloads, then I discovered that was super useful and I loved extracting myself from commercial cloud services (dropbox etc). The point of this story is that I sort of built most of the infrastructure before I was running services that I (or family) depended on - which is where it can become a source of stress rather than fun, which is what I’m guessing you’re finding yourself in.
There’s no real way around this (the pressure you’re feeling), if you are running real services it is going to take some sysadmin work to get to the point where you feel relaxed that you can quickly deal with any problems. There’s lots of good advice elsewhere in this thread about bit and pieces to do this - the exact methods are going to vary according to your needs. Here’s mine (which is not perfect!).
I still have lots of single points of failure - Tailscale, my internet provider, my domain provider etc, but I think I’ve addressed the most common which would be hardware failures at home. My monitoring is also probably sub-par, I’m not really looking at logs unless I’m investigating a problem. Maybe there’s a Netdata or something in my future.
You’ve mentioned that a syncing to a remote server for backups is a step you don’t want to take, if you mean managing your own is a step you don’t want to take, then your solutions are a paid backup service like backblaze or, physically shuffling external USB drives (or extra NASs) back and forth to somewhere - depending on what downtime you can tolerate.
+1 for Syncthing. I run it on a server at home, then on my MacBook over Tailscale. For web access I run FileBrowser (also over Tailscale) against the same directory.
I run two local physical servers, one production and one dev (and a third prod2 kept in case of a prod1 failure), and two remote production/backup servers all running Proxmox, and two VPSs. Most apps are dockerised inside LXC containers (on Proxmox) or just docker on Ubuntu (VPSs). Each of the three locations runs a Synology NAS in addition to the server.
Backups run automatically, and I manually run apt updates on everything each weekend with a single ansible playbook. Every host runs a little golang program that exposes the memory and disk use percent as a JSON endpoint, and I use two instances of Uptime Kuma (one local, and one on fly.io) to monitor all of those with keywords.
So -
My ‘good reason’ is just that it’s super convenient - for backups and painlessly moving apps around between nodes with all their data.
I would run plain LXCs if people nicely packaged up their web apps as LXC templates and made them available on LXCHub for me to run with lxc compose up, but they generally don’t.
I guess another alternate future would be if Proxmox added docker container supervision to their web interface, but you’re still not going to have the self-contained neat snapshot system that includes the data.
In theory you should be able to convert an OCI container layer by layer into an LXC, so I bet there’s projects out there that attempt this.
No answer, but just to say I run most of my services with this setup - Docker in a Debian LXC under Proxmox, and don’t have this issue. The containers are ‘privileged’, and I have ‘nesting’ ticked on, but apart from that all defaults.
There are a heap of general “Linux Administration” courses which will patch a lot of holes in the knowledge of almost all self-taught self hosters. I’d been using Linux for a while but didn’t know you could tab to complete file names in commands till I learned it on Udemy ¯_(ツ)_/¯
I routinely run my homelab services as a single Docker inside an LXC - they are quicker, and it makes backups and moving them around trivial. However, while you’re learning, a VM (with something conventional like Debian or Ubuntu) is probably advised - it’s a more common experience so you’ll get more helpful advice when you ask a question like this.
how to access the NAS and HA separately from the outside knowing that my access provider does not offer a static IP and that access to each VM must be differentiated from Proxmox.
Tailscale, it will take about 5 minutes to set up and cost nothing.
I’m also on Silverbullet, and from OP’s description it sounds like it could be a good fit. I don’t use any of the fancy template stuff - just a bunch of md files in a directory with links between them.
Your workload (a NAS and a handful of services) is going to be a very familiar one to members of the community, so you should get some great answers.
My (I guess slightly wacky) solution for this sort of workload has ended up being a single Docker container inside an LXC container for each service on Proxmox. Docker for ease of management with compose and separate LXCs for each service for ease of snapshots/backups.
Obviously there’s some overhead, but it doesn’t seem to be significant.
On the subject of clustering, I actually purchased three machines to do this, but have ended up abandoning that idea - I can move a service (or restore it from a snapshot to a different machine) in a couple of minutes which provides all the redundancy I need for a home service. Now I keep the three machines as a production server, a backup (that I swap over to for a week or so every month or two) and a development machine. The NAS is separate to these.
I love Proxmox, but most times it get mentioned here people pop up to boost Incus/LXD so that’s something I’d like to investigate, but my skills (and Ansible playbooks) are currently built around Proxmox so I’ve got a bit on inertia.
Is that a mini? I love those little 1L HP’s. I run 3 G2 800’s. These are very nicely built and therefore a joy to work on, and sip power when idling. Highly recommend. Also +1 for Proxmox.
For light touch monitoring this is my approach too. I have one instance in my network, and another on fly.io for the VPSs (my most common outage is my home internet). To make it a tiny bit stronger, I wrote a Go endpoint that exposes the disk and memory usage of a server including with mem_okay and disk_okay keywords, and I have Kuma checking those.
I even have the two Kuma instances checking each other by making a status page and adding checks for each other’s ‘degraded’ state. I have ntfy set up on both so I get the Kuma change notifications on my iPhone. I love ntfy so much I donate to it.
For my VPSs, this is probably not enough, so I am considering the more complicated solutions (I’ve started wanting to know things like an influx of fali2ban bans etc.)
- fiction
- Abbott, Edwin A_
- Flatland
- Flatland - Edwin A. Abbott.epub
- Flatland - Edwin A. Abbott.jpg
- Flatland - Edwin A. Abbott.opf
- Achebe, Chinua
- Things Fall Apart
- Things Fall Apart - Chinua Achebe.epub
- Things Fall Apart - Chinua Achebe.jpg
- Things Fall Apart - Chinua Achebe.opf
So in each directory that I use to delineate a library, I have a subdirectory for each author (in sort order form). Within each author subdirectory is a subdirectory for each book, with just the title, then the book with (edit - the anti-injection code mangled how I was trying to say the book file name. it’s [book name]-[author].[extension])
I didn’t invent this, it’s just what Calibre spits out. When I buy a new book, I ingest it into Calibre, fix any metadata and export it to the NAS. Then I delete the Calibre library - I’m just using it to do the neatening up work.
If this is a question about how to access your server at home from devices anywhere, securely, with a simple setup, then the answer is turn off all that port forwarding, and use Tailscale.
With a somewhat similar usecase, I ended up using Kavita.
Yo dawg, I put most of my services in a Docker container inside their own LXC container. It used to bug me that this seems like a less than optimal use of resources, but I love the management - all the VM and containers on one pane of glass, super simple snapshots, dead easy to move a service between machines, and simple to instrument the LXC for monitoring.
I see other people doing, and I’m interested in, an even more generic system (maybe Cockpit or something) but I’ve been really happy with this. If OP’s dream is managing all the containers and VM’s together, I’d back having a look at Proxmox.
This is where I landed on this decision. I run a Synology which just does NAS on spinning rust and I don’t mess with it. Since you know rsync this will all be a painless setup apart from the upfront cost. I’d trust any 2 bay synology less than 10 years old (I think the last two digits in the model number is the year), then if your budget is tight, grab a couple 2nd hand disks from different batches (or three if you budget stretches to it,).
I also endorse u/originalucifer’s comment about a real machine. Thin clients like the HP minis or lenovos are a great step up.
My step-up from Pi was to ebay HP 800 G1 minis then G2’s. They are really well made, there’s full repair manuals available, and they are just a pleasure to swap bits in and out. I’ve heard good things about, and expect similar build quality from the 1 liter Lenovos.
I agree that RAM is a likely constraint rather than processor for self-hosting workloads. Particularly in my case as I’m on Proxmox and run all my docker containers in separate LXCs. I run 32GB in the G2’s which was a straightforward upgrade (they take laptop like memory). One some of them I’ve upgraded the SSDs, or if not, I’ve added M.2 NVME drives (that the G2’s have a slot for).