Hi all!

I will soon acquire a pretty beefy unit compared to my current setup (3 node server with each 16C, 512G RAM and 32T Storage).

Currently I run TrueNAS and Proxmox on bare metal and most of my storage is made available to apps via SSHFS or NFS.

I recently started looking for “modern” distributed filesystems and found some interesting S3-like/compatible projects.

To name a few:

  • MinIO
  • SeaweedFS
  • Garage
  • GlusterFS

I like the idea of abstracting the filesystem to allow me to move data around, play with redundancy and balancing, etc.

My most important services are:

  • Plex (Media management/sharing)
  • Stash (Like Plex 🙃)
  • Nextcloud
  • Caddy with Adguard Home and Unbound DNS
  • Most of the Arr suite
  • Git, Wiki, File/Link sharing services

As you can see, a lot of download/streaming/torrenting of files accross services. Smaller services are on a Docker VM on Proxmox.

Currently the setup is messy due to the organic evolution of my setup, but since I will upgrade on brand new metal, I was looking for suggestions on the pillars.

So far, I am considering installing a Proxmox cluster with the 3 nodes and host VMs for the heavy stuff and a Docker VM.

How do you see the file storage portion? Should I try a full/partial plunge info S3-compatible object storage? What architecture/tech would be interesting to experiment with?

Or should I stick with tried-and-true, boring solutions like NFS Shares?

Thank you for your suggestions!

    • chtk@feddit.nlEnglish
      3·
      13 days ago

      And if you need to mount a directory over SSH, I can recommend rclone and its mount subcommand.

      • ShortN0te@lemmy.mlEnglish
        3·
        13 days ago

        I assume you are referring to Filesystem Snapshotting? For what reason do you want to do that on the client and not on the FS host?

        • Appoxo@lemmy.dbzer0.comEnglish
          3·
          13 days ago

          I have my NFS storage mounted via 2.5G and use qcow2 disks. It is slow to snapshot…

          Maybe I understand your question wrong?

          • ShortN0te@lemmy.mlEnglish
            1·
            13 days ago

            If i understand you correctly, your Server is accessing the VM disk images via a NFS share?

            That does not sound efficient at all.

            • Appoxo@lemmy.dbzer0.comEnglish
              1·
              13 days ago

              No other easy option I figured out.
              Didnt manage to understand iSCSI in the time I was patient with it and was desperate to finish the project and use my stuff.
              Thus NFS.