Tiernan's Comms Closet

Geek, Programmer, Photographer, network egineer…

ZFS over multiple DVD/BD-R images

A couple of days back, I started thinking about archiving and backup software. I kind of have backups “sorted”, with my MacBook Pro using BackBlaze to backup to the cloud, Time Machine backing it up to my Synology, my VMs on Proxmox being backed up to Proxmox Backup Server off-site, my Synology and QNAPs being backup to B2 and Hetzner and some other bits and bobs… But for the Archiving stuff, I am not really set up… So, I went looking for archiving software. Couldn’t find anything, so asked on r/DataHoarder. Still no options, at the time of posting, but someone did reply with the idea of using DVDs (or Blu Rays) for ZFS...

Ok, that’s just crazy, but in a kind of a good way… kind of like the floppy RAID stuff I have seen… It does help with the storage of data, plus allows for potential loss of data… but it needs some automation to get it fully perfect…

Assuming you are using this for archiving, you could automate building 5 ISOs, just shy of 100Gb each, once a month, create ZFS ZRAID 2 or 3 (depending on how paranoid you are) and then write your data to it. ZRAID lets you lose 1 disk, giving you around 400GB of usage. Z2 brings that up to 2 losable disks, and 300Gb and Z3 is 3 disks and 200GB. I think Z2 would be your best bet, especially if you are using something like MDisk and are storing them safely.

Once finished, unmount and send an email saying you need to write the ISOs to disk. Label each disk with a unique serial number (this is where the archiving software would be handy) plus the set details and number (so, March 2023 Disk 1/5).

If you need something from that backup you stick it in the drives… You can do it with multiple drives, so with 5 disks and ZRAID, you need to mount a minimum of 4 of them. ZRAID2 needs 3 and ZRAID3 needs a minimum of 2… Ideally, you would want 5 of them, allowing you to check all disks (ZFS Scrub) and then get your files off.

A year of archiving would require 5 drives (say 100 quid a pop, USB makes things easier... Internal is possibly cheaper) and 60 disks (I Found 25 100Gb MDisks disks on Amazon for around 500 EUR) costing a total of maybe 2k, with 15 extra disks…

Follow-up questions:

  • Does ZFS allow the mounting of read-only?
  • Could you do this with Rewritable BluRay disks? Could they be mounted directly and written to? Leave them in the drives for the month, let writes do their thing and then archive them once a month? It’s archived, so it doesn’t need to be fast…

Day 52 of #100daysofhomelab

Day 52 of and I have been out of the lab most of the day. Still looking through all the docker containers on docker box 1, which has been running for many years now, and trying to figure out how to move everything is going to be fun… looks like I have an iSCSI volume added to the box. It’s shared between a few docker containers… then there are NFS mounts too… This might take longer than expected… So… more digging through YAML…

Day 51 of #100daysofhomelab

Day 51 of and I am planning my move of some of my Docker instances in the house to new machines… GodBoxV3 is currently running Windows Server 2022 with a couple of HyperV VMs on it. One runs docker containers and the other USIP from Ubiquiti for managing my EdgeSwitches. I am trying to move these VMs off that machine and do a clean-up, and the plan is to either install Proxmox with TrueNAS as a VM with disks passed directly into it, plus some other VMs or TrueNAS direct with VMs on there… Suggestions? Anyway, as part of the clean-up, I put my custom WordPress Container up on GitHub and it builds new builds nightly. The move is going to be fun, so my weekend will be busy… So, other than that, some links.

Day 50 of #100daysofhomelab

Day 50 of (this was stuck in a draft folder, so this is a couple of weeks old… I decided to recycle this as day 50, but it was originally day 37 or something…).

Just about 13 days ago: After running ZFS on my Mac for a few hours, I removed it and installed a trial of the SoftRAID software… I am not sure what was going on, but with ZFS installed, my machine just kept crashing… less than an hour and bang… So, I installed SoftRAID, and the speed ok… Not massive speeds, but not 100% sure I am using the right cables… More testing with cables soon…But in reality, this is software RAID 5 over 5 spinning disks. 270Mb/s read ain’t bad… 115Mb/s write ain’t great, but it’s RAID 5…

Cut to today: The trial of SoftRaid is just about up, and I am not sure I am going to buy it… I have been thinking of installing Proxmox or TrueNAS on GodBoxV3, which already has 8 8Tb Spinning drives, 7 NVMe drives (2 in RAID 0 for boot, and 4 in RAID 5 (ish, Windows Storage Spaces) one not usable for some reason, along with 2 960Gb SSDs). If I use the 5-bay enclosure with GodBoxV3, I can use that as one pool (External) the 8 Spinning disks inside as a second pool, the MVMEs as a third, and the SSDs either as a cache to the 8 internal disks, or possibly a more different pool… But this is something I am still thinking about… Anyway, links to random stuff are below…


Day 49 of #100daysofhomelab

Day 49 of , and I missed this yesterday, but it’s only going to be a link dump… And todays link dump is mostly Mikrotik gear! Some of these are a little cringy (looking at your Solid Rack video) But hilarious nonetheless! Some of these are so new, they don’t even seem to have product pages, just videos announcing them…

Day 48 of #100daysofhomelab

Day 48 of and I have been out of commission for the last couple of days… Havnt been well… cold and flu-like symptoms, but luckily, not Covid… Haven’t done a major amount, so it’s mostly links for today, but I did try a few projects and installed them. Links for those are below.

Day 47 of #100daysofhomelab

Day 47 of and i have missed a few days due to, well, a mix of laziness and being busy… I was working from my new home office today… still need to get some cabling and other stuff sorted, but we are getting close… nearly a year since it was installed! More on that at a later stage… for today, some links.

Day 46 of #100daysofhomelab

Day 46 of and I haven’t had much time to work on the homelab this weekend, but have had some time using it, somewhat indirectly… Plex, Netflix and Disney Plus streaming, etc. Internet is more stable (but not 100%… more messing on that part soon) and the RB5009 is definitely more stable (IPv6 BGP is currently off, and only using 5 of my 14ish BGP sessions I could use… I think 1Gb RAM is struggling, or my filters are wrong… Hopefully, it’s a filter thing, that way I can sort it out without new hardware.

day 45 of #100daysofhomelab

Day 45 (little late) of , and still quite busy with $DayJob stuff… Mostly monitoring of the network and the like… Some links are below:

day 44 of #100daysofhomelab

Day 44 of and still quite busy with $DayJob… so some links below. 

Also, I have been fiddling with some JQ and Zerotier-CLI commands… Not finished, but working on trying to get some data out of the CLI… I have a GitHub Gist with some details… I plan on adding to it over time.