Tiernan's Comms Closet

Geek, Programmer, Photographer, network egineer…

Monthly Archives November 2020

Connecting to my car over ZeroTier

I use ZeroTier on my network for a good few things, including internal network peering between BGP VMs, management of machines, and now, connecting to my car over LTE. This is one of those posts that sounds silly, but is very handy! First, the parts list:

  • Car…
  • 3G/4G/5G modem of some sort. I am using a Huawei Wingle… Can be used without the Router below, but I wanted Zerotier, so I have it in modem only mode…
  • A router that supports Zerotier. I am using a modified TP-Link TL-WR703N upgraded to 16MB ROM and 64MB RAM. This is required for newer OpenWRT builds
  • a dashcam that connects over Wifi. I am using a BlackVue DR750S-2CH
  • Latest ROOter software from Of Modems and Men
  • Patients…

After installing the the latest copy of ROOter on the TPLink (or router of your choice) and getting the modem configured correctly (this took a while) you need to install the Zerotier software though the dashboard. Once installed, I joined my Zerotier network using the CLI (SSH into the router) and the approved it though the my.zerotier.com dashboard. Once its approved and connected, you can now go to the Zerotier IP and get to the router directly. From here, you can either setup a route in Zerotier to point at the internal network behind the router, or, in my case, setup a  SSH tunnel to the dashcam. I found the IP given to the dashcam and used SSH forwarding to get to it. Finally, i used the URLs from Digital-Nebula’s hackview repo to get to the different URLs. I use this to download stuff like GPS logs, emergency videos, etc. I have to clean up some scripts at some stage for this, and plan to upload them at some stage.

If anyone has any questions, leave a comment!

Backups, Backups, Backups!

I have posted about backups a few times on this site in recent years, and its still something I make tweaks to every now and again. The latest setup is probably over the top, but I will give you a walk though on it and some of it could be useful to some of you.

I have a couple of different machines and storage devices running that need backups. Some need daily backups, some could get away with weekly. The list is as follows:

  • GodBoxV1 (2X4 Core Xeon, 82GB RAM, Fedora, 512GB Boot SSD, 5x4TB HDD in ZFS RAIDZ1)
  • GodBoxV3 (2×20 Core Xeon, 192GB RAM, Ubuntu, 2x512GB NVMe SSD in RAID 0 for booth, 4X512GB NVMe SSD ZFS stripe for FAST storage, 8x8TB HDD in ZFS RAIDZ2 for bulk storage)
  • Docker Box (VM, runs a LOT of different containers on the network)
  • Synology DS1817+ (8x8TB HDD in SMR with 48TB usable, 2x10GB + 4x1Gb NICs)
  • QNAP TS-932X (5X8TB HDD in RAID 6 along with 4X512GB SSDs in RAID 5, 2X10Gb NICs)

2020-11-04_10-15-00_IMG_2342

GodBoxV2 and the 4 C6100 boxes are running Widows Server 2019, and I have 4 new C6220s which, when in production, may be either running Server 2019 or VMWare ESXi. More on this in a future post. GodBoxV1 and V3 are being backed up with Borg/Borgmatic, and the Server2019 boxes are running Hyper-V and the VMs are not backed up on a nightly basis, but that is planned in the future…

Borgmatic is basically a very nice and handy wrapper for Borg itself. It allows you to easily configure a YAML file with what you want to backup, what you want to exclude, where you want it backed up to (multiple locations if required) and details on retention, etc. It also allows you to send notices when something completes or fails. I have 3 main machines which are backed up using Borgmatic, but will probably add more at some stage. These three backup to 3 different locations; Local ZFS Storage in house (currently on GodBoxV1), RSync.NET and Hetzner’s Storage Box. [Note: Hetzner have 2 types of storage: Storage Box and Storage Share. Storage Share seems to be NextCloud and does not have BorgBackup installed. Storage Box can be used with BorgBackup though]

[Note: RSync.net have an offer for Borg Storage: 1.5c per Gig. So, 100Gb a year costs only $18. On their signup page, if you enter referral code 2019-09-13_05-27-04, I get some extra storage for backups on my end, and you can help me continue writing random stuff here!]

Nightly, Borgmatic runs and backs up everything important on GodBoxV1, V3 and the Docker Box, to all three locations. Then, on GodBoxV3, we backup some larger files (photos, video and other large data from my cameras) to Hetzner. I also plan on setting up a backup of those larger files to either my Synology or QNAP boxes. The reason the large files are only backed up to one current location is size; they currently weigh in at around 300GB, give or take, and I currently have around 200Gb of usable space with RSync.NET. My plan is to use the QNAP or Synology box as a secondary backup for this storage at some stage.

On a nightly basis, the Synology runs backups to both Backblaze B2, Wasabi and Hetzner using Hyper-Backup. Finally, on a weekly basis, some folders on the Synology are backed up to AWS Glacier.

This gives me a fairly good set of backup options, but there are some tweaks I want to make:

  • Important VMs on the Hyper-V Cluster should be backed up. Daily backup to local storage (QNAP, Synology, ZFS) and one weekly backup external (Hetzner, B2, RSync.net)
  • Large media files backed up to a second location, either local or remote.
  • Intel Nuc, Home Laptop and Mac Mini should also be backed up. 99% of the time they use storage from the ZFS pool or the NAS devices, but they still have local storage.
  • Look into backing up iPhones, Android Phones, iPads, etc, to local storage also. I do use PhotoSync to copy photos from my iPhone to the ZFS storage, which is backed up, but having something to backup the rest of the data, other than iCloud, would be handy.

So, thats my 2020 backup plan. Any comments, questions, etc, shout in the comments section.