Tiernan's Comms Closet

Geek, Programmer, Photographer, network egineer…

Day 35 of #100daysofhomelab

Day 35 of and I have been trying to clean up some stuff for my Mac Book Pro. I have an external enclosure from Yottamaster that has 5 3.5” bays and connects via USB C (USB 3.1). I got 5 8TB Seagate IronWolf drives in there. Currently, I have it set up as RAID 10 with 16Tb usable, which is named Archive, with 1 extra drive non-protected 8Tb drive. The details on setting up RAID 10 on MacOS is in the links section. I was looking at using RAID 5 for the archive pool, but the only option that seems to be available is SoftRAID but it’s USD250 for a license unless you have an OWC enclosure… Given the enclosure cost me that much in the first place, I think I will keep with RAID 10 for the while… RAID 5 would, potentially, give me 32Tb usable on my Archive, but 250 is a bit steep… for now…

IMG 1794

I also have a Sabrent USB 3.0 4 Bay 2.5” enclosure with 4 500Gb Samsung SSDs, named SCRATCH. This is in RAID 0 (I know, I know, if one drive goes MIA, all data is lost… That’s why this is a TEMP folder! It’s backed up to the Archive and also to BackBlaze). This is mostly stuff that is downloaded, and Video work that, when completed, is moved to the Archive Folder. Anyway, files are currently moving, so I will leave that as is.

On an update for the RB5009, It was originally planned for today, but the daddy found a TV show on Netflix, so it will have to be done either this evening or tomorrow morning… We will see… Anyway, some links:

Day 34 of #100daysofhomelab

Day 34 of #100daysofhomelab and I have realized I missed yesterday and also duplicated day 16… (facepalm). So, it’s day 34, I think…

Still working on the RB5009 upgrade. I am “technically” on holiday for the long weekend here in Ireland, so I have been out of the homelab more than I have been in. I need to move stuff around before I can swap in the RB5009, including changes to my VoIP setup (or at least wait till e everyone is asleep and won’t notice it being down) and some rewiring tasks… See below. I did also have to order new cables to try and keep some consistency in length… How well that will work is unknown… Hopefully, I will be back in the homelab a bit more on Tuesday… We see what I can break then.

IMG 1785

Day 31 of #100daysofhomelab

Day 31 of and I am going through the config from my CHR to bring over to my RB5009, and, well, I have no idea what I was doing when I built the original config… Now to try and figure out what the config did, since I want to document it here so I know what I was thinking, but to also possibly help someone else… Mind you, at this stage, it won’t be much help… I also need to figure out how to add my Zerotier Bridge into the mix.

So, as trying to get a high level overview of how this works, lets start with this:

  • The cable modem comes in at 1Gb/s down, 50Mb/s up. It hands off at 1Gb ethernet and plugs into a switch on VLAN 900. Anything on VLAN 900 can get a public IP from that modem (statically assigned, I have 5 usages, the first being the modem to act as a gateway).
  • FTTH comes in and goes to my small quad 2.5Gb box, which then, using CHR (we call this DUB1-BK01), hands off a /29 to VLAN 905. Again, any devices on VLAN 905 can get a public IP from there, and use BK01 as a gateway.
  • For the current CHR (DUB1-BGP01) it being a VM has currently got 3 connections: eth1 is connected to VLAN900, eth2 is connected to VLAN905 and eth3 is connected to VLAN901. VLAN901 has a /27 from my block of /24 addresses, and anything on that VLAN can use an IP from that pool and the IP from DUB1-BGP01 as its gateway.
  • DUB1-BGP01 does some BGP routing to my upstream servers. lon1, which is based in Vultr London, and fra3, which is based in M&M Networks in Frankfurt Germany. lon1 has transit from Vultr and fra3 gets transit from M&M Networks, but also connects to multiple Internet Exchanges: DE-CIX Frankfurt, DE-CIX Dusseldorf, DE-CIX Hamburg, DE-CIX Munich, KleyReX, LocIX and LocIX Dusseldorf. More details of the network and peers, etc, are available on as204994.net.
  • DUB1-BGP01 connects to both lon1 and fra3 over WireGuard connections. All traffic to lon1 is sent over the Cable Modem link. All traffic to fra3 is sent over the FTTH link. Currently, there is no automatic failover if one link dies… This is where (hopefully) Zerotier comes into play.
  • I have a VM running on my i7 2.5Gb box that has connections to both VLAN900 and VLAN905, along with VLAN911. I have a bridge on that box that connects VLAN911 to a Zerotier network which is used only for internal peering. It has a /28 Public IP Range and anything on that bridge can use an IP from that network and talk to other machines. Currently that bridge is directly connected to my UDM Pro, and it gets a public IP and uses fra3 as a gateway. Sometimes traffic goes though fra3 but comes back over lon1 (due to asymmetric routing). But because of the way the network is working, all traffic can flow without issues.
  • The plan is to use that VLAN along with the 2 WireGuard links and give me 2 connections to lon1 and fra3. In theory, if one connection goes down, the traffic should be able to flow the other way…

So, at least that is the theory… How well this will work is anyone’s guess… But more messing with configs is required.

Day 30 of #100daysofhomelab

Day 30 of and I tried to look into getting my RB5009 setup, and well… it has the wrong power supply! EU, not UK/Ireland… More messing is required! [Update] Found the right supply, but fell asleep watching TV… more messing tomorrow…


Day 29 of #100daysofhomelab

Day 29 of #100daysofhomelab and my RB5009 finally arrived! The bad news is I am up to my eyes with some out-of-hours updates for my $DayJob… So, it will probably be tomorrow or Friday before I get to it… It’s been that kind of a day. I am OOF from Friday to next Wednesday, so I should have plenty of time to play with. I also started playing around with Tailscale Funnel. I got my hands on an invite, and it looks like I can invite other people to it… If you are interested, leave a comment. I have not actually done much with it, mostly reading the docs and testing it before i make it public… But should be interesting. Anyway, now for some links.

Day 28 of #100daysofhomelab

Day 28 of #100daysofhomelab and I got some benchmarks for the WordPress site. First, using ab, going directly to WordPress. It does have W3 Total Cache turned on, using Redis for DB and Object Cache, etc. 10000 requests at 100 a go, 682 requests a second and meantime of 146ms per request. Total bandwidth is around 50Mbit/s.

CPU usage while running this is somewhat pegged around the 100% mark.

Next, we run the same but this time direct to Varnish. It is caching the requests and not hitting the Nginx box. We are now at 1899 requests per second (2.7X more) and our meantime is down to 52ms (nearly 3x faster). and the bandwidth is now nearly 140Mb/s, again, nearly 3x higher.

and CPU usage is a little bit lower too!

So, happy days! Tomorrow I will be working on my RB5009 install, so photos, shouting and more will be uploaded then… but for now, some links.

Day 27 of #100daysofhomelab

Day 27 of and it does look like WordPress is running correctly and quite fast… Yesterday’s messing with configs got Varnish, Memcached and Redis all running along with upgrading from PHP8.0 to 8.2. The problem now seems to be related to caching rules… So, some messing with that is required… My RB5009 is now stuck in France and has been there since Friday… It is scheduled for delivery on Wednesday, so that will be a fun day breaking stuff… Its been on quite the trip. Most of that was in 3 days, but it got stuck in France and hasnt moved over the weekend… Fingers crossed it arrives on Wednesday!

So, some links… yea, some are not exactly home lab, but its homelab adjacent?

Day 26 of #100daysofhomelab

Day 26 of #100daysofhomelab and I have been trying to figure out why my internet has been unstable today… it up and down a few times… well, parts of it are… Zerotier seems to be sorting out my main network, it’s smaller parts that are going wonky… I am half thinking of leaving it till next weekend since my RB5009 arrives next week… This should help me sort out my network…

Also, spending time upgrading my WordPress site too… just making sure all is working correctly… Fun times…

[Update]: I have managed to upgrade to PHP 8.2, the latest Nginx and now have Varnish in front of the site… Let’s see what breaks…

Day 25 of #100daysofhomelab

Day 25 of #100daysofhomelab, and not done much in the way of home lab work today, but has tested the bejesus out of the internet connection! I bought a Backblaze License for my Mac Book Pro, which initially has around 2.3Tb to backup. There are my YouTube Videos along with code and other bits… It looks like it has uploaded 290 Gb in the last 24 hours…

I also bought an Xbox Series X, and have downloaded a few games to it too… I previously had an Xbox One S with the Games Pass Ultimate, so those games were downloaded. I think it’s downloaded nearly 200 GB in the last few hours! Finally, my mother got home from the hospital yesterday and found a Netflix TV show she wanted to watch and has binge-watched most of it. That seems to be a bit more sedate 20Gb since last night… Overall, the Zerotier-backed connection seems to be working well!

Other than that, watched the Techno Tim video on MaaS. Looks interesting. And I am also looking into the idea of using Mastodon/Fediverse replies in WordPress… I found this post about doing it on static sites. More digging required i think, but now I’m off to play Flight Simulator!

Day 24 of #100daysofhomelab

Day 24 of #100daysofhomelab and most of it was spent migrating my ADS-B stuff from a VM to a Raspberry Pi (see Day 23 for links). So far, I am “feeding” FlightRadar24, ADSB Exchange, FlightAware and RadarBox. I also love some of the graphs I am getting out of it below. Currently, the antenna I am using is a little small and hanging out of a window, so I am missing some flights. The next plan is to get a better one and move the Pi to the CloudShed where I can mount the antenna better.