It’s been just over a year since I last posted my 2024 Network Upgrade post. In that time, my network has undergone several changes. Here are the major updates:
Upgraded my FTTH link from 2Gb to 5Gb! (Hence the upgrade from the UDM Pro)
Added a UNAS Pro to the network for extra storage. It has 7x8TB drives in RAID 6, providing roughly 40TB of usable space (excluding the usual overhead).
Added Starlink to the mix of internet connections. Total bandwidth into the house is now around 6.5Gb/s, with upload at about 350Mb/s.
In the (hopefully not too distant) future, my cable ISP plans to upgrade from HFC (Hybrid Fiber/Coax) to full FTTH. When this happens, my speed will increase from 1Gb/50Mb to 5Gb/500Mb—and needless to say, I’ll be ordering that as soon as it’s available! This will boost the total download speed to just over 10Gb (about 10.4Gb, since Starlink tops out around 400Mb/s) and upload to 800Mb/s (with Starlink contributing 40-50Mb/s). That’s why I chose the UCG Fiber. It should handle two 5Gb incoming links, one 10Gb LAN connection, plus the Starlink connection (with IDS/IPS off, of course).
If you’re considering using Cloudflare Wrap for specific machines on your network, you can easily install the Warp client directly on them. It supports various operating systems, including Windows, Linux, Mac, iOS, and Android. However, if you need to use it on devices that aren’t compatible with the client installation, for example, NAS Devices or Smart TVs, this tutorial may be helpful.
First, please note that this is not an officially supported option. Cloudflare might modify their configurations at some point, potentially causing this feature to break. You have been informed about this possibility.
What do you need:
UDM Pro (it can work on any Ubiquiti Unifi gateways, but this is the one I have).
Wireguard Configuration File Generator (WGCF). This tool will generate a Wireguard configuration file based on the Cloudflare settings.
I’ve created a script that executes the following commands. It worked on my MacBook Pro, and it should also work on Windows or Linux.
First, install WGCF. I installed it by running
brew install wgcf
on my Mac Book Pro.
Next, run:
wgcf register
This will register a client on your machine. A wgcf-account.toml file will be left in your running folder. Next, run the script again.
wgcf generate
You’ll be left with a wgcf-profile.config file in your running folder. Open this file in a text editor to access the necessary details for your next steps.
Go to your Unifi Network Dashboard, click on “Settings,” and then select “VPN” and “VPN Client.” Click on “Create New” and choose “Wireguard” as the protocol. Then, change the “Setup” to “Manual.”
The configuration file you created earlier should resemble this:
Use the contents of PrivateKey to overwrite the existing Private Key. This will automatically fill in your Public Key. Next, set your Tunnel IP to the value listed for IPv4Address. Remove the trailing slash and use that in the Netmask (my Netmask was a /32). Server Address is the value listed as ServerEndpoint. Check the port and include it as well. The Public Server Key is ServerPublicKey. Finally, add your DNS settings for IPv4 in the configuration and click Apply Changes.
After a few seconds, the status should change to “Connected”.
Next, you need to configure the Policy-Based Routes. This is located under the routing section, specifically under the heading “Policy-Based Routes.”
Here, you can name the rule and decide whether you want to send all traffic or specific traffic.
For all traffic, you can select a specific device or the entire network. For instance, in this example, all traffic from my Guest network will be routed through Warp:
You can also set it to send traffic to specific destinations:
Fallback allows it to fail back to one of the other connections if the Warp connection fails.
Finally, click Add Entry at the bottom. Now, run some tests on that machine and see the traffic counts increase.
That is now it. You can select what devices or networks, or even what destinations you want to send over Cloudflare. Happy hunting.
I’m currently in the midst of a significant network upgrade for the CloudShed. I’ve purchased two Ubiquiti Unifi Hi-Capacity Aggregation Switches, a 24-port Switch Pro POE, a Switch Enterprise 8 PoE, a couple of U7 Pro Access Points, and a U6 In-wall Access Point.
The two Aggregation Switches each have four 25Gb ports and 28 10Gb ports. Two of the 25Gb ports will be connected between the house and the CloudShed. The U6 InWall will be installed in the office, while the two U7 Pros are already in the house and powered by the Switch Enterprise 8 Poe (which supports 2.5Gb Ethernet). The 24-port Poe Switch will replace my older 16-port switch, which lacks 10Gb Ethernet. More details will be provided as I have time to install everything.
Going to be a very quick update here. Things are a little more stable at the moment. I figured out why my FTTH connection was acting up… the VM I moved it too had the default free 1Mb/s license for RouterOS… After moving my unlimited CHR license over, things have gotten better. screenshots over on my mastodon instance:
So, today, not doing much other than monitoring… I am taking a day of rest and will be back tomorrow…
Day 40 of #100daysofhomelab and the internet is a little more stable… Still not 100%, but “stable”. Speed test results have dropped, as you can see in the graph below, but weirdly, ping times are a little better…
Download speeds. The swap over happened around the 8th Feb, 9th was pretty much a wash, 10th things got a bit better…
Upload Speeds. less spikey upload speeds, but also less upload speed…
ping times went from around 38-40ms to around 28-30ms…
I currently have Observium watching the traffic on the routers, and all logs are being written to an ELK stack. Not correctly (links below on how it *should* work, but I don’t have it fully working… yet) but they are being logged nonetheless.
OpenVPN in LXC – Proxmox VE Need to install Zerotier on an LXC container on proxmox. This is how to get it working.
this post is for day 38 and 39 of #100daysofhomelab… and i have finally moved over to my #RB5009… and, well, it has not gone so well… It has rebooted a few times due to memory issues (too many BGP tables being held, so I shut a few down to start with… some cleanup needed there), then the internet connections are a little unstable, and, well, in the last 48 hours, I have spent more time on LTE than on proper internet… It does seem to be working (ish…) now, but not as fast as it was. I am just using the #Zerotier link, so the #Wireguard links are currently off… Anyway, below are some links… I hope to make things work better tomorrow… And i also hope to have a better write up soon too…
Day 31 of #100daysofhomelab and I am going through the config from my CHR to bring over to my RB5009, and, well, I have no idea what I was doing when I built the original config… Now to try and figure out what the config did, since I want to document it here so I know what I was thinking, but to also possibly help someone else… Mind you, at this stage, it won’t be much help… I also need to figure out how to add my Zerotier Bridge into the mix.
So, as trying to get a high level overview of how this works, lets start with this:
The cable modem comes in at 1Gb/s down, 50Mb/s up. It hands off at 1Gb ethernet and plugs into a switch on VLAN 900. Anything on VLAN 900 can get a public IP from that modem (statically assigned, I have 5 usages, the first being the modem to act as a gateway).
FTTH comes in and goes to my small quad 2.5Gb box, which then, using CHR (we call this DUB1-BK01), hands off a /29 to VLAN 905. Again, any devices on VLAN 905 can get a public IP from there, and use BK01 as a gateway.
For the current CHR (DUB1-BGP01) it being a VM has currently got 3 connections: eth1 is connected to VLAN900, eth2 is connected to VLAN905 and eth3 is connected to VLAN901. VLAN901 has a /27 from my block of /24 addresses, and anything on that VLAN can use an IP from that pool and the IP from DUB1-BGP01 as its gateway.
DUB1-BGP01 connects to both lon1 and fra3 over WireGuard connections. All traffic to lon1 is sent over the Cable Modem link. All traffic to fra3 is sent over the FTTH link. Currently, there is no automatic failover if one link dies… This is where (hopefully) Zerotier comes into play.
I have a VM running on my i7 2.5Gb box that has connections to both VLAN900 and VLAN905, along with VLAN911. I have a bridge on that box that connects VLAN911 to a Zerotier network which is used only for internal peering. It has a /28 Public IP Range and anything on that bridge can use an IP from that network and talk to other machines. Currently that bridge is directly connected to my UDM Pro, and it gets a public IP and uses fra3 as a gateway. Sometimes traffic goes though fra3 but comes back over lon1 (due to asymmetric routing). But because of the way the network is working, all traffic can flow without issues.
The plan is to use that VLAN along with the 2 WireGuard links and give me 2 connections to lon1 and fra3. In theory, if one connection goes down, the traffic should be able to flow the other way…
So, at least that is the theory… How well this will work is anyone’s guess… But more messing with configs is required.
I am participating in the #100daysofhomelab challenge and have been posting a lot on Twitter as @tiernano, but some posts and tasks I am doing will require longer-form write-ups. So, some updates will include either Videos (which will be published on my Youtube Channel) or blog posts, which will go here. This is the first of the blob posts.
Up till this morning, if I wanted to update a record, I checked out the DNS records from my private Github repo, made the change, and ran the DNSControl commands on my machine (check for syntax checking the file, preview to show what will change at the provider level, and push to make the changes). But I wanted to have some automation for this. So, enter Github Actions.
I did a bit of digging and found a Github Action from koenrh called dnscontrol-action. The docs on this are quite simple to go through, so I created 2 action files for my Repo: preview and push. a Gist for Preview is below:
and the one for push is as follows:
The important parts are as follows:
In both preview and push, the check command does a syntax check of your DNS config file. Then preview will check the providers to see if any records need an update. When push runs, it will make the changes.
All my required secrets are set in the Github repo as secrets, so when the action is run, it will pull the required keys out. These are put into the environment variables. I use name.com and a registrar for some domains (though most have now moved to Cloudflare, and some, like my .ie domains, are with Blacknight, who are not supported on DNSControl). Cloudflare is used by the majority of my domains, and Route53 is used for 2 domains currently. There are around 53 domains current managed by this, and the plan is to add more. I also plan on getting some more automation around checking configs and sending alerts if anything changes.
So, enough “How it works” and show us it working!
Right. Let’s update my zt.tiernanotoole.net domain, which is used for Zerotier IPs internal to my network. It’s been a while since I did this, so most will be removed and a few adds… first, I create a new branch, called zt-update, and check it out in VSCode. I made my changes, git committed and git pushed to the branch.
at this stage, the actions have NOT run, since this is neither checked in to master, nor a PR for master.
I go into the create PR section, and I can see the changes I have made. in my case, I removed a load of unused records and added extras:
I now create my PR and wait for the checks to complete:
within a short time, I get an alert that all checks have passed, and I can find the results of the changes in the build (It was meant to add a note to the PR with the details, but I might be missing something in my config…)
Also, not sure why it is redacting part of my name here…
I check the rest of the list, and other than the deletes and creates in route53 for this domain, there are no other changes. So, being happy with that, I click the Merge Pull Request and the code is checked into master, and the DNSControl push command runs:
If i now go into Route53, i can see the records on the site:
Happy days! Next challenges to fix:
fix the PR to include the output of check and preview
only run a check and push on the master branch, and no need to run preview again…
run preview once a week and send alerts if anything has changed
A few weeks back, Ubiquiti released a pre-release update for the Unifi Network Controller, version 7.1.61. It got installed on my UDM and I noticed a few interesting bits that you might find handy… First, you will need to be signed up for Unifi Early Access before you can download or even read the release notes, but this is just a quick update based on my findings so far.
The first thing to note: You can see the list of devices connected to switches on the Overview Tab. I can’t remember exactly when that was added, but I think it’s new…
Under the ports tab, you now have a ports insight option:
Clicking this give you:
You can also select multiple ports and make changes at a bulk level:
You can also see a bit more info about each port:
Teleport VPN is also now added. This makes giving someone access to your network a LOT easier than usual. They will need the WifiMan software on Android, iOS or Mac to join. Not sure what happens on a Windows machine… Maybe it’s coming soon? To use it, just generate a new link and send it to your user. Not sure how to remove them afterwards (if you want to give them temp access for example…)
Final Interesting part, and something I have been waiting for for a while, under Traffic Management, you can now create custom traffic rules:
You can set it based on destination Domain Name, IP or even the full internet:
And you can set the Source to be All Devices, group of devices (network) or individual (or multiple targeted) devices.
Finally, you can set the output internet connection.
If you had multiple internet connections, and one had better speeds for stuff like Netflix, or you wanted to send bulk data over a different link, you can do this using this feature. Very cool stuff.
A few months back (well, November 2020) I wrote about connecting to my car with Zerotier. In this post, I mentioned using a TP-Link router running OpenWRT and a Huawei LTE dongle to connect to the internet, which allowed me to then connect to my Blackvue Dashcam and watch remotely… But it had some issues I wanted to fix:
The Huawei Wingle was a little slower on 4G than I would have hoped…
When the power in the car went out, everything stopped working immediately (12V sockets in the car run for about 20 min after the engine shut off)
It did not connect to the WiFi in the house when parked
No option for backing up Video…
So, I went digging to find some alternatives… and I realized I had a load of them floating around the house: the Raspberry Pi. Specifically, the 4GB Pi 4. I got my hands on a Pi UPS Hat, a couple of 18650 Cells and an SSD Expansion board with a 512GB (overkill I know) SSD. I also got a BlackVue Power Magic Battery, B112, which will power the Dash Cam (a BlackVue DR750S-2CH). It has 2 USB ports, which allows me to run both the Pi and the new WIFI router, a Netgear Nighthawk M1.
When the car starts, it powers, via the 12V socket in the boot of the car (trunk for my American friends), which powers the Blackvue Battery. Cables run from there to the front of the car where the front camera is. (there is also a rear-facing camera in the boot too… more cables!) This then also starts the Pi and starts charging the 2 18650 batteries. Finally, well, at the same time really, the Nighthawk starts running too. Because the batter on this was running hot, the battery is removed from this.
The Pi is hooked to the Nighthawk via ethernet and the WIFI is set to connect to the house when it sees it. The BlackVue uses the WIFI from the Nighthawk for its internet requirements. When the pi boots, it connects to Zerotier for management via SSH or VNC (I use VNC to remote into the box and watch the live video when the car is parked or when someone else is driving).
There is also a python script that is scheduled to run every 15 min that downloads the videos from the Dashcam. It also downloads any GPS and other info. The folder these files are downloaded to is on the SSD and is shared with my machine at home via Resilio Sync. To make sure I don’t use all my LTE usage, the machine at home is set to only download what I want to download. So, if the car is somewhere else, I can download specific files when I want, or when at home, I can download full days, if required.
It’s been running for a few weeks now, and so far, so good. I haven’t had to do any clean up of the SSD, yet, but I would guess that eventually, I will need to look into that… With the 4G connection and Zerotier, I can then connect to my car and watch the live video whenever it is online, and whenever it is driving, within 15 min it will start downloading videos. I could, in theory, do a LOT more with the Pi in the car… Some ideas that come to mind:
Turn WIFI off on the Nighthawk and use the Pi as a Router, probably adding a second WIFI adapter to get better range… This could then have PiHole running on it for monitoring DNS traffic…
Since I have access to the GPS files in (somewhat) real-time, use it to map the car in somewhat real-time. Though, I do this already using Ruhavik and a TeltonikaFMC-001.
Connecting to the car’s OBDII port (On-Board Diagnostics) and getting data from the car… Technically, again, the FMC001 does most of this, but in theory, it could be replaced with something else…
Keep an eye on the blog for future possible projects with this… Not sure where this project will get me, but we will figure it out at some stage… Leave a comment if you have questions!
