in a previous post, I talked about setting up a distributed S3 like data storage system using Minio and ZeroTier. Well, this week, the ZeroTier guys tweeted about this.
Distributed S3 data storage using @minio and #ZeroTier — https://t.co/MAXTjo3muh
— ZeroTier, Inc. (@ZeroTier) May 2, 2017
A few people then started asking questions, and looking for a follow up, so here it is…
First, a quick recap. I had 4 machines, all running Linux. Three of them were in 1 time zone (GMT+1) and one was in another (GMT). Looking at the Distributed Minio Quickstart Guide again, there is a mention of times being in sync… which is probably why this did not work as planned… and by “not work as planed”, I mean that Minio would crash, or not be responsive, or not write data in the place it should have… which was a pain. But looking at the documentation again, they do mention that Windows support is “experimental” which means, hopefully, some day it will be not so experimental, and might work… Given that most of my machines in house are Windows boxes, this would be a nice feature.
Now, what about ZeroTier? Given they posted it to their twitter? Well, it worked. it did the inter connect stuff well, and, given bandwidth limitations on a home broadband connection, it was still quite fast.
So, the question is, how fast? Well, on my Surface Book on a WiFi connection in the house, behind a Meraki MX64 firewall, connecting to the GodBoxV2 over FTP though ZeroTier, i get the following result:
the same download over FTP direct (no ZeroTier) does the following:
So, direct over FTP is faster… in this instance by about 70%, but, over the download, it did get slower (seen it hit 12 at one stage) and because its over WiFi, those are a bit wonky…
I did get one last screen shot:
as you can see, the Zerotier network adapter is showing 77.3Mbps, but the main network adapter is showing 80.8Mbps. There would be other traffic there, but if we assume there is nothing but ZeroTier traffic being sent, there is about 5% of an overhead.
So, to wrap up: Minio and its distributed storage system over ZeroTier needs more testing. Ideally, all hosts need to be in the same time zone, or at least have the same time… Will try work on that soon. As for ZeroTier? I am extremely happy with them. Its fast, easy to setup, and easy to configure. What more could you ask for? Oh, and free, unless you need a pro account!
---
This site is hosted on my own ASN on AS204994. More details about that over there. I also use Vultr for transit services and HostUs for LIR Services. Check them out. You can also check some of the gear I use on a daily basis over on kit.co/tiernano. Looking for a Backup Option? Check out Backblaze and get a month free.
