[NOTE] This part 5 in a series of posts. The rest can be found here.
This post is going to be an update and theoretical post. probably very little “new” stuff going on here, mostly updates, and what I am planning on doing later on.
This week, I have been OOF sick, so I have not done much work, but I have been surfing the web, watching videos, downloading stuff, etc., so I have an idea of how things are going. First, as mentioned in the previous post I have MPTCP, Squid, Socks Servers, OpenVPN and IPTables doing their magic. 2 OpenVPN tunnels between the house and Digital Ocean. All TCP Traffic (bar port 80) is sent over socks to the box in the cloud using RedSocks. All UDP traffic is sent direct over OpenVPN. Since MPTCP is in the mix, all socks traffic is actually split over the 2 connections. All port 80 traffic, and 443 (if the client is using local Squid as their proxy) is sent round-robin between the 2 upstream IPs to Squid (2 OpenVPN end points).
Things I have noticed:
Every now and again, RedSocks crashes… just full on dies. It’s just a matter of starting again, but it’s a pain…
I have had to restart squid a couple of times… not too often though
there was a power outage in the house a few days back… so, when everything came back online, it was a bit of a pain bringing all connections back to life. I do have to figure out a better plan
I still have to read more on this ECMP stuff. Hopefully it will do what I am hoping.
Now for the theoretical stuff. I started thinking, could this work outside the house? Could you build this into something smaller, like a Raspberry Pi, and stick 2 or more USB Modems in, connect it back to a server in the cloud, setup P2P OpenVPN connections and then get more than a single modem speed download? The problems I can see are around MPTCP. I am not sure if it has been ported to ARM to run on a Raspberry Pi. Second, the max you could ever get out of it is 100Mbit/s, given the 10/100mb network port on board… and you may need extra power for the USB dongles. Also, getting P2P connections may be complicated, given the non-static IPs on the modems, though, in theory, non-P2P OpenVPN could work… Again, it’s a theory. I had the though and that’s where the title came from… anyway, throwing it out there…
I am also noticing that I am starting to hit the limits of my upstream VM. If downloading or uploading at speed, the processor cores (2 in the case of the box I am currently running) are pegged at pretty much 100% full… Well, 80ish, but that because the other 20% is being used by Dante. I am noticing I can hit a full 72Mbit/s up, but the max currently downloading is about 400, maybe 450… Need a faster box now…
I mentioned port 80 not being set over socks. That’s because its redirected to Squid. Squid (in house) then uses Squid (in cloud) as a parent. There are 2 round-robin parents for squid, one on each OpenVPN connection IP address.
all other traffic (UDP, ICMP, etc.) are sent over the OpenVPN connection… currently only one is picked, but I have a cunning plan…
The cunning plan? Well, if I am reading the internet correctly, and I would like to think I am, I thinkECMP, or Equal Cost Multi-Path Routing, could help… Again, it’s a fledgling idea currently, and I am still reading the documentation, but if it works… Well… I not sure… let’s see…
At the 2016 Build conference, Microsoft announced that Bash on Ubuntu on Windows was coming. Well, it came out last week, and I installed it as soon as I could! My next challenge was to get Jekyll to run and install on it, so I can build and preview this site on my Surface Book.
So, first, I needed to install version 2.0 of Ruby. There is a bit of messing involved for this, but first
apt-get update
apt-get install ruby2.0
Now, when you run
ruby -v
you will still see ruby 1.9.x installed… and the github-pages gem, which includes Jekyll 3, requires ruby 2.0… ugh. after reading this very long bug report I got this:
# Rename original out of the way, so updates / reinstalls don't squash our hack fix
dpkg-divert --add --rename --divert /usr/bin/ruby.divert /usr/bin/ruby
dpkg-divert --add --rename --divert /usr/bin/gem.divert /usr/bin/gem
# Create an alternatives entry pointing ruby -> ruby2.0
update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby2.0 1
update-alternatives --install /usr/bin/gem gem /usr/bin/gem2.0 1
Now, when I run ruby -v, I am told I am on version 2.0! Happy days! Next, i installed bundler using
gem install bundler
which uses a Gemfile to install the required gems, so running
bundle install
should install all required gems but no luck… I tried adding ruby2.0 dev to the mix
apt-get install ruby2.0-dev
but running jekyll from bash said it did not exist, and running
bundle exec jekyll build
failed with a memory issue… ugh…
Anyway, next, I tried
gem install github-pages -V
-v makes it verbose, so you can see what’s going on… after a bit of time, and lots of output to the screen…
jekyll build
bash: /usr/bin/jekyll: No such file or directory
FECK! So, after a bit more digging, i find that jekyll is actually in /usr/local/bin/jekyll
ln /usr/local/bin/jekyll /usr/bin/jekyll
solves that problem!
now running
jekyll build
works perfectly, as does
jekyll serve
HAPPY DAYS! Mind you, I am only using this as a testing and writing system. I have not tried s3_website just yet, but that is being sorted elsewhere anyway. Maybe my next post will explain that…
[NOTE] This part 3 in a series of posts. The rest can be found here.
In Part 1 of this series I explained the why and what I wanted to do for this “project”. In Part 2 I did some basic testing of both MPTCP and MLVPN. I also mentioned trying MMPPP using vtund but it has been a while since I did that testing, and it had not been on bare metal. So, this post is a follow up, where I am using bare metal.
So, first, the setup:
ProLiant box is running Debian 8.3 x64, and has both vtund and ppp installed
Digtial Ocean box also has Debian 8.3, vtund and ppp installed
walked though the guide from John Lewis and made some changes to the configs. the main ones are mentioned below
Once done, i installed both iperf and iftop on both boxes, and ran
iperf -s
on the Digital Ocean box and
iperf -c 192.168.10.1 -d
on the local box. And, well, the results where not as expected. Pretty poor actually:
First, using Squid installed on the DO box, i tried using WGET to download a file using it. If I did this on the DO box itself, i was getting 100MBytes/s… When I ran it over the MLPPP box, well, under 7 was achieved.
Then i though it might have been Squid. So, since the file had been downloaded to DO, i SFTPed into the box over the MLPPP link, and tried again… Again, pretty poor result. I think i seen it hit about 7MB a sec at one stage.
Here is what is showing on the DO box when running the SFTP download. You can see 2 connections from the 2 WAN links at home hitting the box, and they are balanced. Its just nowhere near the speed they are capable of.
I did not get a screen shot of this, but when I tried with iperf, thinking it might have been overhead of SFTP or Squid, I was getting results matching what I was seeing with SFTP. Downloads in the 55-60mbit/s range for download and 40ish for upload. 40 is still faster than 1 link, mind you…
I mentioned that I had made some minor tweaks to the configs from what John had written. Well, mostly it was config changes to how routing was done. In Johns case, he is bonding a DSL and a HSDPA connection, so he had setup to do for logging into his PPP modem and connecting. Also, when he setup the interfaces, he routing tables in there. I have mine setup in a single config file, like as follows:
I have changed the names from adsl1 and 2 to WAN1 and 2, and the IPs are changed from internal IPs to my public IPs. I manually run this when setting up my connection.
Nothing else on his config files have changed. I did not do any of the masquerading stuff, mainly cause this was testing. I just want a tunnel to start with. When reading the vtund.conf file, you can see that encryption and compression are both turned off, and and the same in the ppp configuration. I also don’t think the issue is to do with the CPU performance, since these are the screenshots of top running on both boxes:
in both cases, CPU usage is sub 6% for VTUN and SSH seems to be using less than 10%. So, now, I’m baffled as to why this is not performing as expected… More testing required!
[update 4/4/2016] – fixing images so they are clickable…