So, this part of my article set will be talking specifically about the router and wireless network. At the moment, my router is way overkill:
- Old HP Proliant ML110 G5
- Intel Core2Quad Q6600
- 8 Gb RAM
- total of 12 Gigabit network cards (of which 4 are currently used…)
- 500Gb HDD
I have been playing with some networking in the house and have managed to build some VLANs. The modems are connected both directly to the Router and to a dedicated switch port for a given VLAN. The plan for the upgrade, which i hope to complete sooner than the rest of the network is as follows:
- get the ML110 running ESXi and virtualize PFSense. Give it 2gb of RAM and some processor.
- take some of the network cards out of the box. It does not need 12 ports, but maybe leave the 2 quad ports in there. They should be connected to the main switch trunked. 8 may be overkill, but i never do things by half.
- the PFSEnse VM should be connected to all 3 WAN VlANs (900, 901 and 902) and should also have at least one port to the LAN. There may also be other ports for other internal VLANs.
with the spare processor, i can then add other (small) VMs to this machine.
So, in my last post i talked about the requirements for the home lab, and in this post, im going to talk about a few more updates i have made in the last few weeks.
First, the processors: in the first post, i talked about Xeon D or Xeon E3… Well, i missed one… The Xeon E5. I have 2 of these in GodBox 2, and you can get them into a microATX board. There does seem to be some limits with the microatx boards, but hopefully enough searching will find me what i am looking for. Ideally, i want it to take “normal” DDR3/4 memory (not SODIMMs like the ASRock one above) and also take enough of them to run 64 or 128Gb of ram (thinking 8 would do the job!). Also, i would like to have 4 GigE ports onboard and 1 management port. 4 onboard is not a hard requirement: If i can get one with 2 ports, i can always get a 4 port card for the PCI-Express slot… Finally, i would like it to have at least 6 SATA ports and possibly an MSATA port. Thinking Boot off MSATA (Windows Server 2016 Nano Server would be used), 2 SSDs and 4 HDDs. Using Storage Spaces, use the 2 SSDs as “Fast” storage for the pool.
I also think i moved off the idea of 10Gb. I like the idea of it, but given a small 10Gb switch costs upwards of a grand, and the plan is to build a machine for that price, i would prefer a fifth machine and using my existing Cisco 48 port switch and leave 10Gb as a future upgrade.
Also, changed from last time round is machine count. Originally i was saying 3-4 machines… now i am thinking 6-7… 5-6 of them should be Hyper-V boxes and the last one would be a Media Box.
I also think the Synology or SAN requirement is out… Hyper-V can be setup to do replication between hosts, and with a 4Gb link to the LAN, i think i should be ok. Also, if i have the media box separate, i should be ok there too. I will detail the media centre in a later post.
So, any suggestions or thoughts on what should and shouldn’t be looked at?
So, its that time again… HomeLab upgrade time… Or at least the planning for it. I am in the process of rebuilding my home lab, which involves pull all old servers out of the rack and replacing them with new ones… It also means rewriting the network, possibly upgrading some existing gear and hopefully getting the whole lot done on a budget of some sort…
So, why? Well, biggest reason for all this is currently heat and power usage. We use about 4-6x more electricity than the average house here in Ireland, which means our electricity bill is fairly high. It also means that the lab, which is also my office/bedroom, gets quite warm and uncomfortable during the summer month. There is an Air-Con unit in the room, and, well, that’s costing the most on electricity!
So, what I got is a basic overview of what I want from the homelab and hopefully in the next post, I will have an idea of what it will look like..
- 3-4 machines running a Hyper Visor (HyperV, VMWare ESXi or other). Leaning more towards Hyper-V purely because its what I got currently and its what we use in our main office.
- each machine should be connected to at least 2 networks: one for storage and migration, one for “public” to the LAN. There may be more VLANs for other networks, but 2 is a start.
- ideally, 10Gb connections would be nice, but multiple 1Gb connections would also work.
- shared storage (iSCSI, SMB3, etc) would also be a nice to have, but may bump up the server count (not actually a problem) but would increase power and cooling costs. An off the shelf box, like a Synology could do the job…
- Lower power usage and less heat produced is also a major requirement. Most of the boxes I am decommissioning are older Xeon hardware (5000 series upto a 5200 series process and even an older Xeon P4!). The newer Xeon E3 and the even newer Xeon D are a lot more efficient, use less power, produce less heat and are way faster than what I currently have. The E3 can use up to 32Gb of RAM and the Xeon D top out at 128Gb… Me being me would like more than 32Gb RAM… :)
- smaller machines would also be nice. I have been looking at both Xeon D and Xeon E3 Mini-ITX boards and cases for them. I do have a half height Dell Rack, which I host these machines, and ideally, these machines should be rack mountable, but micro ATX cases could work. 2 per shelf would work grand.
- Onboard IPMI and KVM support is something I want too… I do have a KVMoIP switch in the house, and it works, most of the time, but getting a box that has this embedded into the board would be ideal… A lot of the server boards had it as standard or allowed it to be speced, so that’s all good.
- I am also thinking of upgrading the router to a similar spec board… Possibly a Xeon E3, or even an i5…. Ideally it should have IPMI and KVMoIP on board and should produce less heat. Biggest issues is getting enough network cards into the box…
These are my requirements at a high level overview. Over time things may change, but lets see how we get on…
So, a few weeks back, i got my hands on a Hetzner Dedicated box. It has a quad core Xeon, 32Gb ram, 3x3Tb hdds, RAID controller and KVMoIP. one of the first thing i did was get myself a /29 IP pool (8 total, 6 usuable IPs). There where already 3 IPs given to me: 1 for the KVM, one for the box itself, and 1 as the router for the IP block.
So, i need to setup my own router, so i picked PFSense since its what i run in house. I gave it 2 network connections: 1 connected to the main network adapter on the VMWare ESXi box (public) and one to a virtual switch, which is only used by VMs. The public is the WAN link and it gets a static IP from Hetzner, and the virtual switch is then my “LAN” link. This allows me to have standard NATed network connections to any VM i have, but then, what do i do with those IPs?
So, after a lot of digging, i found the answer. So, this should help.
- Under firewall, click on Virtual IPs.
- click the plus. I then selected IP alias, selected the WAN interface and set the IP to my first public IP i wanted to give. in my case, i was given a /29 block, and my first address was 176. This is the network address. I used 177. Likewise, my last address is 183, but that cannot be used either as its a broadcast address. give it a description and then hit OK. Repease for all IPs you want to use. TIP: Give each a meaningful description!
- Next, click firewall, NAT and 1:1. Click the add button and select your interface as WAN. set the External Subnet IP as the one you want to use and your internal IP as the machine that will have it. Thats all i did on that screen…
- Then go to Firewall, NAT, outbound… this is where things got complicated. Set the mode to “Manual outbound NAT rule generation (AON - Advanced Outbound NAT)” and click save.
- Then create a new rule: Interface: WAN, Source, Network, IP of the internal machine and then under translation, under address select the IP you want to give it. If you followed my tip in step 2, you should see the descriptions in here.
After saving everything and reloading the firewall, visiting a page like WhatsMyIP or ICanHazIP should show you your public IP. You can then create firewall rules to allow access. Quick idea would be:
Firewall/Rules, Add, Interface WAN, Destination: Local IP you want to use, and give whatever “normal” rules you would (HTTP, lock down to source address, etc). Click apply and hitting that address using what ever method (SSH, HTTP, etc) should work.
YMMV, but hopefully this helps! Any questions, leave a comment.
Quick tip for all you with internet facing VMWare ESXi Hosts. I have just got my hands on a box on the Hetzner network (more on that later) and using their LARA system i installed ESXi on it. All was good, then i tried loggin in a couple hours later and i kept getting errors about my password being wrong… So, i tried a few more times, got pissed off and rebooted the box (had to do a hard reboot, since i couldent even get in over KVM). I though this was a hardware issue, or a config issue, and left it… yesterday, i had the console open most of the day, and when looking at somthing i noticed this:
Well, thats why i couldent login! So, tip: create a second user account, name it something other than root, give it a secure password and use that to login to your ESXi box. Ideally, your ESXi box should be behind a firewall, but in the case of a dedicated server, that may not be financially feisable… Hope this helps someone!