Tiernan's Comms Closet

Geek, Programmer, Photographer, network egineer…

Currently Viewing Posts Tagged proxmox

Building Cloud Images for Proxmox

To create an Ubuntu VM for a Kubernetes cluster using Proxmox, follow these steps: download and tweak the base image, sysprep it, create a template with specified configurations, and clone the VM. Adjust settings such as memory, storage, and IP configurations. Fix shared IP issues by resetting the machine ID.

[Updated] Updting this for 2025, including Noble as the image (24.04) and adding some other missing bits..

I needed to create a few Ubuntu VMs for a Kubernetes cluster for testing, and I wanted to make the process as simple as possible using Proxmox and some minimal automation. Here’s what I’ve done:

First, Download the base image:

wget https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img

To be able to run the commands below, you need to install libguestfs-tools on proxmox. An apt-update and apt install libguestfs-tools is all that is needed.

Then, tweak the image. Since I’m using my apt-cacher-ng proxy here, I’ve set the proxy for all VMs. You can remove it or adjust it as needed. If you want to remove it, simply remove the append-line option. Additionally, I’m installing qemu-guest-agent here. You can add any additional items at this point if desired.

virt-customize -a noble-server-cloudimg-amd64.img --install qemu-guest-agent --append-line '/etc/apt/apt.conf.d/00proxy:Acquire::http { Proxy "http://10.244.71.182:3142"; };'

That command does the following:

– installes the QEMU Guest Agent
– adds the proxy script to apt config

NOTE: If you are runing ProxMox 9, you will need to install dhcpcd first. So, apt update and apt install dhcpcd, then run the above commands.

Sysprepping the image resets it to the default stage. If you don’t perform this step, and you clone the machine multiple times, all the clones will have the same machine ID and IP address. [Note: This isn’t working fully for me. See below for the changes I made to the machine ID.]

virt-sysprep -a noble-server-cloudimg-amd64.img

Create the template. I used ID 9000 and assigned a name. You can modify this. Additionally, I’ve tagged mine with VLAN 72 (my Kubernetes VLAN). Feel free to change or remove this tag as needed. Furthermore, I set the disk size to add 50GB. Please replace any references to “godboxv2-tank” with your storage name.

qm create 9000 --name “ubuntu-2404-cloudinit-template" --memory 4096 --cores 2 --net0 virtio,bridge=vmbr0,tag=72

qm importdisk 9000 noble-server-cloudimg-amd64.img godboxv2-tank

qm set 9000 --scsihw virtio-scsi-pci --scsi0 godboxv2-tank:vm-9000-disk-0

qm set 9000 --boot c --bootdisk scsi0

qm disk resize 9000 scsi0 +50G

qm set 9000 --ide2 godboxv2-tank:cloudinit

qm set 9000 --serial0 socket --vga serial0

qm set 9000 --agent enabled=1

qm template 9000

This does the following:

– creates a new VM with id 9000 and sets the name, memory ad core count. It also sets the network card, and in my case the VLAN (remove ,tag=72 if you do not need a VLAN)
– imports the disk to the VM and puts it in the storage path. You will need to change your path here.
– sets the VM to use that disk as a SCSI drive
– sets the disk to be bootable
– sets the size of the disk to +50Gb of what it was originally… important as the disk is only a couple gig and you will run out of space quite quickly…
– sets up cloud init for the VM as an IDE CD drive.
– sets serial output for the VM
– Tells Proxmox the Agent is installed
– set the VM as a template.

Next, clone the template into a new VM.

qm clone 9000 2001 --name k8s-01

qm set 2001 --sshkey godboxv3.pub

qm set 2001 --memory 4096

qm set 2001 --ciuser tiernano

qm set 2001 --ipconfig0 ip=dhcp

Here we are creating a new VM with ID 2001, setting the SSH key, setting the memory and user, and telling the machine to get its IP from DHCP. 

Change tiernano and godboxv3.pub to your settings. Modify the names and memory as necessary.

As mentioned earlier, I’m still encountering the issue of IP addresses being shared. To resolve this, log into the boxes and execute the following command:

echo -n > /etc/machine-id

rm /var/lib/dbus/machine-id

ln -s /etc/machine-id /var/lib/dbus/machine-id

Reboot the computer, and the problem should be resolved.

Day 59 of #100daysofhomelab – Proxmox Updates, LTT Hacked, New Framework Laptops

Day 59 of #100daysofhomelab and Proxmox released 7.4 of their Virtual Environment. I have not upgraded any of my machines to it, just yet, but that’s the plan for the weekend. Other than that, some links:

Day 16 of #100daysofhomelab

day 16 of #100daysofhomelab, and my test from yesterday paid off!

droped from 10 to less than 4gb of RAM used on one of my proxmox boxes by moving the storage from local ZFS to NFS… Mind you, only seems to have made a difference on this box…

On a more different note, this blog is now on the fediverse. You can subscribe by searching for @[email protected] on your fediverse client. It is done with the help of pfefferle/wordpress-activitypub: ActivityPub for WordPress (github.com).

Day 16 of #100daysofhomelab

Day 16 of #100daysofhomelab and not much going on. Busy with work. I am running a test though. I seen the following tweet a few days back:

https://twitter.com/davidnburgess/status/1615394420639293441

Given some of my smaller boxes are running 90%+ memory usage, i have decided to move the VMs from my NUC to my QNAP storage box. Its going to take a while to move them over, but we see what RAM usage is like after.

Just for reference, this is the before:

Day 2 of #100daysofhomelab

Day 2 of #100daysofhomelab and more messing with Kubernetes… So far, I have built, torn down, rebuilt and torn down a second time… and now building for a third time! Techno Tims Ansible scripts for the Win! A couple of notes for today:

  • the script uses K3s version 1.24.8-k3s1. at some stage yesterday I tried changing this to 1.26.0-k3s1, the latest version from the K3s GitHub page… This was a bad idea. Rancher does not like this, and, well, I don’t know what I am doing, so I want to see what Rancher does…
  • ideally, you would have multiple master nodes, but, me being the lazy git that I am, only set up 1… but it does look like it could be changeable later on…
  • I have a total of 6 VMs running my K3S cluster: 3 are 4 Cores with 8GB RAM running on GodBoxV2, which is now running Proxmox. The other 3 are each running on my HP Micro Server, the Quad 2.5Gb Celeron Box and an 8th Gen Intel NUC… each is given 2 cores and 4GB RAM. That gives me a total of (roughly) 18 cores and 36GB RAM. Each VM has around 50 GB of storage and using Longhorn, I have around 250GB of space (master does not seem to contribute space). Replicas are set to 3, so not quite a full 250GB.
  • Why Kubernetes? Well, I have 2 VMs currently running my fleet of docker containers. I have lost count of how many i actually have. So, my plan is to use Kubernetes to move all them from those single docker boxes, and have them more distributed and more HA. This will allow me to move stuff around easier, or at least i think it will… At the very least, i get to play with new tech! :)

More work on the cluster is required. This blog is hosted in-house on one of the docker instances… Hopefully, at some stage, it will be moved to the K3s cluster! That would be the first major move!