Getting started with Proxmox

Detailed image of a server rack with glowing lights in a modern data center.

Virtualization is at the heart of many things today, especially in development. Proxmox is a Debian based OS which is exactly aimed at VMs and container, tried and tested over the years.

I own an older laptop with an OLED display which suffered some burnin. It has 32GB of RAM and 12 cores on the CPU, so it makes it the perfect candidate for a home server on which to experiment.

Read more:

Textbook install

The internet is full of tutorials on how to install Proxmox on a machine, so we will just use one of those to get started:

Some ideas:

  • in case the install interface boots, but once you select the graphical interface nothing happens, restart the install process and before selecting the graphical interface install, press e , add nomodeset on the line starting with /linux , then press Ctrl+x;
  • a wired connection is preferred over a wireless one, due to bridging defaults;
  • having the wired connection set when installing with ease the process;
  • remember to run the post install proxmox script;
  • in case you are using a usb adapter for the ethernet, be aware that the usb adapter is the ethernet card itself, which means you will be bound to it and will have to update the configuration with new ones;

See more:

zsh

I like pretty things, so antigen is something I install on most of my servers, together with zsh:

  • apt install zsh git
  • wget -O /usr/local/bin/install_antigen https://raw.githubusercontent.com/cristidraghici/debian-server-bash-scripts/master/install_antigen.sh
  • chmod +x /usr/local/bin/install_antigen
  • install_antigen --root

ssh

To avoid adding the password every time we connect, we can add public ssh keys to your authorized_keys:

  • cat ~/.ssh/id_rsa.pub
  • copy the value and ssh into your server;
  • nano ~/.ssh/authorized_keys
  • paste the key and save.

This is the manual way of doing it via the command line.

Setup the graphics

With the laptop I own which has an Intel and an Nvidia graphic cards, it's important to setup the Hybrid mode for video in BIOS. What we aim is stop the screen once the OS starts and also be able to use the NVIDIA gpu in a VM for running A.I. locally.

Make sure that the laptop does not go into suspend when closing the lid:

  • nano /etc/systemd/logind.conf
  • alter the following:
    • HandleLidSwitch=ignore
    • HandleLidSwitchExternalPower=ignore
    • HandleLidSwitchDocked=ignore
    • LidSwitchIgnoreInhibited=no
  • systemctl restart systemd-logind

Remove nomodeset from the proxmox installer:

  • nano /etc/default/grub.d/installer.cfg
  • remove nomodeset

Update grub:

  • nano /etc/default/grub
  • change GRUB_TIMEOUT to 0
  • change GRUB_CMDLINE_LINUX_DEFAULT="quiet" to GRUB_CMDLINE_LINUX_DEFAULT="quiet loglevel=0 console=tty3 intel_iommu=on iommu=pt consoleblank=10 vt.global_cursor_default=0"
  • update-grub
  • reboot

Custom resolution

The display I use uses a high resolution by default. That makes it hard to operate when in starts in commands line.

To fix this, we will add some customization:

  • nano /etc/default/console-setup
  • alter the font size to: FONTSIZE="32x16" and save the file and exit;
  • setupcon
  • reboot

If the above worked, but font is not bigger on restart, then it means we need to make it permanent. The choice to do this is via a service:

  • systemctl enable console-setup
  • systemctl restart console-setup

Only AC power

When running out of AC power, we want the server to gracefully shutdown and start once AC is back on. With this laptop, it's possible to have it boot up when AC power is available. And for shutting down, we can create a script and a acpid:

  • apt update
  • apt install acpid
  • systemctl enable acpid
  • systemctl start acpid
  • touch /etc/acpi/events/ac-loss
  • touch /etc/acpi/ac-loss.sh
  • chmod +x /etc/acpi/ac-loss.sh

Create the AC event in /etc/acpi/events/ac-loss:

event=ac_adapter.*
action=/etc/acpi/ac-loss.shCode language: JavaScript (javascript)

Create the shutdown scripts in /etc/acpi/ac-loss.sh:

#!/bin/bash
AC_PATH="/sys/class/power_supply/AC/online"
DELAY=10

STATUS=$(cat $AC_PATH 2>/dev/null)
if [ "$STATUS" = "0" ]; then
    echo "$(date): AC unplugged, starting $DELAY second shutdown timer..."
    for ((i=0;i<DELAY;i++)); do
        sleep 1
        STATUS_NOW=$(cat $AC_PATH 2>/dev/null)
        if [ "$STATUS_NOW" = "1" ]; then
            echo "$(date): AC restored, cancelling shutdown."
            exit 0
        fi
    done
    STATUS_FINAL=$(cat $AC_PATH 2>/dev/null)
    if [ "$STATUS_FINAL" = "0" ]; then
        echo "$(date): AC lost for $DELAY seconds, shutting down!"
        /sbin/shutdown -h now "AC power lost"
    fi
fiCode language: PHP (php)

The last step is to restart the daemon:

  • systemctl restart acpid

Setup wifi

Since we have a wifi card available, we can connect it, for now as a fallback connection to the server, in case ethernet fails.

  • apt update && apt install wpasupplicant -y
  • wpa_passphrase [YOUR_SSID] [YOUR_PASSWORD] | tee /etc/wpa_supplicant/wpa_supplicant.conf
  • ip a

The last command will show the list of network interfaces. First we need to identify the wifi card, then we will update the network interfaces:

  • nano /etc/network/interfaces

Then add the following to the file, but make sure not to conflict with existing configuration:

auto [WIFI_INTERFACE]
iface [WIFI_INTERFACE] inet dhcp
    wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

The last step is to reboot the system.

Troubleshooting

USB adapter sleep

For this setup, we are using a USB adapter for the network connection. It seems that it falls asleep because of a Linux kernel feature called "USB Autosuspent". The immediate fix is just to disable this feature:

  • echo -1 > /sys/module/usbcore/parameters/autosuspend

To make it permanent, we can update grub:

  • nano /etc/default/grub
  • add: GRUB_CMDLINE_LINUX_DEFAULT="... usbcore.autosuspend=-1" and save and close;
  • update-grub
  • reboot

If there are still issues, disable the power management:

  • nano /etc/modprobe.d/pve-network-fix.conf

Add the following content:

# Disable power management for Intel WiFi
options iwlwifi power_save=0
# Disable power management for Realtek Ethernet
options r8152 bus_pwr_cap=0Code language: PHP (php)

Disable PCI Express Active State Power Management:

  • nano /etc/default/grub
  • add: GRUB_CMDLINE_LINUX_DEFAULT="... pcie_aspm=off" and save and close;
  • update-grub
  • reboot

Then reset:

  • update-initramfs -u
  • reboot

Add the ethtool and update the network interfaces:

  • apt update && apt install ethtool
  • nano /etc/network/interfaces
  • append to the ethernet interface: post-up /sbin/ethtool -K eno1 tso off gso off
  • systemctl restart networking

Add a systemd service for unloading the network drivers:

  • nano /etc/systemd/system/unload-network-modules.service
  • Add and save the following content:
[Unit]
Description=Remove network drivers before reboot to fix warm boot hang
DefaultDependencies=no
Before=shutdown.target reboot.target halt.target kexec.target

[Service]
Type=oneshot
ExecStart=-/sbin/modprobe -r e1000e iwlmvm iwlwifi mac80211 cfg80211
TimeoutSec=10

[Install]
WantedBy=shutdown.target reboot.target halt.target kexec.targetCode language: JavaScript (javascript)
  • systemctl enable unload-network-modules.service

Delay the init of the network:

  • nano /etc/network/interfaces
  • append pre-up the following to the interfaces:
iface eno1 inet manual
    pre-up sleep 5

Periodical network check and reset

We will use a script to reset the network connection periodically, to make sure everything is fine within our server:

#!/bin/bash

# Target to ping (your router/gateway is best so it doesn't trigger if your ISP drops)
TARGET="192.168.0.1"

# Ping the target 3 times. If it fails, initiate recovery.
if ! ping -c 3 $TARGET &> /dev/null; then
    echo "$(date): Network unreachable. Initiating driver reload..." >> /var/log/network-fix.log

    # Remove Wi-Fi and Ethernet kernel modules
    modprobe -r iwlmvm iwlwifi mac80211 cfg80211
    modprobe -r e1000e
    sleep 2

    # Reload the modules
    modprobe e1000e
    modprobe iwlwifi
    sleep 2

    # Restart Proxmox networking
    systemctl restart networking
    
    echo "$(date): Recovery complete." >> /var/log/network-fix.log
fiCode language: PHP (php)

You must adjust the ip of your gateway and you might need to update some of the lines inside the script.

  • nano /usr/local/bin/network-fix.sh
  • chmod +x /usr/local/bin/network-fix.sh
  • crontab -e
  • append: */5 * * * * /usr/local/bin/network-watchdog.sh

Periodical reboot

This is a homelab approach and to make sure that the server is available and such issues like the usb adapter problem above don't prevent access, we can setup a periodical reboot for the device. With implementing this, we should be aware that 1) any process running in VMs will be interrupted and 2) VMs will not start if they are not marked to start automatically with the system.

The easy way to set this up is to use crontab:

  • crontab -e

You can set either of these, depending on the cadence you want for the restarts:

  • daily at 3am: 0 3 * * * /sbin/shutdown -r now
  • weekly on Sunday at 3am: 0 3 * * 0 /sbin/shutdown -r now
  • monthly, last Sunday at 3am: 0 3 * * 0 [ $(date -d "+7 days" +\%m) != $(date +\%m) ] && /sbin/shutdown -r now

As a sanity check after editing, crontab -l shows the active schedule, and for the monthly version you can dry‑run the condition with date -d "+7 days" +%m; date +%m on any Sunday to see whether it would have fired.

Tags