Convert an old laptop into a LLM machine

With the same laptop used for the Proxmox experiment, we will directly install the latest Ubuntu version to convert the machine in a full LLM server.

The summary for the configuration is Lenovo P1 Gen 2, with an Intel i7, 32GB of RAM, and an NVIDIA T1000 GPU (4GB VRAM). We don't expect rocket speeds from it, we only aim to asses if it can decently run local LLMs for basic tasks on private data.

Create the install USB

For a good way to have multiple operating systems available for installing, we will use Ventoy on the USB stick. Given that my main driver is a macbook now and ventoy does not have an install medium for macos, we will use Virtualbox and Ubuntu Desktop.

The internet is full of tutorials for setting up VMs, so the following is a high level overview of what needs be done:

Download VirtualBox and install it;
Download the latest version of Ubuntu Desktop, create a new VM and make sure that the VM has access to usb devices;
Start the new VM and download the latest Ventoy for Linux, extract everything from the archive and then run the bash for the web, then use the browser to install Ventoy on the stick;
The last step is to go to macos and copy the operating systems we want directly on the stick.

More info:

Setup Ubuntu Server

Same as for Ventoy, there are plenty of tutorials on how to install Ubuntu Server baremetal. Make sure to select the minimal installantion and also setup your wifi connection, if you intend to use that as the main connection to the internet. Another interesting option is to import your keys from your github account, should you have one.

Make sure that the system will not suspend when the lid is closed:

sudo sed -i \
-e 's/^#\?HandleLidSwitch=.*/HandleLidSwitch=ignore/' \
-e 's/^#\?HandleLidSwitchExternalPower=.*/HandleLidSwitchExternalPower=ignore/' \
-e 's/^#\?HandleLidSwitchDocked=.*/HandleLidSwitchDocked=ignore/' \
-e 's/^#\?LidSwitchIgnoreInhibited=.*/LidSwitchIgnoreInhibited=no/' \
/etc/systemd/logind.conf && sudo systemctl restart systemd-logindCode language: JavaScript (javascript)

Then increase the font on the laptop (this is optional, depending on your computer's display):

sudo grep -q '^FONTSIZE=' /etc/default/console-setup && \
sudo sed -i 's/^FONTSIZE=.*/FONTSIZE="32x16"/' /etc/default/console-setup || \
echo 'FONTSIZE="32x16"' | sudo tee -a /etc/default/console-setup >/dev/null; \
sudo setupcon && sudo reboot && sudo systemctl enable console-setup && sudo
systemctl restart console-setupCode language: JavaScript (javascript)

Now, we need to make sure that the system gracefully shuts down when AC power is lost:

sudo apt update
sudo apt install acpid
sudo systemctl enable acpid
sudo systemctl start acpid
sudo mkdir -p /etc/acpi/events && printf 'event=ac_adapter.*\naction=/etc/acpi/ac-loss.sh\n' | sudo tee /etc/acpi/events/ac-loss >/dev/null

Then we need to create the script shutdown script:

sudo tee /etc/acpi/ac-loss.sh >/dev/null <<'EOF'
#!/bin/bash
AC_PATH="/sys/class/power_supply/AC/online"
DELAY=10

STATUS=$(cat $AC_PATH 2>/dev/null)
if [ "$STATUS" = "0" ]; then
    echo "$(date): AC unplugged, starting $DELAY second shutdown timer..."
    for ((i=0;i<DELAY;i++)); do
        sleep 1
        STATUS_NOW=$(cat $AC_PATH 2>/dev/null)
        if [ "$STATUS_NOW" = "1" ]; then
            echo "$(date): AC restored, cancelling shutdown."
            exit 0
        fi
    done
    STATUS_FINAL=$(cat $AC_PATH 2>/dev/null)
    if [ "$STATUS_FINAL" = "0" ]; then
        echo "$(date): AC lost for $DELAY seconds, shutting down!"
        /sbin/shutdown -h now "AC power lost"
    fi
fi
EOFCode language: JavaScript (javascript)

And the last step is to provide the rights and restart the acpid service:

sudo chmod +x /etc/acpi/ac-loss.sh
sudo systemctl restart acpid

A fallback for errors will be a reset every night at 5am:

sudo apt update && sudo apt install -y cron
sudo systemctl enable --now cron
(sudo crontab -l 2>/dev/null; echo '0 5 * * * /sbin/shutdown -r now') | sudo crontab -

One last thing to do, specific for this laptop is to setup closing the display on a time. To do that, we need to update the grub:

sudo nano /etc/default/grub
find GRUB_CMDLINE_LINUX_DEFAULT and append this to its value: loglevel=0 console=tty3 consoleblank=10 vt.global_cursor_default=0
sudo update-grub
sudo reboot

More info:

Setup an LLM on ubuntu

Setup the nvidia drivers:

sudo apt update && sudo apt upgrade -y
sudo apt-get purge nvidia
ubuntu-drivers devices
- from the list shown find the recommended, then install it
sudo apt install -y nvidia-driver-595-open
sudo reboot
nvidia-smi
- this should output the information about your nvidia card

Then run an LLM from the snap package. We will use Gemma 4, because of it being the latest and greatest:

snap version
sudo snap install gemma4
gemma4 status

To be able to access the LLM from other machines, we should change the bind host:

gemma4 get
- if we see 127.0.0.1 as host, we need to update the value as shown below
sudo gemma4 set http.host=0.0.0.0
sudo gemma4 set webui.http.host=0.0.0.0
gemma4 chat

For monitoring the performance, we could install and run:

sudo apt install -y nvtop
nvtop

More info: