How to Isolate QEMU Hosts

A practical, layered approach you can use to keep a group of QEMU guests isolated from each other and from the rest of your infrastructure. The idea is to combine Linux kernel‑level isolation mechanisms (network namespaces, cgroups, and security modules) with virtual networking tricks (bridges, VLANs, firewall rules). By stacking these controls you get defense‑in‑depth while still retaining flexibility for management.

---

1. Separate the VMs at the process level

a. Use cgroups (systemd slices or manual cgcreate)

• Create a dedicated slice for the whole set, e.g. system.slice/qemu‑group.slice.
• Assign CPU, memory, and I/O limits to the slice so the VMs can’t starve other workloads.
• Example (systemd unit snippet for a VM):

[Unit]
Description=QEMU VM – web‑app‑01
Slice=qemu-group.slice

[Service]
ExecStart=/usr/bin/qemu-system-x86_64 …   # your usual QEMU command line

b. Apply AppArmor or SELinux confinement

• Write a profile that only allows the VM process to read its own disk image, access its network tap device, and write logs.
• This prevents a compromised guest from reaching unrelated files on the host.

---

2. Isolate the network

a. Put each VM in its own network namespace

• Create a namespace per VM (or per logical group) and attach a veth pair:

ip netns add ns‑vm‑01
ip link add veth‑host‑01 type veth peer name veth‑guest‑01
ip link set veth‑guest‑01 netns ns‑vm‑01
ip netns exec ns‑vm‑01 ip addr add 192.168.100.2/24 dev veth‑guest‑01
ip netns exec ns‑vm‑01 ip link set veth‑guest‑01 up
ip link set veth‑host‑01 up

• In the QEMU command line, bind the guest NIC to the host side of the veth (-netdev tap,id=tap0,ifname=veth-host-01,script=no,downscript=no).

b. Connect the host‑side veth interfaces to a bridge that is VLAN‑segmented

• Create a Linux bridge (e.g., br‑isolated).
• Tag the bridge traffic with a dedicated VLAN ID (say 200) so it never mixes with the production LAN.

ip link add name br‑isolated type bridge
ip link set dev br‑isolated up
bridge vlan add dev br‑isolated vid 200 pvid untagged self

• Attach each veth‑host‑XX to the bridge and ensure the bridge port is also VLAN‑aware.

c. Harden the bridge with iptables/nftables rules

• Drop any traffic that tries to leave the VLAN or reach the host’s main interfaces:

# nft example
table inet filter {
    chain forward {
        type filter hook forward priority 0;
        iifname "br-isolated" oifname != "br-isolated" drop
        iifname != "br-isolated" oifname "br-isolated" drop
    }
}

• Optionally allow only specific ports (e.g., SSH to a bastion VM) by adding explicit accept rules.

---

3. Optional: Use libvirt for easier management

If you already run libvirt, you can let it create the network namespace and bridge for you:

<network>
  <name>isolated-net</name>
  <forward mode='bridge'/>
  <bridge name='br-isolated'/>
  <virtualport type='openvswitch'/>
  <vlan>
    <tag id='200'/>
  </vlan>
</network>

Then attach each domain with <interface type='network' source network='isolated-net'/>. Libvirt will automatically place the VM’s tap device inside the appropriate namespace.

---

4. Add a bastion / jump host (optional but recommended)

• Deploy a minimal VM in the same isolated network that you expose to the outside world (e.g., via a VPN or a tightly‑controlled SSH gateway).
• All admin traffic to the other VMs goes through this bastion, giving you a single choke point for logging and MFA enforcement.

---

5. Monitoring & Auditing

Layer	What to watch	Tooling suggestion
cgroup slice	CPU/memory spikes, OOM events	systemd-cgtop, journalctl -u qemu-group.slice
AppArmor/SELinux	Denied syscalls	auditd logs (ausearch -m avc)
Network namespace	Unexpected connections	ip netns exec ns‑vm‑01 ss -tulnp, tcpdump -i veth‑host‑XX
Bridge/firewall	Rule hits, dropped packets	nft list ruleset, iptables -L -v
Bastion	Login attempts	Fail2Ban, SSH logs

---

Quick “starter script” (bash)

Below is a minimal script that creates a namespace, a veth pair, attaches it to a VLAN‑tagged bridge, and launches a QEMU VM. Adjust paths, image names, and resource limits as needed.

#!/usr/bin/env bash
set -euo pipefail

VM_NAME="vm01"
NS="ns-${VM_NAME}"
BR="br-isolated"
VLAN_ID=200
HOST_IF="veth-${VM_NAME}-h"
GUEST_IF="veth-${VM_NAME}-g"
IMG="/var/lib/libvirt/images/${VM_NAME}.qcow2"

# 1. Namespace
ip netns add "${NS}"

# 2. Veth pair
ip link add "${HOST_IF}" type veth peer name "${GUEST_IF}"
ip link set "${GUEST_IF}" netns "${NS}"

# 3. Bridge (create once globally)
ip link add name "${BR}" type bridge || true
ip link set dev "${BR}" up
bridge vlan add dev "${BR}" vid "${VLAN_ID}" pvid untagged self

# 4. Attach host side to bridge
ip link set "${HOST_IF}" master "${BR}"
ip link set "${HOST_IF}" up

# 5. Configure guest side IP (example static)
ip netns exec "${NS}" ip addr add 192.168.${VLAN_ID}.10/24 dev "${GUEST_IF}"
ip netns exec "${NS}" ip link set "${GUEST_IF}" up
ip netns exec "${NS}" ip route add default via 192.168.${VLAN_ID}.1

# 6. Launch QEMU (simplified)
qemu-system-x86_64 \
  -enable-kvm \
  -m 2048 \
  -drive file="${IMG}",format=qcow2,if=virtio \
  -netdev tap,id=net0,ifname="${HOST_IF}",script=no,downscript=no \
  -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:56 \
  -cpu host \
  -snapshot \
  -daemonize

Run the script as root (or with appropriate sudo privileges). Repeat the block for each additional VM, changing VM_NAME and IP addresses.

---

TL;DR Checklist

1. Create a systemd slice / cgroup for the whole group.
2. Apply AppArmor/SELinux confinement to the QEMU processes.
3. Put each VM in its own network namespace with a dedicated veth pair.
4. Bridge those veth ends onto a VLAN‑tagged Linux bridge (e.g., VLAN 200).
5. Lock down the bridge with firewall rules so traffic stays inside the VLAN.
6. (Optional) Use libvirt to automate the above steps.
7. (Optional) Deploy a bastion VM for controlled external access.
8. Monitor cgroup, security, and network metrics continuously.

Following these steps gives you strong logical separation, resource control, and auditability for a set of QEMU hosts, while still allowing you to manage them centrally. If you need more detailed configuration for a particular component (e.g., an AppArmor profile template), just let me know!