VM Management¶

Compute providers manage virtual machines through the vm.Manager interface. The production implementation uses Lima (Linux virtual machines via QEMU) to create isolated Ubuntu 24.04 VMs with SSH access. VMs are provisioned on lease acceptance and torn down on settlement. Boot time is approximately 21 seconds from lease acceptance to a running VM with SSH.

Manager interface¶

type Manager interface {
    Provision(leaseHash string, res Resources, accessPubKey string) (*Info, error)
    Teardown(leaseHash string) error
    Exec(leaseHash string, command string) (*ExecResult, error)
    DialSSH(leaseHash string) (net.Conn, error)
    Get(leaseHash string) (*Info, error)
    List() []*Info
}

Method	Description
`Provision`	Create and start a VM with the specified resources. Injects the consumer's SSH public key via cloud-init. Returns VM info including the SSH local port.
`Teardown`	Stop and delete the VM via `limactl delete --force`. Called automatically on lease settlement.
`Exec`	Execute a shell command inside the VM via `limactl shell`. Returns exit code, stdout, and stderr.
`DialSSH`	Open a TCP connection to the VM's SSH port on localhost. Used by the tunnel protocol to proxy SSH sessions from remote consumers.
`Get`	Retrieve the current state of a VM by lease hash from the in-memory map.
`List`	Return all VMs managed by this provider.

Lima implementation¶

The LimaManager (vm/lima_manager.go) manages QEMU-based VMs using the limactl CLI tool from the Lima project.

Architecture¶

xe-node process (provider)
    │
    ├── LimaManager
    │     ├── limactl create (from YAML template)
    │     ├── limactl start (boots QEMU VM)
    │     ├── limactl shell (exec commands)
    │     └── limactl delete --force (teardown)
    │
    └── QEMU processes (one per active lease)
          ├── xe-<leaseHash[:12]> → Ubuntu 24.04 VM
          └── xe-<leaseHash[:12]> → Ubuntu 24.04 VM

Initialisation¶

When a provider node starts with --provide, the LimaManager is created:

mgr, err := vm.NewLimaManager(cfg.DataDir, cfg.LimactlPath)

LIMA_HOME is set to {dataDir}/lima — all VM state, disk images, and sockets live here
Template directory is {dataDir}/lima-templates — YAML templates are written here before limactl create
Images directory is {dataDir}/images — the Ubuntu cloud image is cached here
On startup, limactl --version is executed to verify the tool is available
If the Ubuntu cloud image is not present locally, it is downloaded automatically from the upstream Ubuntu cloud images server (~600 MB)
The manager logs the Lima version and advertised resource capacity

VM naming convention¶

VMs are named xe-{leaseHash[:12]} — the first 12 hex characters of the lease block hash. This keeps names short while avoiding collisions. For example, lease hash 6226fb52501a9052b533... creates a VM named xe-6226fb52501a.

Provision flow¶

When Provision(leaseHash, resources, accessPubKey) is called:

Convert access key — the accessPubKey (ed25519 public key as 64-char hex string) is converted to SSH authorized_keys format. This is the consumer's key for SSH access to the VM.

Render template — a Lima YAML template is generated referencing the local Ubuntu 24.04 cloud image with the requested resources and SSH key:

vmType: "qemu"
images:
  - location: "file:///var/lib/xe-node/images/ubuntu-24.04-x86_64.img"
    arch: "x86_64"
cpus: 1
memory: "512MiB"
disk: "5GiB"
mounts: []
containerd:
  system: false
  user: false
provision:
  - mode: system
    script: |
      #!/bin/bash
      mkdir -p /root/.ssh && chmod 700 /root/.ssh
      echo '<ssh-ed25519 key>' >> /root/.ssh/authorized_keys
      chmod 600 /root/.ssh/authorized_keys
      sed -i 's/#PermitRootLogin.*/PermitRootLogin prohibit-password/' /etc/ssh/sshd_config
      sed -i 's/#PubkeyAuthentication.*/PubkeyAuthentication yes/' /etc/ssh/sshd_config
      sed -i 's/#PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
      systemctl restart sshd

The image is referenced via file:// from the local cache at {dataDir}/images/. SSH is pre-installed in the Ubuntu cloud image — the provision script only injects the consumer's key and restarts sshd.

Write template — saved to {dataDir}/lima-templates/{vmName}.yaml
Create VM — limactl create --name={vmName} {templatePath} uses the cached Ubuntu cloud image as a copy-on-write backing file (instant, no download)
Start VM — limactl start {vmName} boots QEMU with KVM acceleration. Ubuntu boots in ~15 seconds, cloud-init injects the SSH key, and Lima confirms SSH readiness. Total: ~21 seconds.
Wait for SSH — polls limactl list --json every 2 seconds for up to 2 minutes until the VM reports an SSH port. This port is on 127.0.0.1 and is the tunnel target.
Store state — the VM info (lease hash, SSH port, status, resources) is stored in the in-memory vms map

Resource constraints¶

Resource	Minimum	Default	Description
vCPUs	1	From lease	QEMU `-smp` parameter
Memory	512 MiB	From lease	QEMU `-m` parameter
Disk	5 GiB	From lease	QCOW2 disk image size

If the lease requests less than the minimum, the minimum is used.

Teardown¶

When Teardown(leaseHash) is called (triggered by lease settlement):

Execute limactl delete --force {vmName} — this stops the QEMU process and removes all VM artifacts (disk, sockets, logs)
Delete the template file from {dataDir}/lima-templates/
Remove the entry from the in-memory vms map

Command execution¶

Exec(leaseHash, command) runs commands inside the VM via Lima's shell mechanism:

limactl shell {vmName} -- sh -c "{command}"

Returns an ExecResult with: - ExitCode — the command's exit code - Stdout — captured standard output - Stderr — captured standard error

SSH connection¶

DialSSH(leaseHash) opens a raw TCP connection to the VM's SSH port:

net.Dial("tcp", fmt.Sprintf("127.0.0.1:%d", vm.sshPort))

This is used by the tunnel protocol to proxy SSH sessions from remote consumers through libp2p streams to the VM's local SSH server.

Data types¶

Resources¶

type Resources struct {
    VCPUs    uint64 `json:"vcpus"`
    MemoryMB uint64 `json:"memory_mb"`
    DiskGB   uint64 `json:"disk_gb"`
}

Info¶

type Info struct {
    LeaseHash   string     `json:"lease_hash"`
    Status      string     `json:"status"`
    Resources   *Resources `json:"resources,omitempty"`
    CreatedAt   int64      `json:"created_at"`
    Error       string     `json:"error,omitempty"`
}

VM status values¶

Status	Description
`provisioning`	VM is being created (limactl create/start in progress)
`running`	VM is active, SSH port is available, accepting commands and tunnel connections
`stopped`	VM has been torn down after lease settlement
`error`	VM encountered a fatal error during provisioning or execution

ExecResult¶

type ExecResult struct {
    ExitCode int    `json:"exit_code"`
    Stdout   string `json:"stdout"`
    Stderr   string `json:"stderr"`
}

VM lifecycle¶

lease_accept confirmed
        │
        ▼
  ┌──────────────┐    vm_credentials     ┌──────────┐
  │ Provision    │ ──────────────────────►│ Consumer │
  │ (limactl     │    (direct msg)       └──────────┘
  │  create+start)│
  └──────────────┘
        │
        ▼
  ┌──────────────┐
  │ Running      │ ◄── SSH tunnel from consumer (via libp2p)
  │ (Ubuntu VM)  │ ◄── Exec commands via API or messaging
  └──────────────┘
        │
        │  lease expires + settle
        ▼
  ┌──────────────┐
  │ Teardown     │
  │ (limactl     │
  │  delete)     │
  └──────────────┘

Provision — triggered by autoAcceptLease() after the lease_accept block is confirmed (~21 seconds). Lima creates an Ubuntu 24.04 VM from the cached cloud image with the consumer's SSH key injected via cloud-init.
Credential delivery — the provider sends a vm_credentials direct message to the consumer with the VM's SSH connection details.
Running — the consumer can access the VM via:
- SSH tunnel through the SSH gateway (remote access via libp2p)
- Exec API via POST /vms/{lease}/exec (HTTP API)
- Direct messaging via the vm_exec protocol (libp2p)
Teardown — triggered by settleLease() after the lease_settle block is confirmed. The VM is deleted and its resources freed.

VM access methods¶

SSH gateway (recommended)¶

The SSH gateway provides transparent remote access to VMs. The consumer connects via standard SSH:

ssh -i ~/.ssh/lease_key -p 2222 <leaseHash>@<gateway-host>

The gateway authenticates using the AccessPubKey from the lease block, then tunnels the session through libp2p to the provider node, which proxies to the VM's local SSH server. See SSH Gateway & Tunnel Protocol for full details.

HTTP exec API¶

For programmatic access, the provider exposes an exec endpoint:

POST /vms/{leaseHash}/exec
Content-Type: application/json

{"command": "uname -a"}

Response:

{
    "exit_code": 0,
    "stdout": "Linux lima-xe-6226fb52501a 6.8.0-106-generic #106-Ubuntu SMP ...\n",
    "stderr": ""
}

Direct messaging (vm_exec)¶

Consumers can also execute commands via the libp2p messaging protocol:

Consumer                    Provider
   │                           │
   │  vm_exec {lease, cmd}     │
   │──────────────────────────►│
   │                           │  verify consumer identity
   │                           │  limactl shell {vm} -- sh -c {cmd}
   │  ExecResult {code, out}   │
   │◄──────────────────────────│

The provider node verifies the requesting peer owns the consumer account before executing.

Message handlers¶

The node registers VM-related message handlers:

Message	Direction	Description
`identify`	Any	Returns the node's account address
`vm_credentials`	Provider → Consumer	Delivers VM SSH connection info after provisioning
`vm_exec`	Consumer → Provider	Executes a command in the VM via limactl shell
`vm_status`	Consumer → Provider	Returns current VM info

Provider node flags¶

Providers enable compute leasing with the following CLI flags:

Flag	Type	Default	Description
`-provide`	bool	false	Enable provider mode
`-vcpus`	int	2	Number of vCPUs available for leasing
`-memory`	int	2048	Memory in MB available for leasing
`-disk`	int	20	Disk in GB available for leasing
`-ssh-port`	int	0	SSH gateway listen port (0 = disabled)
`-limactl-path`	string	limactl	Path to limactl binary

xe-node -provide -vcpus 4 -memory 8192 -disk 100 -ssh-port 2222

When -provide is set, the node:

Initialises the LimaManager with the configured limactl path.
Sets up the tunnel protocol handler for incoming SSH tunnel requests.
Advertises available resources via marketplace gossip every 5 minutes.
Watches for incoming lease blocks every 5 seconds and automatically accepts them.
Runs the settleLoop() to auto-settle expired leases every 10 seconds.

Marketplace discovery¶

Providers and consumers find each other through the marketplace gossip topic (xe/marketplace):

Consumer publishes a ResourceRequest with desired vCPUs, memory, disk, and duration.
Provider receives the request, checks available resources, and replies with a ResourceOffer including the computed cost.
Consumer receives the offer and creates the lease block.

The entire negotiation happens over gossip before any on-chain blocks are created.

Infrastructure requirements¶

Lima uses QEMU as its virtualisation backend. The host must have:

QEMU installed (qemu-system-x86_64)
Lima installed (limactl v1.0.6+)
KVM support — Lima requires /dev/kvm (hardware virtualisation extensions: Intel VT-x or AMD-V). Hosts without KVM (most cloud VPS) cannot run Lima VMs.
KVM permissions — the xe user must be able to access /dev/kvm. Since pm2's uid/gid setting drops supplementary groups, add a udev rule:
```
# /etc/udev/rules.d/99-kvm.rules
KERNEL=="kvm", MODE="0666"
```
Non-root execution — Lima refuses to run as root. The node process must run as a non-root user (e.g., the xe service user).
Disk space — the Ubuntu cloud image is ~600 MB. Each running VM uses additional disk for its QCOW2 overlay (copy-on-write from the shared base image).

KVM requirement

Standard cloud VPS instances typically do not expose hardware virtualisation. Bare-metal servers or VPS with nested virtualisation enabled are required for Lima VM support. Without KVM, the lease lifecycle (creation, acceptance, attestation, settlement) still works — only the actual VM boot fails.