sledtools/pika branch #81

pika-cloud-runtime-plan

Add shared Incus runtime plans

Target branch: master

Merge Commit: 410c564715293412363c6d9ed2c414e2e01cced2

branch: merged tutorial: ready ci: success

Open CI Details

Continuous Integration

CI: success

Compact status on the review page, with full logs on the CI page.

Open CI Details

Latest run #106 success

7 passed

head 2723e9c754280f47f09b8c044d8d9fc90b7a04a0 · queued 2026-03-26 00:37:07 · 7 lane(s)

queued 1m 38s · ran 1m 57s

check-notifications · success check-agent-contracts · success check-pikachat · success check-pikachat-typescript · success check-apple-host-sanity · success check-pikachat-openclaw-e2e · success check-fixture · success

Summary

This branch introduces a shared Incus runtime plan layer in the pika-cloud crate and refactors both pika-server and pikaci to use it. A new IncusRuntimePlan struct and its builder (RuntimeSpec::build_incus_plan) centralize the translation from a declarative RuntimeSpec into a concrete, validated plan for launching Incus VMs — covering identity, resource limits, mount devices, bootstrap configuration, and lifecycle paths. Comprehensive validation logic (RuntimeSpec::validate) enforces invariants such as non-empty fields, absolute normalized paths, path containment under the lifecycle root, unique mount guest paths, and consistent bootstrap request paths. The server-side agent API and the CI executor are both migrated from ad-hoc, inline Incus configuration to plan-driven construction, eliminating duplicated device-building logic, hardcoded memory limits, and inconsistent device naming. Deterministic device names are generated via FNV-1a hashing, and read-only mounts automatically receive the virtiofs I/O bus setting. New unit tests cover plan construction, validation error cases, mount planning, and round-trip serialization.

Tutorial Steps

Add the `IncusMountPlan` and `IncusRuntimePlan` structs

Intent: Define the shared data model that represents a fully resolved, ready-to-execute Incus runtime configuration including planned mount devices with deterministic names and I/O bus settings.

Affected files: crates/pika-cloud/src/incus.rs

Evidence

@@ -0,0 +1,150 @@
+pub struct IncusMountPlan {
+    pub kind: RuntimeMountKind,
+    pub source: String,
+    pub guest_path: String,
+    pub device_name: String,
+    pub read_only: bool,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub io_bus: Option<String>,
+    #[serde(default)]
+    pub required: bool,
+}

@@ -0,0 +1,150 @@
+pub struct IncusRuntimePlan {
+    pub identity: RuntimeIdentity,
+    pub incus: IncusRuntimeConfig,
+    pub resources: RuntimeResources,
+    pub lifecycle_root: String,
+    pub paths: RuntimePaths,
+    pub policies: RuntimePolicies,
+    pub bootstrap: RuntimeBootstrap,
+    #[serde(default)]
+    pub mounts: Vec<IncusMountPlan>,

A new module crates/pika-cloud/src/incus.rs introduces two key structs:

IncusMountPlan — a resolved mount device entry carrying the device name, source path, guest path, read-only flag, optional I/O bus override, and whether the mount is required.
IncusRuntimePlan — the full plan combining identity, Incus project/profile/image config, resource limits, lifecycle paths, policies, bootstrap config, resolved mounts, labels, and metadata.

The plan_mounts function converts raw RuntimeMount entries into IncusMountPlan entries, automatically setting io_bus to virtiofs for read-only mounts via the INCUS_READ_ONLY_DISK_IO_BUS constant.

Implement deterministic device naming with FNV-1a hashing

Intent: Generate short, stable, collision-resistant Incus device names from mount metadata so that device names are reproducible across runs and environments.

Affected files: crates/pika-cloud/src/incus.rs

Evidence

@@ -0,0 +1,150 @@
+fn mount_device_name(mount: &RuntimeMount) -> String {
+    let kind = mount_kind_label(mount.kind);
+    let source = sanitize_device_component(&mount.source);
+    let guest = sanitize_device_component(&mount.guest_path);
+    let seed = format!("{kind}-{source}-{guest}");
+    let digest = short_fnv1a_hex(&seed);
+    let kind_stub = kind.chars().take(6).collect::<String>();
+    format!("pk-{kind_stub}-{digest}")

@@ -0,0 +1,150 @@
+fn short_fnv1a_hex(value: &str) -> String {
+    let mut hash = 0xcbf29ce484222325u64;
+    for byte in value.as_bytes() {
+        hash ^= u64::from(*byte);
+        hash = hash.wrapping_mul(0x100000001b3);
+    }
+    format!("{:08x}", hash as u32)
+}

Device names follow the pattern pk-{kind_stub}-{hex_digest}, where:

kind_stub is the first 6 characters of a human-readable mount kind label (e.g., persis for PersistentVolume, snapsh for ReadOnlySnapshot).
hex_digest is the lower 32 bits of an FNV-1a hash computed over a seed string built from the sanitized kind, source, and guest path.

The sanitize_device_component helper lowercases alphanumeric characters and replaces everything else with hyphens, then trims leading/trailing hyphens. This ensures deterministic, filesystem-safe names like pk-persis-b09b5ac0.

Add comprehensive `RuntimeSpec::validate` with structured error types

Intent: Enforce invariants on the runtime specification before it can be turned into a plan, catching configuration errors early with descriptive, structured error variants.

Affected files: crates/pika-cloud/src/spec.rs

Evidence

@@ -69,44 +69,217 @@
+pub enum RuntimeSpecError {
+    EmptyField { field: &'static str },
+    InvalidPath { field: &'static str, path: String, reason: &'static str },
+    MismatchedStateRoot { lifecycle_root: String, state_dir: String },
+    DuplicateMountGuestPath { guest_path: String },
+    MismatchedGuestRequestPath { bootstrap_path: String, runtime_path: String },

@@ -69,44 +69,217 @@
+    pub fn validate(&self) -> Result<(), RuntimeSpecError> {
+        require_non_empty("identity.runtime_id", &self.identity.runtime_id)?;
+        ...
+        validate_absolute_normalized_path("lifecycle_root", &self.lifecycle_root)?;
+        ...
+        if self.lifecycle_root != self.paths.state_dir {
+            return Err(RuntimeSpecError::MismatchedStateRoot { ... });
+        }

The RuntimeSpecError enum provides five structured error variants:

Variant	Purpose
`EmptyField`	Required string fields must be non-empty
`InvalidPath`	Paths must be absolute and free of `..` traversal
`MismatchedStateRoot`	`lifecycle_root` must equal `paths.state_dir`
`DuplicateMountGuestPath`	No two mounts may share the same guest path
`MismatchedGuestRequestPath`	Bootstrap and paths guest request paths must agree

The validate method checks all identity and Incus config fields for emptiness, validates every path field for absolute + normalized form, ensures all sub-paths stay under lifecycle_root via ensure_under_root, detects duplicate mount guest paths using a BTreeSet, and cross-checks the optional bootstrap guest request path against paths.guest_request_path.

Implement `RuntimeSpec::build_incus_plan` as the single plan builder

Intent: Provide a single entry point that validates the spec and transforms it into a ready-to-use `IncusRuntimePlan`, replacing scattered inline construction logic.

Affected files: crates/pika-cloud/src/spec.rs

Evidence

@@ -69,44 +69,217 @@
+    pub fn build_incus_plan(&self) -> Result<IncusRuntimePlan, RuntimeSpecError> {
+        self.validate()?;
+        Ok(IncusRuntimePlan {
+            identity: self.identity.clone(),
+            incus: self.incus.clone(),
+            resources: self.resources.clone(),
+            ...
+            mounts: plan_mounts(&self.mounts)?,
+            labels: self.labels.clone(),
+            metadata: self.metadata.clone(),
+        })
+    }

build_incus_plan is the canonical way to produce an IncusRuntimePlan. It calls validate() first, then assembles the plan by cloning validated fields and delegating mount resolution to plan_mounts. Both the server (pika-server) and CI executor (pikaci) now call this method instead of manually assembling Incus device maps and config entries.

Export new types from `pika-cloud` public API

Intent: Make the new Incus plan types and the spec error type available to downstream crates without requiring them to reach into internal module paths.

Affected files: crates/pika-cloud/src/lib.rs

Evidence

@@ -1,9 +1,11 @@
+pub mod incus;
 pub mod lifecycle;
...
+pub use incus::{INCUS_READ_ONLY_DISK_IO_BUS, IncusMountPlan, IncusRuntimePlan};
...
+    RuntimeSpecError,

The incus module is made public, and three key items are re-exported at the crate root: INCUS_READ_ONLY_DISK_IO_BUS, IncusMountPlan, and IncusRuntimePlan. Additionally, RuntimeSpecError is added to the existing spec re-exports so consumers can handle validation errors without importing the spec module directly.

Refactor `pika-server` agent API to use plan-driven VM creation

Intent: Replace hardcoded device maps, memory limits, and device names in the server's managed Incus runtime provider with plan-derived values, ensuring consistency with the shared spec.

Affected files: crates/pika-server/src/agent_api.rs

Evidence

@@ -1561,6 +1562,7 @@
+        let runtime_plan = self.build_runtime_plan(vm_id, volume_name)?;

@@ -1570,15 +1572,39 @@
-        devices.insert(
-            INCUS_PERSISTENT_VOLUME_DEVICE_NAME.to_string(),
-            serde_json::json!({ ... }),
-        );
+        for mount in &runtime_plan.mounts {
+            let mut device = serde_json::Map::from_iter([ ... ]);
+            if mount.read_only { ... }
+            if let Some(io_bus) = mount.io_bus.as_deref() { ... }
+            devices.insert(mount.device_name.clone(), ...);

@@ -1785,6 +1819,45 @@
+    fn build_runtime_plan(
+        &self,
+        vm_id: &str,
+        volume_name: &str,
+    ) -> anyhow::Result<IncusRuntimePlan> {
+        RuntimeSpec { ... }
+        .build_incus_plan()
+        .context("build managed Incus runtime plan")

Key changes in the server's IncusManagedRuntimeProvider:

New build_runtime_plan method — Constructs a RuntimeSpec from the provider's resolved params (project, profile, image alias, storage pool) plus the VM ID and volume name, then calls build_incus_plan().
Device iteration replaces hardcoded insert — The old INCUS_PERSISTENT_VOLUME_DEVICE_NAME constant and its single devices.insert call are replaced by a loop over runtime_plan.mounts, building each device map dynamically with conditional readonly and io.bus keys.
Resource limits from the plan — The hardcoded INCUS_DEV_VM_MEMORY_LIMIT string constant is replaced by INCUS_DEV_VM_MEMORY_MIB: u32 = 2048, and both limits.memory and limits.cpu are now set conditionally from runtime_plan.resources.
Profile and image from the plan — The instance creation payload now reads runtime_plan.incus.profile and runtime_plan.incus.image_alias instead of self.resolved directly, ensuring the plan is the single source of truth.

Refactor `pikaci` executor to use plan-driven Incus configuration

Intent: Migrate the CI executor's remote Incus VM setup from ad-hoc device construction and hardcoded resource strings to the shared runtime plan, eliminating duplicated logic.

Affected files: crates/pikaci/src/executor/incus.rs, crates/pikaci/src/executor.rs

Evidence

@@ -63,6 +68,68 @@
+fn build_remote_incus_runtime_plan(
+    job: &JobSpec,
+    remote: &RemoteIncusContext,
+) -> anyhow::Result<IncusRuntimePlan> {

@@ -95,16 +162,17 @@
+    let runtime_plan = build_remote_incus_runtime_plan(job, remote)?;
     write_remote_incus_json(
         remote,
-        GUEST_REQUEST_PATH,
+        runtime_plan.paths.guest_request_path.as_str(),

@@ -247,25 +316,25 @@
-            remote.incus_project.as_str(),
+            runtime_plan.incus.project.as_str(),
             ...
-            "limits.cpu=2",
+            &format!("limits.cpu={}", incus_cpu_limit(&runtime_plan)),
             ...
-            "limits.memory=4GiB",
+            &format!("limits.memory={}", incus_memory_limit(&runtime_plan)),

The pikaci Incus executor receives parallel changes:

build_remote_incus_runtime_plan — Constructs a RuntimeSpec with the remote context's project, profile, image alias, instance name, a read-only snapshot mount, 2 vCPUs, 4096 MiB memory, and bootstrap config pointing to GUEST_REQUEST_PATH and REMOTE_LINUX_VM_INCUS_RUN_BINARY, then calls build_incus_plan().
snapshot_mount_plan helper — Extracts the snapshot mount from the plan by matching on MountKind::ReadOnlySnapshot and the expected guest path, replacing the old snapshot_mount_record() / snapshot_mount_device_prefix() functions.
incus_cpu_limit / incus_memory_limit helpers — Format resource limits from the plan with sensible defaults (2 vCPUs, 4096 MiB) if the plan fields are None.
ensure_remote_incus_runtime — Now builds the plan first and passes it through to configure_remote_incus_devices, which forwards it to add_snapshot_mount. The incus init command uses plan-derived project, profile, CPU/memory limits, image alias, and instance name.
build_remote_incus_process_command — Uses runtime_plan.paths.guest_request_path instead of the hardcoded GUEST_REQUEST_PATH constant for both the JSON write and launch command.

Update snapshot mount device wiring to use `IncusMountPlan`

Intent: Replace the old `PreparedOutputPayloadMountRecord`-based snapshot mount with the plan's `IncusMountPlan`, passing device name, I/O bus, and read-only flag from the plan to the actual Incus device add call.

Affected files: crates/pikaci/src/executor/incus.rs

Evidence

@@ -550,36 +619,44 @@
-fn snapshot_mount_record() -> PreparedOutputPayloadMountRecord { ... }
-fn snapshot_mount_device_prefix() -> &'static str { ... }
-fn add_snapshot_mount(remote: &RemoteIncusContext, log_path: &Path) -> anyhow::Result<()> {
-    add_declared_payload_mount(...)
+fn add_snapshot_mount(
+    remote: &RemoteIncusContext,
+    runtime_plan: &IncusRuntimePlan,
+    log_path: &Path,
+) -> anyhow::Result<()> {
+    let mount = snapshot_mount_plan(runtime_plan)?;
+    add_remote_incus_disk_device(
+        remote,
+        &mount.device_name,
+        Path::new(&mount.source),
+        &mount.guest_path,
+        mount.read_only,
+        mount.io_bus.as_deref().unwrap_or(REMOTE_LINUX_VM_INCUS_READ_ONLY_DISK_IO_BUS),
+        log_path,
+    )

The old snapshot_mount_record() and snapshot_mount_device_prefix() functions are deleted. The new add_snapshot_mount takes the IncusRuntimePlan, extracts the snapshot mount via snapshot_mount_plan, and calls add_remote_incus_disk_device with the plan's device name, source, guest path, read-only flag, and I/O bus. This ensures the device configuration is derived from the same validated plan used everywhere else.

Add unit tests for plan construction, validation, and mount planning

Intent: Verify that the new plan builder, validator, and mount planner produce correct results and reject invalid configurations with the right error variants.

Affected files: crates/pika-cloud/src/incus.rs, crates/pika-cloud/src/spec.rs, crates/pika-server/src/agent_api.rs, crates/pikaci/src/executor.rs

Evidence

@@ -0,0 +1,150 @@
+    fn read_only_mounts_use_virtiofs_and_deterministic_device_names() {
+        ...
+        assert_eq!(planned[0].device_name, "pk-snapsh-7bc0883f");

@@ -69,44 +69,217 @@
+    fn validate_rejects_duplicate_mount_paths() {
+        ...
+        assert_eq!(error, RuntimeSpecError::DuplicateMountGuestPath { ... });

@@ -69,44 +69,217 @@
+    fn validate_rejects_paths_outside_lifecycle_root() {
+        ...
+        assert_eq!(error, RuntimeSpecError::InvalidPath { ... reason: "must stay under lifecycle_root" });

@@ -4142,6 +4215,22 @@
+    fn build_runtime_plan_uses_shared_incus_spec_for_persistent_state() {
+        ...
+        assert_eq!(plan.mounts[0].device_name, "pk-persis-1c7cfe10");

@@ -2701,15 +2701,20 @@
+    fn remote_linux_incus_snapshot_mount_uses_declared_mount_contract() {
+        let mount = incus::build_snapshot_mount_plan_for_test(...).expect("mount plan");
+        assert_eq!(mount.device_name, "pk-snapsh-b34434da");

The branch adds extensive test coverage across all affected crates:

pika-cloud/src/incus.rs:

read_only_mounts_use_virtiofs_and_deterministic_device_names — verifies virtiofs I/O bus, read-only flag, and exact device name hash for a snapshot mount.
read_write_mounts_do_not_force_an_io_bus — confirms io_bus is None for read-write mounts.

pika-cloud/src/spec.rs:

build_incus_plan_preserves_validated_runtime_fields — end-to-end plan construction with mixed mount types.
validate_rejects_duplicate_mount_paths — duplicate guest path detection.
validate_rejects_paths_outside_lifecycle_root — path containment enforcement.
validate_rejects_mismatched_bootstrap_guest_request_path — bootstrap/paths consistency check.
Existing tests refactored to use a shared sample_runtime_spec() helper.

pika-server/src/agent_api.rs:

build_runtime_plan_uses_shared_incus_spec_for_persistent_state — verifies the server's plan builder produces correct identity, Incus config, resources, and mount device names.
A shared test_incus_provider() helper is introduced.

pikaci/src/executor.rs:

remote_linux_incus_snapshot_mount_uses_declared_mount_contract — updated to use the plan-based test helper, asserting MountKind, device name, I/O bus, and read-only flag.

pika-cloud-runtime-plan

CI: success

Summary

Tutorial Steps

Add the `IncusMountPlan` and `IncusRuntimePlan` structs

Implement deterministic device naming with FNV-1a hashing

Add comprehensive `RuntimeSpec::validate` with structured error types

Implement `RuntimeSpec::build_incus_plan` as the single plan builder

Export new types from `pika-cloud` public API

Refactor `pika-server` agent API to use plan-driven VM creation

Refactor `pikaci` executor to use plan-driven Incus configuration

Update snapshot mount device wiring to use `IncusMountPlan`

Add unit tests for plan construction, validation, and mount planning

Diff