Back to feed

sledtools/pika branch #92

pika-orch-incus-cleanup-21

Introduce staged payload specs

Target branch: master

Merge Commit: 933b7e5b6e2018c21a5e5d47b19d76ceda0198f7

branch: merged tutorial: ready ci: success
Open CI Details

Continuous Integration

CI: success

Compact status on the review page, with full logs on the CI page.

Open CI Details

Latest run #118 success

6 passed

head b5d4ff8dedf10de129f23865c1f442596de521a3 · queued 2026-03-26 01:30:40 · 6 lane(s)

queued 7s · ran 30s

check-notifications · success check-agent-contracts · success check-pikachat · success check-pikachat-typescript · success check-pikachat-openclaw-e2e · success check-fixture · success

Summary

This branch refactors the staged Linux Rust payload system in pikaci by introducing a dedicated StagedPreparedPayload specification struct and a factory function that centralizes payload metadata construction. Previously, payload metadata (node IDs, installables, output names, descriptions) was computed inline during run-plan construction, interleaving spec derivation with plan-graph wiring. The new design extracts all payload specification logic into a standalone staged_linux_payload_specs function that produces fully-described StagedPreparedPayload values upfront, then the plan builder simply iterates over those specs. This separation makes the payload contract explicit, reduces duplication between the HostContext mount list and the prepare-node loop, and makes it straightforward to test or extend payload definitions independently of plan assembly.

Tutorial Steps

Rename and extend the payload mount struct to StagedPreparedPayload

Intent: Replace the narrow StagedPayloadMount (which only carried a local path and device prefix) with a richer StagedPreparedPayload that also captures the prepare node ID, Nix installable, output name, and human-readable description. This makes the struct a self-contained specification of everything needed to prepare and mount a staged payload.

Affected files: crates/pikaci/src/executor.rs

Evidence 
@@ -27,9 +27,13 @@ mod incus;
-pub struct StagedPayloadMount {
+pub struct StagedPreparedPayload {
+    pub prepare_node_id: String,
+    pub installable: String,
+    pub output_name: &'static str,
     pub local_mount_path: PathBuf,
     pub device_prefix: String,
+    pub prepare_description: String,
 }
@@ -45,7 +49,7 @@ pub struct HostContext {
-    pub staged_payload_mounts: Vec<StagedPayloadMount>,
+    pub staged_payloads: Vec<StagedPreparedPayload>,

The core data model change renames StagedPayloadMount to StagedPreparedPayload and adds four new fields:

FieldTypePurpose
prepare_node_idStringUnique ID for the prepare node in the execution plan graph
installableStringFull Nix installable URI (e.g. path:/snapshot#ci.x86_64-linux.workspaceDeps)
output_name&'static strNix output attribute name
prepare_descriptionStringHuman-readable description used in plan records

The existing local_mount_path and device_prefix fields are preserved. On HostContext, the field is renamed from staged_payload_mounts to staged_payloads to reflect the broader scope of the struct.

Update internal references in executor and incus modules

Intent: Propagate the struct and field rename through the executor internals: the staged_payload_remote_dirs helper and the Incus device configuration loop both need to use the new field name.

Affected files: crates/pikaci/src/executor.rs, crates/pikaci/src/executor/incus.rs

Evidence 
@@ -1910,13 +1914,13 @@ fn staged_payload_remote_dirs(
-    ctx.staged_payload_mounts
+    ctx.staged_payloads
         .iter()
-        .map(|mount| {
+        .map(|payload| {
             remote_job_path(
                 &ctx.job_dir,
                 &shared.remote_job_dir,
-                &mount.local_mount_path,
+                &payload.local_mount_path,
@@ -839,9 +839,9 @@ fn configure_remote_incus_devices(
-        for staged_mount in &ctx.staged_payload_mounts {
+        for staged_payload in &ctx.staged_payloads {
             let source_root = staged_payload_source_root(
-                &staged_mount.local_mount_path,
+                &staged_payload.local_mount_path,

Two consuming sites are updated mechanically:

  1. staged_payload_remote_dirs (executor.rs:1914) — iterates ctx.staged_payloads instead of ctx.staged_payload_mounts, binding the iterator variable as payload rather than mount.

  2. configure_remote_incus_devices (executor/incus.rs:839) — the loop variable becomes staged_payload, and both local_mount_path and device_prefix are read from it. No behavioral change; this is purely a rename pass.

Introduce the staged_linux_payload_specs factory function

Intent: Extract payload specification construction into a single pure function that takes a snapshot directory, job directory, and lane, and returns a Vec<StagedPreparedPayload>. This centralizes the logic that was previously split between mount creation and prepare-node creation inside build_run_plan.

Affected files: crates/pikaci/src/run.rs

Evidence 
@@ -1734,6 +1709,31 @@
+fn staged_linux_payload_specs(
+    snapshot_dir: &Path,
+    job_dir: &Path,
+    lane: StagedLinuxRustLane,
+) -> Vec<StagedPreparedPayload> {
+    let prefix = lane.shared_prepare_node_prefix();
+    lane.payload_roles()
+        .into_iter()
+        .map(|role| StagedPreparedPayload {
+            prepare_node_id: format!("prepare-{prefix}-{}", role.prepare_node_suffix()),
+            installable: staged_linux_rust_installable(snapshot_dir, lane, role),
+            output_name: lane.payload_output_name(role),
+            local_mount_path: job_dir
+                .join("staged-linux-rust")
+                .join(role.mount_dir_name()),
+            device_prefix: role.device_prefix().to_string(),
+            prepare_description: format!(
+                "{} for {}",
+                role.prepare_description(),
+                lane.shared_prepare_description()
+            ),
+        })
+        .collect()
+}

The new staged_linux_payload_specs function at run.rs:1709 is the heart of the refactor. It:

  1. Derives the prepare_node_id by combining the lane's shared prefix with each role's suffix.
  2. Builds the full Nix installable string via the existing staged_linux_rust_installable helper.
  3. Computes output_name, local_mount_path, device_prefix, and prepare_description from the lane and role.

All of this was previously done in two separate places inside build_run_plan: once when building staged_payload_mounts and again when constructing prepare nodes. Now a single call produces the complete specification list.

Simplify build_run_plan to consume pre-built payload specs

Intent: Replace the inline payload construction in build_run_plan with a call to staged_linux_payload_specs, then iterate the resulting specs to wire up prepare nodes. This eliminates the redundant lookup that previously cross-referenced mounts by device_prefix and removes the duplicated derivation of node IDs, installables, and descriptions.

Affected files: crates/pikaci/src/run.rs

Evidence 
@@ -1442,6 +1442,10 @@
+        let staged_payloads = job
+            .staged_linux_rust_lane()
+            .map(|lane| staged_linux_payload_specs(&snapshot.snapshot_dir, &job_dir, lane))
+            .unwrap_or_default();
@@ -1467,88 +1471,59 @@
-            staged_payload_mounts: job
-                .staged_linux_rust_lane()
-                .map(|lane| {
-                    lane.payload_roles()
-                        .into_iter()
-                        .map(|role| StagedPayloadMount {
-                            local_mount_path: job_dir
-                                .join("staged-linux-rust")
-                                .join(role.mount_dir_name()),
-                            device_prefix: role.device_prefix().to_string(),
-                        })
-                        .collect()
-                })
-                .unwrap_or_default(),
+            staged_payloads: staged_payloads.clone(),
@@ -1476,7 +1480,7 @@
-        if let Some(lane) = job.staged_linux_rust_lane() {
-            let prefix = lane.shared_prepare_node_prefix();
+        if job.staged_linux_rust_lane().is_some() {
             let mut previous_node_id = None;
-            for role in lane.payload_roles() {
+            for payload in &staged_payloads {

The plan builder now follows a cleaner two-phase pattern:

Phase 1 — Specification (before HostContext construction):

let staged_payloads = job
    .staged_linux_rust_lane()
    .map(|lane| staged_linux_payload_specs(...))
    .unwrap_or_default();

Phase 2 — Plan wiring (the prepare-node loop):

for payload in &staged_payloads {
    // reads payload.prepare_node_id, payload.installable, etc.
}

Key simplifications:

  • The lane binding is no longer needed inside the loop — the guard changes from if let Some(lane) to if job.staged_linux_rust_lane().is_some().
  • The find() lookup that matched mounts by device_prefix is eliminated entirely, since each StagedPreparedPayload already carries its own local_mount_path.
  • prepare_node_id, installable, output_name, and prepare_description are read directly from the spec rather than re-derived from lane/role methods.

The net diff removes roughly 30 lines of inline construction and cross-referencing logic.

Update tests to use the new struct and field names

Intent: Ensure all test HostContext constructions and the staged-payload integration test use StagedPreparedPayload with the new fields, keeping the test suite green.

Affected files: crates/pikaci/src/executor.rs

Evidence 
@@ -2449,7 +2453,7 @@ mod tests {
-        RemoteLinuxVmSharedContext, StagedPayloadMount, attach_remote_linux_vm_execution,
+        RemoteLinuxVmSharedContext, StagedPreparedPayload, attach_remote_linux_vm_execution,
@@ -2918,7 +2922,7 @@
-            staged_payload_mounts: Vec::new(),
+            staged_payloads: Vec::new(),
@@ -3632,18 +3636,26 @@
-            staged_payload_mounts: vec![
-                StagedPayloadMount {
+            staged_payloads: vec![
+                StagedPreparedPayload {
+                    prepare_node_id: "prepare-pika-core-linux-rust-workspace-deps".to_string(),
+                    installable: "path:/tmp/run/snapshot#ci.x86_64-linux.workspaceDeps".to_string(),
+                    output_name: "ci.x86_64-linux.workspaceDeps",
                     local_mount_path: PathBuf::from(
                         "/tmp/run/jobs/job/staged-linux-rust/workspace-deps",
                     ),
                     device_prefix: "workspace-deps".to_string(),
+                    prepare_description: "Build staged Linux Rust dependencies for pika_core staged Linux Rust lane".to_string(),
                 },

All test HostContext instances are updated:

  • Import rename: StagedPayloadMount → StagedPreparedPayload.
  • Empty-vec tests (8 occurrences): staged_payload_mounts: Vec::new() → staged_payloads: Vec::new(). These cover host-local, remote-VM, and dev-env test scenarios.
  • Non-empty test (staged_payload_remote_dirs test at line 3636): Both entries are expanded with the new fields (prepare_node_id, installable, output_name, prepare_description), providing concrete values that match the expected naming conventions (e.g. "prepare-pika-core-linux-rust-workspace-deps").

This validates that the enriched struct carries realistic data and that downstream consumers (remote directory resolution, Incus device configuration) continue to function correctly with the new shape.

Diff