Back to feed

sledtools/pika branch #37

pika-orch-incus-cleanup-4

Persist pikaci run metadata for pika-news

Target branch: master

branch: closed tutorial: ready ci: failed
Open CI Details

Continuous Integration

CI: failed

Compact status on the review page, with full logs on the CI page.

Open CI Details

Latest run #45 failed

9 passed 1 failed

head 7ee67b97888277e02a0d1e86653df7ddfee266c9 · queued 2026-03-24 16:22:38 · 10 lane(s)

queued 2m 08s · ran 24m 31s

check-pika-rust · success check-pika-followup · success check-notifications · success check-agent-contracts · success check-rmp · success check-pikachat · success check-pikachat-typescript · success check-apple-host-sanity · failed check-pikachat-openclaw-e2e · success check-fixture · success

Summary

This branch adds persistent pikaci run metadata support to pika-news and performs a significant cleanup of the Incus VM executor in pikaci. On the pika-news side, a new pikaci_state_root function computes a per-repo directory for storing pikaci run artifacts (run.json, host/guest logs). The CI runner passes this path via PIKACI_STATE_ROOT to the pikaci subprocess, and new web API endpoints expose the persisted run records and logs. On the pikaci executor side, the dual Incus mode system (transfer vs single_host_shared) is collapsed into a single streamlined path: the transfer mode and its associated closure-import, guest filesystem preparation, and snapshot staging logic are entirely removed. The guest runner script is no longer generated inline but instead delegated to a NixOS-managed binary (pikaci-incus-run) invoked with environment variables. Mount paths are simplified, the /nix/store shared mount is dropped, and the Nix CI expression switches from a closure-transfer model to a pre-baked image approach. A companion migration plan document captures the rationale and rollout sequence.

Tutorial Steps

Introduce pikaci_state_root for per-repo run metadata storage

Intent: Define a deterministic filesystem path under the canonical git directory where pikaci run artifacts (run records, job logs) will be persisted, using a sanitized repo slug to avoid path issues.

Affected files: crates/pika-news/src/forge.rs

Evidence
@@ -100,6 +100,22 @@ impl Drop for MirrorLockGuard {
+pub fn pikaci_state_root(repo: &ForgeRepoConfig) -> PathBuf {
+    let repo_slug = repo
+        .repo
+        .chars()
+        .map(|ch| match ch {
+            'a'..='z' | 'A'..='Z' | '0'..='9' | '-' | '_' => ch,
+            _ => '-',
+        })
+        .collect::<String>();
+    canonical_git_dir(repo)
+        .parent()
+        .unwrap_or_else(|| Path::new("."))
+        .join("pikaci-state")
+        .join(repo_slug)
+}

A new public function pikaci_state_root is added to forge.rs. It computes the storage directory for pikaci run metadata by:

  1. Sanitizing the repo name into a filesystem-safe slug (replacing any character outside [a-zA-Z0-9_-] with -).
  2. Resolving the parent of the canonical git directory and appending pikaci-state/<repo-slug>.

This gives each configured forge repo an isolated directory tree where run records and job logs can accumulate across CI invocations. The function is used by both the CI runner (to set PIKACI_STATE_ROOT) and the web layer (to locate persisted data for API responses).

Pass PIKACI_STATE_ROOT environment variable to pikaci subprocesses

Intent: When a CI command targets a structured pikaci run, create the state root directory on disk and inject it into the subprocess environment so pikaci can write run metadata there.

Affected files: crates/pika-news/src/forge.rs

Evidence
@@ -324,6 +340,9 @@ where
+    let structured_pikaci_state_root = structured_pikaci_target
+        .as_ref()
+        .map(|_| pikaci_state_root(repo));
@@ -336,6 +355,11 @@ where
+        if let Some(state_root) = structured_pikaci_state_root.as_ref() {
+            fs::create_dir_all(state_root)
+                .with_context(|| format!("create pikaci state root {}", state_root.display()))?;
+            cmd.env("PIKACI_STATE_ROOT", state_root);
+        }

Inside the CI command execution path, the branch adds two steps:

  1. Compute the state root — if the command is a recognized structured pikaci target, pikaci_state_root is called to derive the directory path.
  2. Provision and inject — before spawning the child process, fs::create_dir_all ensures the directory exists, and cmd.env("PIKACI_STATE_ROOT", ...) makes it available to the pikaci binary.

This is the bridge between pika-news (which owns the filesystem layout) and pikaci (which writes run.json and log files). The existing integration test script is updated to assert PIKACI_STATE_ROOT is set and to write fixture run data under it.

Add web API endpoints for pikaci run records and logs

Intent: Expose persisted pikaci run metadata through three new capabilities: enriching the existing branch-logs response with run/log metadata, and two standalone endpoints for fetching a run record or job logs by run ID.

Affected files: crates/pika-news/src/web.rs

Evidence
@@ -645,6 +646,32 @@ struct ForgeBranchLogsResponse {
+    pikaci_run: Option<RunRecord>,
+    pikaci_log_metadata: Option<RunLogsMetadata>,
@@ -1308,6 +1335,14 @@ pub async fn serve(
+        .route(
+            "/news/api/forge/pikaci/run/:run_id",
+            get(api_forge_pikaci_run_handler),
+        )
+        .route(
+            "/news/api/forge/pikaci/logs/:run_id",
+            get(api_forge_pikaci_logs_handler),
+        )

The web layer gains several new components:

Enriched branch logs response

ForgeBranchLogsResponse is extended with optional pikaci_run (the full RunRecord) and pikaci_log_metadata (a summary of which log files exist per job). When the CI lane has a pikaci_run_id, the handler loads both from disk and includes them in the JSON response.

Standalone run record endpoint

GET /news/api/forge/pikaci/run/:run_id returns the deserialized run.json for a given pikaci run. Returns 404 if the run doesn't exist on disk.

Standalone logs endpoint

GET /news/api/forge/pikaci/logs/:run_id?job=<job_id>&kind=<host|guest|both> reads the actual log file contents and returns them as JSON strings. The ForgePikaciLogsQuery struct supports filtering by job ID and log kind (host, guest, or both).

All endpoints delegate to helper functions (load_pikaci_run, load_pikaci_logs, etc.) that resolve the forge repo config, compute the state root, and call into the pikaci crate's load_run_record / load_logs / load_logs_metadata functions. Authentication is enforced on all new endpoints.

Remove the dual Incus mode system (transfer vs single_host_shared)

Intent: Eliminate the transfer-based Incus execution path entirely, collapsing the executor to a single streamlined mode that uses host-shared virtiofs mounts and a NixOS-managed guest runner binary.

Affected files: crates/pikaci/src/executor.rs

Evidence
@@ -81,7 +81,6 @@ struct GuestRunnerConfig {
-    incus_mode: RemoteLinuxVmIncusMode,
@@ -97,22 +96,6 @@ struct RemoteLinuxVmContext {
-    remote_incus_closure_dir: PathBuf,
-
-#[derive(Clone, Copy, Debug, Eq, PartialEq)]
-enum RemoteLinuxVmIncusMode {
-    Transfer,
-    SingleHostShared,
-}
@@ -2987,18 +2944,7 @@ fn ensure_remote_incus_runtime(
-    match remote.incus_mode {
-        RemoteLinuxVmIncusMode::Transfer => {
-            import_remote_incus_closures(remote, log_path)?;
-            prepare_remote_incus_guest_filesystem(job, remote, log_path)?;
-            stage_snapshot_into_incus_guest(remote, log_path)?;
-            finalize_remote_incus_transfer_workspace(job, remote, log_path)
-        }
-        RemoteLinuxVmIncusMode::SingleHostShared => {
-            prepare_remote_incus_guest_filesystem(job, remote, log_path)
-        }
-    }
+    wait_for_remote_incus_instance(remote, log_path)

This is the largest change in the branch. The RemoteLinuxVmIncusMode enum and all code paths gated on it are removed:

  • RemoteLinuxVmIncusMode enum, its as_str() method, and the REMOTE_LINUX_VM_INCUS_MODE_ENV / remote_linux_vm_incus_mode() configuration are deleted.
  • incus_mode field removed from RemoteLinuxVmContext.
  • Transfer-only functions removed: import_remote_path_closure_into_incus, import_remote_incus_closures, stage_snapshot_into_incus_guest, finalize_remote_incus_transfer_workspace, build_remote_incus_transfer_workspace_finalize_command.
  • Guest filesystem preparation removed: prepare_remote_incus_guest_filesystem and build_remote_incus_prepare_script (the large inline script that generated /usr/local/bin/pikaci-incus-run inside the guest).
  • configure_remote_incus_single_host_shared_devices renamed to configure_remote_incus_devices since there's only one mode now.
  • ensure_remote_incus_runtime simplified to: create instance → configure devices → start → wait for ready. No more conditional branches.
  • remote_incus_closure_dir removed from context and from the directory creation command.

The net effect is ~280 lines of deleted executor code, removing a complex dual-path that was a maintenance burden.

Delegate guest command execution to NixOS-managed pikaci-incus-run binary

Intent: Instead of generating a guest runner script inline and pushing it into the VM, pass the guest command and configuration as environment variables to a binary that's pre-installed in the NixOS-based VM image.

Affected files: crates/pikaci/src/executor.rs

Evidence
@@ -2815,7 +2771,9 @@ fn build_remote_microvm_launch_command(remote: &RemoteLinuxVmContext) -> String
-fn build_remote_incus_launch_command(remote: &RemoteLinuxVmContext) -> String {
+fn build_remote_incus_launch_command(job: &JobSpec, remote: &RemoteLinuxVmContext) -> String {
+    let (guest_command, run_as_root) = compiled_guest_command(job);
+    let run_as_root_value = if run_as_root { "1" } else { "0" };
@@ -2824,7 +2782,11 @@ fn build_remote_incus_launch_command(remote: &RemoteLinuxVmContext) -> String {
-                "/usr/local/bin/pikaci-incus-run",
+                "env",
+                &format!("PIKACI_INCUS_GUEST_COMMAND={guest_command}"),
+                &format!("PIKACI_INCUS_TIMEOUT_SECS={}", job.timeout_secs),
+                &format!("PIKACI_INCUS_RUN_AS_ROOT={run_as_root_value}"),
+                REMOTE_LINUX_VM_INCUS_RUN_BINARY,

build_remote_incus_launch_command now takes a &JobSpec parameter in addition to the remote context. Instead of calling a script that was dynamically generated and pushed into the guest, the launch command:

  1. Compiles the guest command and determines whether it needs root.
  2. Passes PIKACI_INCUS_GUEST_COMMAND, PIKACI_INCUS_TIMEOUT_SECS, and PIKACI_INCUS_RUN_AS_ROOT as environment variables via env.
  3. Invokes /run/current-system/sw/bin/pikaci-incus-run — a binary that's baked into the NixOS VM image.

This moves the guest runner logic from a fragile inline bash heredoc into a proper NixOS-managed binary, making it testable and versionable independently of the executor code.

Simplify Incus mount paths and remove /nix/store shared mount

Intent: Consolidate mount path constants to reflect the single execution mode and remove the no-longer-needed /nix/store passthrough mount.

Affected files: crates/pikaci/src/executor.rs

Evidence
@@ -238,16 +221,14 @@ const PREPARED_OUTPUT_FULFILLMENT_SSH_REMOTE_WORK_DIR_DEFAULT: &str =
-const REMOTE_LINUX_VM_INCUS_MODE_ENV: &str = "PIKACI_REMOTE_LINUX_VM_INCUS_MODE";
-const REMOTE_LINUX_VM_INCUS_MODE_DEFAULT: &str = "single_host_shared";
+const REMOTE_LINUX_VM_INCUS_RUN_BINARY: &str = "/run/current-system/sw/bin/pikaci-incus-run";
-const REMOTE_LINUX_VM_INCUS_SHARED_SNAPSHOT_MOUNT_PATH: &str = "/mnt/pikaci-snapshot";
-const REMOTE_LINUX_VM_INCUS_SHARED_NIX_STORE_MOUNT_PATH: &str = "/mnt/pikaci-nix-store";
-const REMOTE_LINUX_VM_INCUS_SHARED_WORKSPACE_DEPS_MOUNT_PATH: &str = "/mnt/pikaci-workspace-deps";
-const REMOTE_LINUX_VM_INCUS_SHARED_WORKSPACE_BUILD_MOUNT_PATH: &str = "/mnt/pikaci-workspace-build";
+const REMOTE_LINUX_VM_INCUS_SNAPSHOT_MOUNT_PATH: &str = "/workspace/snapshot";
+const REMOTE_LINUX_VM_INCUS_WORKSPACE_DEPS_MOUNT_PATH: &str = "/staged/linux-rust/workspace-deps";
+const REMOTE_LINUX_VM_INCUS_WORKSPACE_BUILD_MOUNT_PATH: &str = "/staged/linux-rust/workspace-build";

Mount path constants are renamed and simplified:

Old constantNew constant
REMOTE_LINUX_VM_INCUS_SHARED_SNAPSHOT_MOUNT_PATH (/mnt/pikaci-snapshot)REMOTE_LINUX_VM_INCUS_SNAPSHOT_MOUNT_PATH (/workspace/snapshot)
REMOTE_LINUX_VM_INCUS_SHARED_WORKSPACE_DEPS_MOUNT_PATH (/mnt/pikaci-workspace-deps)REMOTE_LINUX_VM_INCUS_WORKSPACE_DEPS_MOUNT_PATH (/staged/linux-rust/workspace-deps)
REMOTE_LINUX_VM_INCUS_SHARED_WORKSPACE_BUILD_MOUNT_PATH (/mnt/pikaci-workspace-build)REMOTE_LINUX_VM_INCUS_WORKSPACE_BUILD_MOUNT_PATH (/staged/linux-rust/workspace-build)

The SHARED_NIX_STORE_MOUNT_PATH (/mnt/pikaci-nix-store) and its corresponding add_remote_incus_disk_device call are removed entirely — the NixOS VM image already has its own /nix/store, so the host passthrough is unnecessary.

The new paths match the guest binary's expectations, placing the snapshot directly at /workspace/snapshot rather than requiring symlinks from /mnt/ paths.

Clean up executor tests with shared helper constructors

Intent: Reduce test verbosity by extracting sample_shell_job and sample_remote_context helpers, and remove tests for deleted transfer-mode functionality.

Affected files: crates/pikaci/src/executor.rs

Evidence
@@ -4108,36 +3753,6 @@ mod tests {
-    fn with_incus_mode_env<T>(value: Option<&str>, action: impl FnOnce() -> T) -> T {
@@ -4151,6 +3766,44 @@ mod tests {
+    fn sample_shell_job(command: &'static str) -> JobSpec {
+    fn sample_remote_context(backend: RemoteLinuxVmBackend) -> RemoteLinuxVmContext {

The test module is updated to match the simplified executor:

  • with_incus_mode_env helper deleted (no more mode env var to test).
  • sample_shell_job and sample_remote_context helpers added to replace the verbose inline RemoteLinuxVmContext / JobSpec construction that was duplicated across many tests.
  • Tests for build_remote_incus_prepare_script, build_remote_incus_transfer_workspace_finalize_command, and the single-host-shared prepare script are removed since those functions no longer exist.
  • The remaining Incus tests (remote_linux_incus_launch_uses_incus_exec_runner, remote_linux_incus_read_only_disk_device_uses_virtiofs_bus) are updated to use the new helpers and assert the new environment-variable-based launch command.

Add pikaci run metadata persistence to main executor

Intent: When PIKACI_STATE_ROOT is set, write the run record (run.json) and job logs into the state directory after execution completes.

Affected files: crates/pikaci/src/main.rs

Evidence
@@ -20,12 +20,16 @@ use pikaci::{
+    persist_run_record,
@@ -318,6 +322,14 @@ fn do_main() -> anyhow::Result<()> {
+    if let Ok(state_root) = std::env::var("PIKACI_STATE_ROOT") {
+        let state_root = PathBuf::from(state_root);
+        if let Err(err) = persist_run_record(&state_root, &run) {
+            eprintln!("[pikaci] WARNING: failed to persist run record: {err:#}");
+        }
+        if let Err(err) = persist_job_logs(&state_root, &run) {
+            eprintln!("[pikaci] WARNING: failed to persist job logs: {err:#}");
+        }
+    }

At the end of do_main(), after the run completes, the executor checks for the PIKACI_STATE_ROOT environment variable. If present:

  1. persist_run_record writes <state_root>/runs/<run_id>/run.json containing the serialized RunRecord.
  2. persist_job_logs copies each job's host and guest log files into <state_root>/runs/<run_id>/jobs/<job_id>/.

Both operations are wrapped in soft error handling — failures are logged as warnings but don't abort the process. This ensures the run output is always reported even if persistence fails (e.g., due to a disk issue).

Implement persistence and loading functions in pikaci library

Intent: Provide the public API for writing and reading pikaci run records and logs from the state root directory structure.

Affected files: crates/pikaci/src/main.rs

Evidence
@@ -20,12 +20,16 @@ use pikaci::{
+    persist_run_record,
+    persist_job_logs,
+    load_run_record, load_logs, load_logs_metadata, LogKind, RunLogsMetadata, RunRecord,

The pikaci crate exposes several new public functions (imported by both main.rs and pika-news/web.rs):

  • persist_run_record(state_root, run) — serializes the run record to <state_root>/runs/<run_id>/run.json.
  • persist_job_logs(state_root, run) — copies host/guest log files from their original locations into the state directory tree.
  • load_run_record(state_root, run_id) — deserializes a RunRecord from disk.
  • load_logs(state_root, run_id, job_id, kind) — reads log file contents, filtered by job ID and log kind.
  • load_logs_metadata(state_root, run_id, job_id) — returns a RunLogsMetadata summary indicating which log files exist and their sizes without reading the full contents.
  • LogKind enum (Host, Guest, Both) and RunLogsMetadata / RunRecord types are re-exported for use by consumers.

The directory layout follows the convention:

<state_root>/runs/<run_id>/run.json
<state_root>/runs/<run_id>/jobs/<job_id>/host.log
<state_root>/runs/<run_id>/jobs/<job_id>/guest.log

Update NixOS CI expression and VM image for new execution model

Intent: Align the Nix build infrastructure with the simplified Incus executor: remove closure-transfer tooling and ensure the pikaci-incus-run binary is available in the VM image.

Affected files: nix/ci/linux-rust.nix, nix/incus/pikaci-image.nix

Evidence
diff --git a/nix/ci/linux-rust.nix
diff --git a/nix/incus/pikaci-image.nix

Two Nix expressions are updated:

nix/ci/linux-rust.nix

The CI expression is adjusted to remove any references to the closure-transfer build steps that were part of the old transfer mode. The staged outputs now assume the single-host-shared model where dependencies are mounted via virtiofs.

nix/incus/pikaci-image.nix

The VM image definition ensures that pikaci-incus-run is available at /run/current-system/sw/bin/pikaci-incus-run, matching the REMOTE_LINUX_VM_INCUS_RUN_BINARY constant in the executor. This binary encapsulates all the guest-side setup logic (user switching, environment configuration, timeout wrapping, result JSON writing) that was previously generated as an inline script.

Add Incus migration plan documentation

Intent: Capture the design rationale, rollout sequence, and verification criteria for the Incus cleanup migration in a persistent document.

Affected files: docs/incus-migration-plan.md

Evidence
diff --git a/docs/incus-migration-plan.md

A new docs/incus-migration-plan.md file documents the migration from the dual-mode Incus executor to the simplified single-mode approach. This serves as a reference for:

  • Why the transfer mode was removed (complexity, performance overhead of nix-store exports, fragility of inline script generation).
  • What the new model looks like (virtiofs mounts, NixOS-managed guest binary, environment variable passing).
  • How to roll out (the sequencing of image updates, executor changes, and verification steps).

This is particularly valuable since the change touches the CI infrastructure's execution model, and the document provides context for anyone debugging or extending the system later.

Add comprehensive integration and unit tests for persisted metadata

Intent: Verify end-to-end that pikaci run metadata is correctly written during CI execution and correctly served through the web API.

Affected files: crates/pika-news/src/forge.rs, crates/pika-news/src/web.rs

Evidence
@@ -1566,6 +1597,11 @@ mod tests {
+        let state_root = super::pikaci_state_root(&forge_repo);
+        assert!(state_root.join("runs/pikaci-run-123/run.json").is_file());
+        assert!(state_root
+            .join("runs/pikaci-run-123/jobs/job-one/host.log")
+            .is_file());
@@ -5802,6 +5971,127 @@ mod tests {
+    async fn api_forge_branch_logs_includes_persisted_pikaci_run_metadata() {
@@ +5971,127 @@ mod tests {
+    async fn api_forge_pikaci_handlers_load_persisted_run_and_logs() {

The branch adds three key test scenarios:

Forge integration test (forge.rs)

The existing CI integration test (which uses a shell script stub) is extended to:

  • Assert that PIKACI_STATE_ROOT is required and set.
  • Write fixture run.json, host.log, and guest.log files from within the test script.
  • Verify after execution that state_root/runs/pikaci-run-123/run.json and state_root/runs/pikaci-run-123/jobs/job-one/host.log exist on disk.

Branch logs API test (web.rs)

api_forge_branch_logs_includes_persisted_pikaci_run_metadata sets up a full branch + CI lane + pikaci run ID chain, writes fixture data using write_pikaci_run_fixture, then calls the branch logs handler and asserts the response includes pikaci_run.run_id and pikaci_log_metadata.jobs[0].host_log_exists.

Standalone pikaci API tests (web.rs)

api_forge_pikaci_handlers_load_persisted_run_and_logs tests both the run-record and logs endpoints directly, verifying that run_id, job, host, and guest fields are correctly populated from fixture data.

Test helpers forge_test_config_with_git_dir and write_pikaci_run_fixture are introduced to reduce boilerplate across these tests.

Diff