pika-orch-incus-cleanup-8

A new migration (0024) adds a nullable TEXT column structured_pikaci_target_id to both branch_ci_run_lanes and nightly_run_lanes. The column is nullable because not every lane is a structured pikaci lane—legacy lanes and lanes that don't use pikaci remain unaffected.

The migration is registered in storage.rs as version 24 via the standard Migration struct, ensuring it runs automatically on startup.

Four structs gain structured_pikaci_target_id: Option<String>:

• PendingBranchCiLaneJob — the struct returned when claiming a queued branch CI lane.
• PendingNightlyLaneJob — the struct returned when claiming a queued nightly lane.
• BranchLaneRerunSource — the struct used to read a lane's data before creating a rerun.
• NightlyLaneRerunSource — the nightly equivalent of the rerun source.

All four use Option<String> to maintain backward compatibility with lanes that predate the migration.

This is the bulk of the mechanical change. Every SQL statement that touches lane rows is updated:

Initial queue (branch + nightly)

The INSERT INTO branch_ci_run_lanes and INSERT INTO nightly_run_lanes statements in queue_branch_ci_run_for_head and queue_nightly_run now include structured_pikaci_target_id as a bound parameter. The value comes from lane.staged_linux_target on the input lane config.

Rerun source reads

The SELECT queries in the rerun paths (rerun_branch_ci_lane and rerun_nightly_lane) now fetch structured_pikaci_target_id and map it into the BranchLaneRerunSource / NightlyLaneRerunSource structs. The rerun INSERT statements then copy the value forward so reruns inherit the structured target metadata.

Job claiming

The claim queries (claim_pending_branch_ci_lane_runs and claim_pending_nightly_lane_runs) add the column to their SELECT lists. All subsequent row.get(N) indices are shifted by one to accommodate the new column, and the new field is populated on PendingBranchCiLaneJob / PendingNightlyLaneJob.

The core execution function run_ci_command_for_head_with_heartbeat in forge.rs gains a new parameter structured_pikaci_target_id: Option<&str>. The resolution logic becomes:

1. If the explicit target ID is Some, use it directly.
2. Otherwise, fall back to staged_pikaci_target_from_command(command) which inspects the command array for known wrapper patterns.

This means lanes configured with a staged_linux_target in their lane definition will always be treated as structured, regardless of their wrapper script name. The fallback ensures pre-existing lanes without the DB column populated continue to work.

Both execute_branch_job and execute_nightly_job in ci.rs pass job.structured_pikaci_target_id.as_deref() through to this function.

Previously, ensure_structured_pikaci_output silently accepted any --output value and returned the command as-is. Now it:

1. Returns anyhow::Result<Vec<String>> instead of bare Vec<String>.
2. If --output is present, validates the next argument exists and equals "jsonl". If the value is missing, it bails with a clear error. If the value is something other than jsonl (e.g., json), it bails with a descriptive message.
3. If --output is absent, appends --output jsonl as before.

The caller in run_ci_command_for_head_with_heartbeat now uses ? to propagate the error, failing the lane early instead of running with incorrect output format.

Four new tests cover the critical behaviors:

claimed_branch_lane_keeps_structured_pikaci_target_hint

Queues a branch CI lane with sample_lane_with_target("pika_rust", "pre-merge-pika-rust"), claims it, and asserts that job.structured_pikaci_target_id is Some("pre-merge-pika-rust"). This validates the full DB round-trip for branch lanes.

claimed_nightly_lane_keeps_structured_pikaci_target_hint

Same pattern for the nightly lane path: queue → claim → assert the target ID is preserved.

explicit_structured_pikaci_target_does_not_depend_on_wrapper_name

This is the key integration test. It creates a custom wrapper script (scripts/custom-structured-lane.sh) whose name does not match the legacy pikaci naming convention. When executed with structured_pikaci_target_id: Some("pre-merge-pika-rust"), the lane is still correctly treated as structured—verifying that state root, run metadata, and log collection all work without depending on command-name heuristics.

explicit_structured_pikaci_target_rejects_non_jsonl_output

Passes --output json (not jsonl) with an explicit target ID and asserts the function returns an error containing "structured pikaci command must use \--output jsonl`". This validates the new validation logic in ensure_structured_pikaci_output`.

A new SKILL.md file documents the orchestration pattern used for multi-chunk implementation programs in the pika project. Key sections include:

• Default shape: One integration branch, one implementation worker on the critical path, one explorer/reviewer in parallel.
• Worktree rules: Run ./scripts/agent-brief in new sessions, use dedicated branches per chunk, separate worktrees for conflicting chunks.
• Staged Linux remote safety: Warnings about the shared /var/tmp/pikaci-prepared-output root causing collisions in parallel runs.
• Delegation pattern: Orchestrator → Worker → Explorer/reviewer roles with clear write-set ownership.
• Landing pattern: Forge-native flow, checkpoint before risk, push dedicated branches.
• Project-specific heuristics: Lessons distilled from prior work—prefer Nix-declared contracts over bash assembly, treat migration-only knobs as debt, inspect consumer seams before inventing new protocols, use persistent state roots for temp-worktree CI, and validate remote-authoritative paths.

This skill document serves as institutional memory, preventing the team from re-learning collision and state-root issues that surfaced during earlier pikaci cleanup iterations.