Back to feed

sledtools/pika branch #148

apple-ci-cache-redesign

Drop mutable iOS build number from cache key

Target branch: master

Merge Commit: 27b28d9a1b7e4cc44b54e3c6f7bdfcce7efb4d1e

branch: merged tutorial: ready ci: success
Open CI Details

Continuous Integration

CI: success

Compact status on the review page, with full logs on the CI page.

Open CI Details

Latest run #185 success

10 passed

head 77d58735ebbb9716e3f75cd14781e11f7853140f · queued 2026-03-27 15:47:48 · 10 lane(s)

queued 17s · ran 1m 13s

check-pika-rust · success check-pika-followup · success check-notifications · success check-agent-contracts · success check-rmp · success check-pikachat · success check-pikachat-typescript · success check-apple-host-sanity · success check-pikachat-openclaw-e2e · success check-fixture · success

Summary

This branch redesigns the Apple CI cache architecture by splitting the monolithic apple_host_sanity pre-merge lane into two independent, compile-only lanes: apple_desktop_compile and apple_ios_compile. The mutable iOS build number is dropped from the cache key in favor of deterministic Nix-built artifacts (appleDesktopCompile and appleIosXcframework packages). A new nix/ci/apple-rust.nix derivation validates workspace dependency closure at build time using cargo tree, ensuring the Nix fileset never silently omits real build dependencies. The remote runner protocol gains per-scope isolation (compile vs runtime), a stable content-addressed cache key script, and a centralized ci/apple-host-assets.toml that pins Xcode version, iOS runtime, and simulator device in one place. Supporting tooling (ios-runtime-doctor, ios-sim-ensure, apple-host-asset, xcode-run) and documentation are updated to match the new two-lane contract.

Tutorial Steps

Centralize Apple host asset pins in a TOML manifest

Intent: Replace scattered hard-coded Xcode and iOS runtime version strings with a single source of truth that both Nix expressions and shell scripts can consume.

Affected files: ci/apple-host-assets.toml, flake.nix, scripts/pikaci-apple-host-bootstrap.sh, tools/apple-host-asset

Evidence
@@ -0,0 +1,4 @@
+xcode_version = "26.2"
+ios_runtime = "iOS 26.2"
+ios_device_type = "iPhone 15"
+ios_device_name = "Pika iPhone 15"
@@ -546,8 +546,9 @@
-        xcodeVersion = "26.2";
+        appleHostAssets = builtins.fromTOML (builtins.readFile ./ci/apple-host-assets.toml);
+        xcodeVersion = appleHostAssets.xcode_version;
@@ -57,7 +57,8 @@
-xcode_version="26.2"
+xcode_version="$("$repo_root/tools/apple-host-asset" xcode_version)"
+ios_runtime="$("$repo_root/tools/apple-host-asset" ios_runtime)"

A new ci/apple-host-assets.toml file becomes the canonical pin for four Apple CI parameters: xcode_version, ios_runtime, ios_device_type, and ios_device_name.

In flake.nix, the previously hard-coded xcodeVersion = "26.2" is replaced by reading the TOML at evaluation time via builtins.fromTOML. The bootstrap script (pikaci-apple-host-bootstrap.sh) now shells out to a new tools/apple-host-asset helper to read these values instead of embedding them inline.

This means changing the Xcode or iOS simulator version for CI is now a single-file edit to ci/apple-host-assets.toml, and every consumer — Nix, shell scripts, cache key computation — picks it up automatically.

Add the Nix apple-rust.nix derivation with dependency closure validation

Intent: Create hermetic, cacheable Nix packages for desktop compilation and iOS xcframework generation that validate their workspace dependency closures at build time.

Affected files: nix/ci/apple-rust.nix, flake.nix

Evidence
@@ -0,0 +1,189 @@
+{ pkgs, rustToolchain, xcodeWrapper, src }:
+
+let
+  cargoLock = {
+    lockFile = src + "/Cargo.lock";
+    outputHashes = {
@@ -660,6 +661,82 @@
+        workspaceCrateManifestFiles = [
+          ./crates/pikaci/Cargo.toml
+          ./crates/pika-agent-protocol/Cargo.toml
@@ -938,6 +1031,10 @@
+          // pkgs.lib.optionalAttrs pkgs.stdenv.isDarwin {
+            appleDesktopCompile = appleDesktopCi.desktopCompile;
+            appleIosXcframework = appleRustCi.iosXcframework;
+          }

The new nix/ci/apple-rust.nix defines a mkAppleRustPackage builder that wraps buildRustPackage with Xcode tooling and an embedded Python dependency-closure validator.

The validator runs cargo tree over every target package and asserts that only the declared allowedWorkspacePackages appear as transitive workspace dependencies. If the Nix fileset omits a real dependency, the build fails immediately with a clear error message — catching drift between the fileset and Cargo workspace that would otherwise surface as a confusing compile error.

Two packages are exposed in flake.nix:

  • appleDesktopCompile — compiles pika-desktop test binaries via cargo-with-xcode test --no-run, using a minimal appleDesktopWorkspaceSrc fileset.
  • appleIosXcframework — runs ios-build ios-xcframework and copies out Bindings/, NSEBindings/, ShareBindings/, and Frameworks/, using the broader appleIosWorkspaceSrc fileset.

Both filesets share a workspaceCrateManifestFiles list so that Cargo.toml stubs are generated for members not included in the fileset, keeping the workspace metadata valid.

Split apple_host_sanity into apple_desktop_compile and apple_ios_compile forge lanes

Intent: Replace the single blocking pre-merge Apple lane with two independent lanes that can run concurrently under a shared concurrency group, each with precisely scoped file-trigger paths.

Affected files: crates/pikaci/src/forge_lanes.rs, crates/pikaci/src/targets.rs

Evidence
@@ -237,13 +237,18 @@
-                "apple_host_sanity",
-                "check-apple-host-sanity",
+                "apple_desktop_compile",
+                "check-apple-desktop-compile",
+                "apple_desktop_compile",
+                Some("apple-compile"),
@@ -253,17 +258,71 @@
+            pikaci_target_lane(
+                "apple_ios_compile",
+                "check-apple-ios-compile",
+                "apple_ios_compile",
+                Some("apple-compile"),
@@ -289,6 +289,20 @@
+        "apple_desktop_compile" => Ok(host_shell_target_spec(
+            "apple_desktop_compile",
+            "Run the Apple desktop compile lane via pikaci",
+            &[],
+            3600,
+            "./scripts/pikaci-apple-remote.sh run --just-recipe apple-host-desktop-compile",
+        )),
+        "apple_ios_compile" => Ok(host_shell_target_spec(

In forge_lanes.rs, the single apple_host_sanity branch lane is replaced by two lanes:

  1. apple_desktop_compile — triggers on desktop Rust crate paths, pika-desktop, pika-cloud, Nix files, and the new ci/apple-host-assets.toml. Runs just apple-host-desktop-compile.
  2. apple_ios_compile — triggers on iOS-specific paths (ios/**, pika-nse, pika-share, scripts/ios-build, simulator tooling). Runs just apple-host-ios-compile.

Both lanes share the concurrency group apple-compile, so the Mac mini schedules them without conflict. The nightly bundle lane's concurrency group is updated from apple-host to apple-runtime to avoid collision.

In targets.rs, corresponding host_shell_target_spec entries are added for both new targets, each invoking the appropriate just recipe via pikaci-apple-remote.sh. The original apple_host_sanity target spec is retained for backward compatibility.

Implement the just recipes for the two compile lanes

Intent: Wire the Nix-backed compile packages into actionable just recipes that CI can invoke, and restructure the existing sanity/bundle recipes accordingly.

Affected files: just/checks.just, justfile

Evidence
@@ -117,20 +117,54 @@
-apple-host-sanity:
+apple-host-desktop-compile:
     #!/usr/bin/env bash
     set -euo pipefail
+    ./scripts/apple-host-record-phase apple-host-desktop-compile.nix rust-nix -- \
+      nix --extra-experimental-features "nix-command flakes" build .#appleDesktopCompile --no-link
@@ -117,20 +117,54 @@
+apple-host-ios-compile:
+    result_path="$(
+      ./scripts/apple-host-record-phase apple-host-ios-compile.nix rust-nix -- \
+        nix --extra-experimental-features "nix-command flakes" build .#appleIosXcframework --print-out-paths --no-link
+    )"
+    cache_key="$(
+      PIKACI_APPLE_IOS_XCFRAMEWORK_RESULT="$result_path" \
+        ./scripts/apple-host-cache-key ios-generic-sim
+    )"
@@ -228,6 +228,8 @@
+alias apple-host-desktop-compile := checks::apple-host-desktop-compile
+alias apple-host-ios-compile := checks::apple-host-ios-compile

Two new just recipes in just/checks.just:

apple-host-desktop-compile simply runs nix build .#appleDesktopCompile --no-link inside the apple-host dev shell, recording the phase timing.

apple-host-ios-compile is more involved:

  1. Builds .#appleIosXcframework and captures the Nix store output path.
  2. Computes a content-addressed cache key via ./scripts/apple-host-cache-key ios-generic-sim.
  3. Derives a PIKA_IOS_DERIVED_DATA_PATH from that cache key so Xcode derived data is reused across identical inputs.
  4. Runs ./scripts/ios-build ios-build-generic-sim with the prebuilt xcframework result injected via PIKACI_APPLE_IOS_XCFRAMEWORK_RESULT.

The existing apple-host-sanity recipe becomes a thin wrapper calling both compile lanes. The apple-host-bundle recipe is updated to invoke the sanity wrapper first, then runs desktop-ui-test as a separate phase (previously it was the entire sanity lane), and passes the Nix-built xcframework into the iOS UI test step.

Add the content-addressed cache key script

Intent: Provide a deterministic, content-addressed cache key that replaces the mutable iOS build number previously used in cache keys.

Affected files: scripts/apple-host-cache-key

Evidence
@@ -0,0 +1,56 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+kind="${1:-}"
+
+emit_path_hashes() {
+  local path
+  for path in "$@"; do
+    if [[ -f "$path" ]]; then
+      printf 'path=%s\n' "$path"
+      shasum -a 256 "$path"
+    elif [[ -d "$path" ]]; then
+      while IFS= read -r -d '' file; do

The new scripts/apple-host-cache-key script computes a SHA-256 cache key from:

  • The kind argument (e.g., ios-generic-sim)
  • Xcode version and iOS runtime version (read from apple-host-asset)
  • Content hashes of flake.nix, flake.lock, and ci/apple-host-assets.toml
  • For ios-generic-sim: the Nix store path of the prebuilt xcframework (PIKACI_APPLE_IOS_XCFRAMEWORK_RESULT), plus content hashes of the iOS project files, build scripts, and tooling

All inputs are piped through shasum -a 256 to produce a single deterministic hash. This replaces the previous approach where the iOS build number (which changes on every commit) was baked into the cache key, causing unnecessary cache misses. Now the cache key changes only when the actual build inputs change.

Add ios-build-generic-sim subcommand and prebuilt xcframework sync

Intent: Enable the iOS build script to consume prebuilt Nix xcframework outputs and perform a generic-simulator compile without running tests.

Affected files: scripts/ios-build

Evidence
@@ -31,6 +31,44 @@
+sync_prebuilt_ios_xcframework_result() {
+  local result_path="${PIKACI_APPLE_IOS_XCFRAMEWORK_RESULT:-}"
+  if [[ -z "$result_path" ]]; then
+    return 1
+  fi
+  if [[ ! -d "$result_path/Frameworks" ]]; then
+    echo "error: prebuilt iOS xcframework result missing Frameworks/: $result_path" >&2
@@ -286,6 +328,23 @@
+run_ios_generic_simulator_xcodebuild() {
+  local derived_data_path="${PIKA_IOS_DERIVED_DATA_PATH:-ios/build-generic}"
+  ./tools/xcodebuild-compact \
+    -destination 'generic/platform=iOS Simulator' \
@@ -296,6 +355,12 @@
+run_ios_build_generic_sim() {
+  run_ios_xcframework
+  run_ios_xcodeproj
+  run_ios_generic_simulator_xcodebuild
+}

The scripts/ios-build script gains two key additions:

sync_prebuilt_ios_xcframework_result() — When PIKACI_APPLE_IOS_XCFRAMEWORK_RESULT is set, this function copies the Nix store outputs (Bindings/, NSEBindings/, ShareBindings/, Frameworks/) into the working tree, making them writable. The existing run_ios_xcframework() function is patched to short-circuit via this sync when the environment variable is present, avoiding a redundant rebuild.

run_ios_generic_simulator_xcodebuild() — A new xcodebuild invocation targeting generic/platform=iOS Simulator with CODE_SIGNING_ALLOWED=NO. Unlike the existing simulator build, this uses a configurable PIKA_IOS_DERIVED_DATA_PATH (defaulting to ios/build-generic) and is intentionally compile-only — no test execution.

ios-build-generic-sim — A new top-level subcommand composing xcframework → xcodeproj → generic simulator build.

Restructure the remote runner protocol with execution scopes

Intent: Isolate compile-scoped and runtime-scoped remote runs into separate directory trees so they maintain independent caches, locks, and prepared worktrees.

Affected files: scripts/pikaci-apple-remote.sh

Evidence
@@ -79,8 +79,28 @@
+    apple-host-desktop-compile)
+      printf '%s\n' "desktop"
+      ;;
+    apple-host-ios-compile)
+      printf '%s\n' "ios"
+      ;;
+execution_scope_for_recipe() {
+  case "$1" in
+    apple-host-bundle)
+      printf '%s\n' "runtime"
+    apple-host-desktop-compile|apple-host-ios-compile|apple-host-sanity)
+      printf '%s\n' "compile"
@@ -269,21 +291,22 @@
-remote_run_dir="${resolved_remote_root}/runs/${run_id}"
-remote_prepared_dir="${resolved_remote_root}/prepared/${resolved_commit}"
+remote_scope_root="${resolved_remote_root}/scopes/${execution_scope}"
+remote_run_dir="${remote_scope_root}/runs/${run_id}"
+remote_prepared_dir="${remote_scope_root}/prepared/${resolved_commit}"
@@ -387,13 +412,15 @@
+execution_scope="${13}"
 
-run_dir="${resolved_remote_root}/runs/${run_id}"
+scope_root="${resolved_remote_root}/scopes/${execution_scope}"
+run_dir="${scope_root}/runs/${run_id}"

The pikaci-apple-remote.sh script introduces an execution scope concept. Each just recipe maps to a scope:

  • apple-host-desktop-compile, apple-host-ios-compile, apple-host-sanitycompile
  • apple-host-bundleruntime
  • Everything else → generic

All remote paths (runs/, prepared/, repo.git, shared-target/, run.lock) are now nested under scopes/<execution_scope>/ instead of being flat under the remote root. This means compile lanes and runtime lanes have independent locks and don't block each other, and their Cargo target directories don't interfere.

The prepare profile satisfaction logic is also updated from a simple numeric rank comparison to an explicit compatibility matrix: desktop is satisfied by desktop, compile, or bundle; ios is satisfied by ios, compile, or bundle; compile requires compile or bundle; and bundle requires bundle exactly.

The run ID format is updated to include the sanitized recipe name and PID, making concurrent runs distinguishable.

Update the prepare script for new profiles and Nix-first builds

Intent: Align the apple-host-prepare.sh prewarm phases with the new lane structure, replacing cargo-based preparation with Nix builds.

Affected files: scripts/apple-host-prepare.sh

Evidence
@@ -3,7 +3,7 @@
-usage: ./scripts/apple-host-prepare.sh <generic|sanity|bundle> <phase-timings-file>
+usage: ./scripts/apple-host-prepare.sh <generic|desktop|ios|compile|bundle> <phase-timings-file>
@@ -60,6 +60,17 @@
+build_apple_desktop_compile_nix() {
+  nix --extra-experimental-features "nix-command flakes" build .#appleDesktopCompile --no-link
+}
+
+ios_xcframework_result=""
+build_apple_ios_xcframework_nix() {
+  ios_xcframework_result="$(
+    nix --extra-experimental-features "nix-command flakes" build .#appleIosXcframework --print-out-paths --no-link
+  )"
@@ -68,14 +79,18 @@
-  sanity)
-    record_phase desktop-tests-no-run \
+  desktop)
+    record_phase apple-desktop-compile-nix build_apple_desktop_compile_nix
+  ios)
+    record_phase apple-ios-xcframework-nix build_apple_ios_xcframework_nix
+  compile)
+    record_phase apple-desktop-compile-nix build_apple_desktop_compile_nix
+    record_phase apple-ios-xcframework-nix build_apple_ios_xcframework_nix

The prepare script's profile system expands from three profiles (generic, sanity, bundle) to five (generic, desktop, ios, compile, bundle).

The old sanity profile ran cargo-with-xcode test -p pika-desktop --no-run to prewarm compiled test binaries. The new desktop and ios profiles instead invoke Nix builds (appleDesktopCompile and appleIosXcframework respectively), and the compile profile runs both.

For the bundle profile, the iOS xcframework Nix build runs early in the prepare phase, and its store path is captured in ios_xcframework_result. This path is then passed via PIKACI_APPLE_IOS_XCFRAMEWORK_RESULT to the later ios-build-for-testing step, which can skip the redundant xcframework build. The standalone ios-xcframework just recipe call that previously existed in the bundle prepare phase is removed entirely.

Add iOS runtime doctor and simulator ensure tooling to bootstrap

Intent: Ensure the CI bootstrap process validates and installs the pinned iOS simulator runtime, preventing runtime-missing failures during the iOS compile lane.

Affected files: scripts/pikaci-apple-host-bootstrap.sh, tools/ios-runtime-doctor, tools/ios-sim-ensure

Evidence
@@ -221,9 +222,40 @@
+ensure_ios_runtime() {
+  if [[ "$check_only" -eq 1 ]]; then
+    say "verifying pinned iOS simulator runtime: $ios_runtime"
+  else
+    say "ensuring pinned iOS simulator runtime: $ios_runtime"
+  fi
+  (
+    cd "$repo_root"
+    export PIKA_XCODE_INSTALL_PROMPT=0
+    if [[ "$check_only" -eq 1 ]]; then
+      nix --extra-experimental-features 'nix-command flakes' \
+        develop .#apple-host \
+        -c bash -lc '
+          set -euo pipefail
+          if ! ./tools/ios-runtime-doctor --quiet-check; then

The bootstrap script gains a new ensure_ios_runtime() step that runs after Xcode and dev shell verification. In check-only mode it validates the runtime is installed; in full mode it calls ./tools/ios-runtime-doctor --ensure to install it if missing.

The tools/ios-runtime-doctor and tools/ios-sim-ensure scripts (new files in this branch) read the pinned runtime and device configuration from ci/apple-host-assets.toml via the apple-host-asset tool, then use xcrun simctl to verify/install the runtime and create the named simulator device. This completes the chain: the TOML manifest pins the versions, the bootstrap ensures they're installed, and the compile lanes consume them.

Update Rust tests to match the new lane names

Intent: Keep the Rust test suite green by updating all assertions that reference the old `apple_host_sanity` lane ID to the new split lane IDs.

Affected files: crates/pika-git/src/ci_manifest.rs, crates/pika-git/src/poller.rs, crates/pikaci/src/forge_lanes.rs, crates/pikahut/tests/guardrails.rs

Evidence
@@ -36,7 +36,8 @@
-        assert!(ids.contains(&"apple_host_sanity".to_string()));
+        assert!(ids.contains(&"apple_desktop_compile".to_string()));
+        assert!(ids.contains(&"apple_ios_compile".to_string()));
@@ -184,7 +184,8 @@
-        assert!(lane_ids.contains(&"apple_host_sanity"));
+        assert!(lane_ids.contains(&"apple_desktop_compile"));
+        assert!(lane_ids.contains(&"apple_ios_compile"));
@@ -574,8 +634,8 @@
-            .find(|lane| lane.id == "apple_host_sanity")
+            .find(|lane| lane.id == "apple_desktop_compile")
+        assert_eq!(
+            apple_lane.concurrency_group.as_deref(),
+            Some("apple-compile")
+        );
@@ -1193,7 +1193,7 @@
-    let pikaci = fs::read_to_string(root.join("crates/pikaci/src/main.rs"))?;
+    let pikaci_targets = fs::read_to_string(root.join("crates/pikaci/src/targets.rs"))?;

Four test files are updated:

  • ci_manifest.rs and poller.rs — Assertions that the manifest contains apple_host_sanity are replaced with assertions for both apple_desktop_compile and apple_ios_compile.
  • forge_lanes.rs — The lane lookup test switches from finding apple_host_sanity to apple_desktop_compile, and adds an assertion verifying the apple-compile concurrency group. The nightly lane test is also updated to verify nightly_apple_host_bundle with its apple-runtime concurrency group.
  • guardrails.rs — The pikachat Apple split guardrail test now reads from targets.rs instead of main.rs, matching a prior refactor of where the pikachat_apple_followup_jobs function lives.

Update CI documentation to reflect the new two-lane contract

Intent: Keep the ci-selectors and integration-matrix docs accurate as the authoritative reference for what each CI lane owns.

Affected files: docs/testing/ci-selectors.md, docs/testing/integration-matrix.md

Evidence
@@ -28,7 +28,8 @@
-| `check-apple-host-sanity` | `just apple-host-sanity` on the Mac mini via `./scripts/pikaci-apple-remote.sh run --just-recipe apple-host-sanity`. This is the tiny blocking Mac smoke lane: `just desktop-ui-test` only. |
+| `check-apple-desktop-compile` | `just apple-host-desktop-compile` on the Mac mini via `./scripts/pikaci-apple-remote.sh run --just-recipe apple-host-desktop-compile`. This is the blocking desktop compile/link smoke lane and is intentionally compile-only via the Nix-addressable `appleDesktopCompile` package. |
+| `check-apple-ios-compile` | `just apple-host-ios-compile` on the Mac mini via `./scripts/pikaci-apple-remote.sh run --just-recipe apple-host-ios-compile`. This is the blocking iOS compile smoke lane
@@ -67,7 +67,9 @@
-| `just apple-host-sanity` | `just desktop-ui-test` | retained non-selector Apple-host smoke lane | deterministic | pre-merge CI-owned: `check-apple-host-sanity` | host-macos, xcode | Tiny blocking Apple-host smoke lane on the Mac mini.
+| `just apple-host-desktop-compile` | `nix build .#appleDesktopCompile` | retained non-selector Apple-host desktop compile lane | deterministic | pre-merge CI-owned: `check-apple-desktop-compile` | host-macos, xcode | Blocking Mac mini desktop compile/link smoke.
+| `just apple-host-ios-compile` | `nix build .#appleIosXcframework` + generic-simulator `xcodebuild build` | retained non-selector Apple-host iOS compile lane | deterministic | pre-merge CI-owned: `check-apple-ios-compile`
+| `just apple-host-sanity` | `just apple-host-desktop-compile` + `just apple-host-ios-compile` | retained non-selector Apple-host compile bundle | deterministic | advisory/convenience

Both documentation files are comprehensively updated:

ci-selectors.md — The lane table replaces check-apple-host-sanity with the two new lanes, updates the nightly bundle description to mention desktop-ui-test as retained nightly coverage (not pre-merge), and corrects the deferred-mismatch section to reference the split compile lanes.

integration-matrix.md — The integration matrix gains rows for both compile lanes and demotes apple-host-sanity to an advisory/convenience wrapper. The "Non-Owner Entry Points" table is updated to show desktop-ui-test ownership moving from the pre-merge sanity lane to the nightly bundle. The Apple Silicon contract note and deferred asks section are reworded to describe the compile-only pre-merge contract.

Miscellaneous: .gitignore, xcode-run tool, and global __pycache__ pattern

Intent: Clean up ancillary files affected by the branch changes.

Affected files: .gitignore, tools/xcode-run

Evidence
@@ -77,6 +77,7 @@
+__pycache__/

A global __pycache__/ entry is added to .gitignore, complementing the existing directory-specific patterns. This prevents Python bytecode from the new inline Python dependency validator in apple-rust.nix (or any other Python tooling) from being accidentally committed.

The tools/xcode-run script is included in the affected files list as it's now referenced in the forge lane trigger paths for both compile lanes, ensuring changes to it correctly trigger the appropriate CI lanes.

Diff