Back to feed

sledtools/pika branch #134

ph-crane

nix: cache ph with crane deps split

Target branch: master

Merge Commit: 3d38fa7868aa14e527470eb3691ae8a1e4c8c01d

branch: merged tutorial: ready ci: failed
Open CI Details

Continuous Integration

CI: failed

Compact status on the review page, with full logs on the CI page.

Open CI Details

Latest run #169 failed

8 passed 1 failed

head 6ed2ddfa4366595ea8a50dee2b6caef88248d398 · queued 2026-03-27 00:55:23 · 9 lane(s)

queued 42s · ran 2m 11s

check-pika-rust · success check-pika-followup · success check-notifications · success check-agent-contracts · success check-rmp · success check-pikachat · success check-apple-host-sanity · failed check-pikachat-openclaw-e2e · success check-fixture · success

Summary

This branch migrates the Nix build of the ph binary from the standard rustPlatform.buildRustPackage builder to the Crane framework, splitting the build into a dependency-only phase (buildDepsOnly) and a final package phase (buildPackage). This two-phase approach lets Nix cache compiled third-party crate artifacts independently of source changes, dramatically improving incremental rebuild times. The change introduces a dedicated mkPhSrc file-set helper, removes the old mkPhPkg builder function, instantiates craneLib from the existing Rust toolchain, and wires everything together with shared argument records.

Tutorial Steps

Add a dedicated source filter for the ph package

Intent: Create a new `mkPhSrc` helper that produces a minimal source tree containing only the directories needed by the `ph` crate. This keeps the source hash stable when unrelated files change, which is important for cache hits in the Crane dependency layer.

Affected files: flake.nix

Evidence 
@@ -193,6 +193,17 @@
+      mkPhSrc = lib: lib.fileset.toSource {
+        root = ./.;
+        fileset = lib.fileset.unions [
+          ./Cargo.toml
+          ./Cargo.lock
+          ./cli
+          ./crates
+          ./rust
+          ./uniffi-bindgen
+        ];
+      };

A new top-level function mkPhSrc is added alongside the existing mkPikaciSrc. It uses lib.fileset.toSource with lib.fileset.unions to include only:

  • Cargo.toml / Cargo.lock (workspace manifest and lockfile)
  • cli/, crates/, rust/, uniffi-bindgen/ (source directories required by the ph crate graph)

This is intentionally a separate filter from mkPikaciSrc because the two packages may diverge in which directories they need, and keeping them independent avoids unnecessary cache invalidation.

Remove the legacy mkPhPkg builder

Intent: Delete the old `mkPhPkg` function that used `rustPlatform.buildRustPackage`. It is being replaced entirely by Crane-based build phases and is no longer called anywhere.

Affected files: flake.nix

Evidence 
@@ -214,27 +225,6 @@
-      mkPhPkg = pkgs: src:
-        pkgs.rustPlatform.buildRustPackage {
-          pname = "ph";
-          version = "0.1.0";
-          inherit src;
-          cargoLock = {
-            lockFile = ./Cargo.lock;
-            outputHashes = {
-              "hypernote-mdx-0.3.0" = ...
-              "mdk-core-0.7.1" = ...
-              "moq-lite-0.14.0" = ...
-            };
-          };
-          cargoBuildFlags = [ "-p" "ph" ];
-          cargoTestFlags = [ "-p" "ph" ];
-          doCheck = false;
-          nativeBuildInputs = pkgs.lib.optionals pkgs.stdenv.isDarwin [ pkgs.libiconv ];
-          meta = {
-            mainProgram = "ph";
-          };
-        };

The entire mkPhPkg function is removed. Key differences between the old builder and the new Crane-based one include:

AspectOld (buildRustPackage)New (Crane)
Dependency cachingNone (full rebuild on any src change)buildDepsOnly phase cached separately
Output hash formatCrate-name-based short keysFull git-URL-based keys (Crane convention)
Build inputsOnly libiconv on Darwinpkg-config + openssl added unconditionally

Removing this function is safe because no other flake outputs reference mkPhPkg after the per-system phPkg binding is rewritten in a later hunk.

Instantiate craneLib from the project Rust toolchain

Intent: Create a `craneLib` binding scoped to the per-system outputs so that Crane uses the exact same Rust toolchain already defined in the flake (via `rustToolchain`), ensuring consistent compiler versions across all build phases.

Affected files: flake.nix

Evidence 
@@ -472,6 +462,7 @@
+        craneLib = (crane.mkLib pkgs).overrideToolchain rustToolchain;

Inside the per-system attribute set, craneLib is created by calling crane.mkLib pkgs and then overriding its default toolchain with the flake's own rustToolchain.

This is a standard Crane pattern: mkLib produces a library set tied to nixpkgs, and overrideToolchain swaps in the project-specific Rust version so that both the dependency layer and the final build use the same compiler, linker, and standard library.

Wire up the Crane two-phase build for ph

Intent: Replace the single-step `mkPhPkg` call with a shared argument record (`phCargoCommonArgs`), a dependency-only derivation (`phCargoArtifacts`), and a final `buildPackage` derivation (`phPkg`) that consumes the cached artifacts.

Affected files: flake.nix

Evidence 
@@ -604,7 +595,36 @@
-        phPkg = mkPhPkg pkgs (mkPikaciSrc pkgs.lib);
+        phSrc = mkPhSrc pkgs.lib;
+        phCargoCommonArgs = {
+          pname = "ph";
+          version = "0.1.0";
+          src = phSrc;
+          cargoLock = ./Cargo.lock;
+          outputHashes = {
+            "git+https://github.com/futurepaul/hypernote-mdx?rev=db23b3d..." = "sha256-...";
+            "git+https://github.com/marmot-protocol/mdk?rev=ca0663e..." = "sha256-...";
+            "git+https://github.com/kixelated/moq?rev=5ad5c06..." = "sha256-...";
+          };
+          cargoExtraArgs = "--locked -p ph";
+          strictDeps = true;
+          doCheck = false;
+          nativeBuildInputs = [ pkgs.pkg-config ];
+          buildInputs = [ pkgs.openssl ] ++ pkgs.lib.optionals pkgs.stdenv.isDarwin [ pkgs.libiconv ];
+        };
+        phCargoArtifacts = craneLib.buildDepsOnly ((builtins.removeAttrs phCargoCommonArgs [ "src" ]) // {
+          pname = "ph-deps";
+          dummySrc = craneLib.mkDummySrc phCargoCommonArgs;
+        });
+        phPkg = craneLib.buildPackage (phCargoCommonArgs // {
+          cargoArtifacts = phCargoArtifacts;
+          meta = {
+            mainProgram = "ph";
+          };
+        });

This is the core of the change. Three new bindings replace the old single-line phPkg = mkPhPkg pkgs ...:

phCargoCommonArgs

A shared attribute set holding every option that both build phases need:

  • src points to the new mkPhSrc-filtered source.
  • cargoLock / outputHashes tell Crane how to fetch git dependencies (note the keys are now full git URLs instead of short crate names).
  • cargoExtraArgs = "--locked -p ph" restricts the build to the ph package within the workspace.
  • buildInputs adds openssl (and libiconv on Darwin) while nativeBuildInputs adds pkg-config so the OpenSSL sys crate can link.

phCargoArtifacts — dependency layer

craneLib.buildDepsOnly ((builtins.removeAttrs phCargoCommonArgs [ "src" ]) // {
  pname = "ph-deps";
  dummySrc = craneLib.mkDummySrc phCargoCommonArgs;
});

buildDepsOnly compiles every dependency in Cargo.lock against a dummy source tree generated by mkDummySrc. Because the dummy source is derived deterministically from Cargo.toml/Cargo.lock, the resulting derivation hash only changes when dependencies change — not when application source code changes. The real src is explicitly removed from the args and replaced by dummySrc.

phPkg — application layer

craneLib.buildPackage (phCargoCommonArgs // {
  cargoArtifacts = phCargoArtifacts;
  meta.mainProgram = "ph";
});

buildPackage receives the pre-built cargoArtifacts and compiles only the workspace-local crate code. On incremental rebuilds where only ph source changed, Nix reuses the cached phCargoArtifacts and only re-runs this final compilation step, significantly reducing build times.

Diff