Back to feed

sledtools/pika branch #88

pika-git-ci-store-1

Extract CI persistence into ci_store

Target branch: master

Merge Commit: 80225f36182757f28106e0365aa911e12e260528

branch: merged tutorial: ready ci: success
Open CI Details

Continuous Integration

CI: success

Compact status on the review page, with full logs on the CI page.

Open CI Details

Latest run #114 success

2 passed

head c202ff4406466fe08f6d927d0edcedad99e93ba6 · queued 2026-03-26 01:11:30 · 2 lane(s)

queued 11s · ran 29s

check-notifications · success check-agent-contracts · success

Summary

This branch extracts all CI persistence logic—covering branch CI runs, nightly runs, lane management, CI target health tracking, and the associated worker claim/heartbeat/finish lifecycle—out of branch_store.rs and into a dedicated ci_store.rs module. The refactoring removes roughly 1,800 lines of CI-specific structs, queries, and helper functions from branch_store.rs, re-homes them in ci_store.rs as a standalone impl Store block, and updates all call sites in ci.rs, forge_service.rs, web.rs, and main.rs to import from the new module. No behavioral changes are introduced; this is a pure separation-of-concerns restructuring that makes the branch store focused on branch lifecycle (upsert, merge, close, artifact generation) while the CI store owns everything related to CI execution persistence.

Tutorial Steps

Remove CI-related structs and constants from branch_store.rs

Intent: Slim down `branch_store.rs` by deleting all CI-specific data types (BranchCiRunRecord, BranchCiLaneRecord, PendingBranchCiLaneJob, NightlyFeedItem, NightlyRunRecord, NightlyLaneRecord, PendingNightlyLaneJob, and the internal rerun/recovery source structs) along with the `CI_LANE_LEASE_LOST` constant and the now-unnecessary imports they required.

Affected files: crates/pika-news/src/branch_store.rs

Evidence
@@ -1,20 +1,8 @@
-use std::collections::{HashMap, HashSet};
-use std::str::FromStr;
-
-use anyhow::{bail, Context};
-use chrono::{DateTime, SecondsFormat, Utc};
+use anyhow::Context;
 use rusqlite::{params, Connection, OptionalExtension, TransactionBehavior};
 
-use crate::ci_manifest::ForgeLane;
-use crate::ci_state::{
-    classify_ci_failure, configured_target_key_for_lane, next_target_cooloff_until,
-    CiLaneExecutionReason, CiLaneFailureKind, CiLaneStatus, CiTargetHealthSnapshot,
-    CiTargetHealthState, CI_TARGET_HEALTH_INFRA_FAILURE_THRESHOLD,
-};
 use crate::storage::Store;
-
-pub const CI_LANE_LEASE_LOST: &str = "ci lane lease lost";
@@ -95,169 +83,6 @@
-#[derive(Debug, Clone)]
-pub struct BranchCiRunRecord {
-    pub id: i64,
-    ...
-}
-
-#[derive(Debug, Clone)]
-pub struct BranchCiLaneRecord {
-    ...
-}
-
-#[derive(Debug, Clone)]
-pub struct PendingBranchCiLaneJob {
-    ...
-}
-
-#[derive(Debug, Clone)]
-pub struct NightlyFeedItem {
-    ...
-}
-
-#[derive(Debug, Clone)]
-pub struct NightlyRunRecord {
-    ...
-}
-
-#[derive(Debug, Clone)]
-pub struct NightlyLaneRecord {
-    ...
-}
-
-#[derive(Debug, Clone)]
-pub struct PendingNightlyLaneJob {
-    ...
-}

The first major change is the removal of ~160 lines of struct definitions and ~12 import lines from branch_store.rs. Every CI-related type—run records, lane records, pending job structs, nightly feed items, and the internal rerun/recovery helper structs—is deleted. The only constant removed is CI_LANE_LEASE_LOST.

The surviving imports narrow to just anyhow::Context, rusqlite::{params, Connection, OptionalExtension, TransactionBehavior}, and crate::storage::Store. This signals that branch_store.rs no longer touches CI tables at all.

Remove all CI persistence methods from branch_store.rs impl Store

Intent: Delete the ~1,530 lines of `impl Store` methods that handled CI run queuing, lane claiming, heartbeats, finishing, rerunning, failing, requeueing, recovering, nightly scheduling, and CI target health—moving branch_store back to pure branch-lifecycle responsibility.

Affected files: crates/pika-news/src/branch_store.rs

Evidence
@@ -605,1820 +430,289 @@ impl Store {
-    pub fn list_branch_ci_runs(
-    pub fn queue_branch_ci_run_for_head(
-    pub fn queue_branch_ci_failure_for_head(
-    pub fn list_recent_nightly_runs(
-    pub fn get_nightly_run(
-    pub fn queue_nightly_run(
-    pub fn rerun_branch_ci_lane(
-    pub fn rerun_nightly_lane(
-    pub fn fail_branch_ci_lane(
-    pub fn requeue_branch_ci_lane(
-    pub fn recover_branch_ci_run(
-    pub fn fail_nightly_lane(
-    pub fn requeue_nightly_lane(
-    pub fn recover_nightly_run(
-    pub fn claim_pending_branch_ci_lane_jobs(
-    pub fn finish_branch_ci_lane(
-    pub fn heartbeat_branch_ci_lane(
-    pub fn expire_stale_branch_ci_lanes(
-    pub fn claim_pending_nightly_lane_jobs(
-    pub fn finish_nightly_lane(
-    pub fn heartbeat_nightly_lane(
-    pub fn expire_stale_nightly_lanes(
-    pub fn get_or_create_ci_target_health(
-    pub fn update_ci_target_health(

Every method that interacted with the branch_ci_runs, branch_ci_run_lanes, nightly_runs, nightly_run_lanes, or ci_target_health tables is removed from the impl Store block in branch_store.rs. This includes:

  • Branch CI runs: list_branch_ci_runs, queue_branch_ci_run_for_head, queue_branch_ci_failure_for_head
  • Nightly runs: list_recent_nightly_runs, get_nightly_run, queue_nightly_run
  • Rerun/fail/requeue/recover for both branch and nightly lanes
  • Worker lifecycle: claim_pending_branch_ci_lane_jobs, finish_branch_ci_lane, heartbeat_branch_ci_lane, expire_stale_branch_ci_lanes (and nightly equivalents)
  • CI target health: get_or_create_ci_target_health, update_ci_target_health
  • Free functions: list_branch_ci_run_lanes, list_nightly_run_lanes, update_branch_ci_suite_status, update_nightly_run_status, manual_lane_failure_note, append_log_note

What remains in branch_store.rs are the branch upsert, lookup, merge, close, generation job claim, and artifact version management methods—a clean branch-lifecycle API.

Remove free helper functions from branch_store.rs

Intent: Delete the module-level helper functions (`list_branch_ci_run_lanes`, `list_nightly_run_lanes`, `update_branch_ci_suite_status`, `update_nightly_run_status`, `manual_lane_failure_note`, `append_log_note`) that supported the removed CI methods.

Affected files: crates/pika-news/src/branch_store.rs

Evidence
@@ -605,1820 +430,289 @@
-fn list_branch_ci_run_lanes(
-fn list_nightly_run_lanes(
-fn update_branch_ci_suite_status(
-fn update_nightly_run_status(
-fn manual_lane_failure_note(
-fn append_log_note(

Six free (non-method) functions that were only used by the removed CI methods are also deleted from branch_store.rs. These include:

  • list_branch_ci_run_lanes / list_nightly_run_lanes — fetched lane rows for a given run ID
  • update_branch_ci_suite_status / update_nightly_run_status — recomputed aggregate run status after lane state changes
  • manual_lane_failure_note / append_log_note — constructed log text for manual fail actions

All six are re-created identically in ci_store.rs.

Create the new ci_store.rs module

Intent: Introduce `ci_store.rs` as the single home for all CI persistence logic, containing the relocated structs, constants, `impl Store` methods, and free helper functions.

Affected files: crates/pika-news/src/ci_store.rs

Evidence
@@ -0,0 +1,1693 @@
+use std::collections::{HashMap, HashSet};
+use std::str::FromStr;
+
+use anyhow::{bail, Context};
+use chrono::{DateTime, SecondsFormat, Utc};
+use rusqlite::{params, Connection, OptionalExtension, TransactionBehavior};
+
+use crate::ci_manifest::ForgeLane;
+use crate::ci_state::{
+    classify_ci_failure, configured_target_key_for_lane, next_target_cooloff_until,
+    CiLaneExecutionReason, CiLaneFailureKind, CiLaneStatus, CiTargetHealthSnapshot,
+    CiTargetHealthState, CI_TARGET_HEALTH_INFRA_FAILURE_THRESHOLD,
+};
+use crate::storage::Store;
+
+pub const CI_LANE_LEASE_LOST: &str = "ci lane lease lost";

The new file ci_store.rs (1,693 lines) is an almost line-for-line relocation of the CI code removed from branch_store.rs. It opens with the same imports that were removed from the old file (HashMap, HashSet, FromStr, bail, chrono, ForgeLane, ci_state::*) and re-exports CI_LANE_LEASE_LOST.

The file contains:

  1. All CI struct definitionsBranchCiRunRecord, BranchCiLaneRecord, PendingBranchCiLaneJob, NightlyFeedItem, NightlyRunRecord, NightlyLaneRecord, PendingNightlyLaneJob, and the internal BranchLaneRerunSource, BranchLaneRecoverySource, NightlyLaneRerunSource, NightlyLaneRecoverySource.
  2. The full impl Store block with every CI method: queuing, listing, claiming, heartbeating, finishing, rerunning, failing, requeueing, recovering for both branch CI and nightly CI, plus CI target health management.
  3. The six free helper functions (list_branch_ci_run_lanes, list_nightly_run_lanes, update_branch_ci_suite_status, update_nightly_run_status, manual_lane_failure_note, append_log_note).

No logic changes—every SQL query, parameter binding, and error context string is preserved verbatim.

Register ci_store module in main.rs

Intent: Add `mod ci_store;` to the crate root so the new module is compiled and its public items are accessible.

Affected files: crates/pika-news/src/main.rs

Evidence
@@ -7,6 +7,7 @@ mod branch_store;
 mod ci;
+mod ci_store;
 mod ci_manifest;

A single line is added to main.rs to register the new module:

mod ci_store;

It is inserted alphabetically between mod ci; and mod ci_manifest;, following the existing convention.

Update imports in ci.rs

Intent: Switch the CI worker module from importing CI types out of `branch_store` to importing them from the new `ci_store` module.

Affected files: crates/pika-news/src/ci.rs

Evidence
@@ -5,8 +5,8 @@ use tracing::{error, info, warn};
 
-use crate::branch_store::{
-    BranchCiLaneRecord, BranchCiRunRecord, NightlyRunRecord, PendingBranchCiLaneJob,
-    PendingNightlyLaneJob, CI_LANE_LEASE_LOST,
+use crate::ci_store::{
+    BranchCiLaneRecord, BranchCiRunRecord, NightlyRunRecord, PendingBranchCiLaneJob,
+    PendingNightlyLaneJob, CI_LANE_LEASE_LOST,
 };

In ci.rs, the use crate::branch_store::{...} import is changed to use crate::ci_store::{...}. The imported symbols are identical:

  • BranchCiLaneRecord
  • BranchCiRunRecord
  • NightlyRunRecord
  • PendingBranchCiLaneJob
  • PendingNightlyLaneJob
  • CI_LANE_LEASE_LOST

No other changes in ci.rs—all method calls go through Store which Rust resolves from the impl Store block regardless of which file defines it.

Update imports in forge_service.rs

Intent: Redirect the forge service from importing nightly types out of `branch_store` to `ci_store`.

Affected files: crates/pika-news/src/forge_service.rs

Evidence
@@ -3,7 +3,7 @@ use tracing::{info, warn};
 
 use crate::branch_store::BranchUpsertInput;
-use crate::branch_store::NightlyFeedItem;
+use crate::ci_store::NightlyFeedItem;

forge_service.rs previously pulled both BranchUpsertInput and NightlyFeedItem from branch_store. After the split:

  • BranchUpsertInput stays imported from crate::branch_store (it's a branch-lifecycle type)
  • NightlyFeedItem moves to crate::ci_store

This is a minimal two-line change that reflects the new module boundary.

Update imports in web.rs

Intent: Point the web handler module's CI type imports at `ci_store` instead of `branch_store`.

Affected files: crates/pika-news/src/web.rs

Evidence
@@ -12,8 +12,8 @@ use crate::branch_store::{
-    BranchCiRunRecord, BranchLookupRecord, NightlyRunRecord,
+    BranchLookupRecord,
 };
+use crate::ci_store::{BranchCiRunRecord, NightlyRunRecord};

In web.rs, the use crate::branch_store block is narrowed to only BranchLookupRecord, while BranchCiRunRecord and NightlyRunRecord are now imported from crate::ci_store.

This is the final import update. After this change, no file outside ci_store.rs references CI types from branch_store, completing the extraction.

Diff