Back to feed

sledtools/pika branch #65

pika-git-mirror-1

Decouple mirror sync from poll loop

Target branch: master

Merge Commit: 9cd6f51b22da9bf40e250ea690d906ae052a10c8

branch: merged tutorial: ready ci: success
Open CI Details

Continuous Integration

CI: success

Compact status on the review page, with full logs on the CI page.

Open CI Details

Latest run #84 success

2 passed

head a795bcd605a2d1940472568b111dc2963901b792 · queued 2026-03-25 20:54:37 · 2 lane(s)

queued 6s · ran 23s

check-notifications · success check-agent-contracts · success

Summary

This branch decouples the git mirror synchronization pass from the main poll loop in pika-news. Previously, mirror sync ran inline as part of the blocking spawn_blocking tuple alongside issue collection, polling, and worker passes, meaning a slow mirror operation would block the entire iteration. The refactor moves mirror sync into its own independently-spawned background task (maybe_start_background_mirror_pass), introduces a new request_mirror_from_webhook method that sets the force flag with a Webhook wake reason (distinct from the existing request_mirror which uses MirrorRequested), and adds webhook payload parsing (summarize_webhook_ref_updates) so that only branch ref updates (refs/heads/*) trigger a forced mirror pass. The change is covered by unit tests for the wake-reason semantics, the ref-update summarizer, and integration-level tests for the webhook handler's mirror-request behavior.

Tutorial Steps

Introduce `request_mirror_from_webhook` to separate webhook-triggered mirrors from explicit requests

Intent: Create a dedicated method for webhook-initiated mirror requests that sets the `mirror_requested` flag but uses `WakeReason::Webhook` instead of `WakeReason::MirrorRequested`. This preserves the distinction between a webhook waking the system and an explicit mirror request, which matters for scheduling logic downstream.

Affected files: crates/pika-news/src/forge_runtime.rs

Evidence
@@ -163,17 +163,27 @@ impl ForgeRuntime {
+    pub(crate) fn request_mirror_from_webhook(&self) {
+        self.mirror_requested.store(true, Ordering::Release);
+        self.notify_with_reason(WakeReason::Webhook);
+    }

A new request_mirror_from_webhook method is added to ForgeRuntime. It stores true into the mirror_requested atomic (just like request_mirror) but notifies with WakeReason::Webhook rather than WakeReason::MirrorRequested.

This distinction is important: the main loop can differentiate between a mirror triggered as a side-effect of a webhook (which should not delay the poll/worker pass) and an explicit mirror request (which may carry different scheduling semantics). A #[cfg(test)] accessor mirror_requested_for_test is also added to support assertions in test code without exposing the atomic publicly.

Move mirror sync out of the blocking poll tuple into its own spawned task

Intent: Decouple the mirror pass from the main `spawn_blocking` call so that a slow git fetch/push does not block issue collection, polling, or worker passes. The mirror now runs in a separate `tokio::spawn` + `spawn_blocking` chain.

Affected files: crates/pika-news/src/forge_runtime.rs

Evidence
@@ -190,19 +200,14 @@ impl ForgeRuntime {
-                        iteration_runtime.run_scheduled_mirror_pass(
-                            &iteration_context.store,
-                            &iteration_context.config,
-                        ),
                     )
                 })
                 .await
                 {
-                    Ok((issues, poll_result, worker_result, mirror_result)) => {
+                    Ok((issues, poll_result, worker_result)) => {
@@ -325,6 +330,47 @@ impl ForgeRuntime {
+    fn maybe_start_background_mirror_pass(self: &Arc<Self>, context: ForgeRuntimeContext) {
+        let Some(forge_repo) = context.config.effective_forge_repo() else {
+            return;
+        };
+        if forge_repo.mirror_remote.is_none() {
+            return;
+        }
+        let background_enabled = forge_repo.mirror_poll_interval_secs.unwrap_or(0) > 0;
+        let force_requested = self.mirror_requested.load(Ordering::Acquire);
+        if !background_enabled && !force_requested {
+            return;
+        }
+        if self.mirror_running.load(Ordering::Acquire) {
+            return;
+        }

The run_scheduled_mirror_pass call is removed from the 4-element tuple inside spawn_blocking. The main loop now produces a 3-element tuple (issues, poll_result, worker_result) and handles only those results.

Instead, maybe_start_background_mirror_pass is called at the top of each loop iteration, before the blocking poll. This method performs several guard checks:

  1. A forge repo with a mirror_remote must be configured.
  2. Either background polling is enabled (mirror_poll_interval_secs > 0) or a force mirror was explicitly requested.
  3. A mirror is not already running (mirror_running atomic guard).

If all checks pass, it spawns a new tokio::spawn task that runs run_scheduled_mirror_pass inside spawn_blocking. On completion, if a mirror was attempted and another request arrived in the meantime, it re-notifies with WakeReason::MirrorRequested to avoid missing a queued request.

Parse webhook payloads to selectively trigger mirror sync only for branch ref updates

Intent: Avoid triggering an expensive mirror pass for tag pushes or other non-branch ref updates. Only `refs/heads/*` updates should force a mirror sync.

Affected files: crates/pika-news/src/web.rs

Evidence
@@ -3549,16 +3549,40 @@ async fn webhook_handler(
-    state.forge_runtime.wake_webhook();
-    let update_count = String::from_utf8_lossy(&body)
-        .lines()
-        .filter(|line| !line.trim().is_empty())
-        .count();
-    eprintln!("webhook: received {} ref updates", update_count);
+    let (update_count, branch_ref_update_count) = summarize_webhook_ref_updates(&body);
+    if branch_ref_update_count > 0 {
+        state.forge_runtime.request_mirror_from_webhook();
+    } else {
+        state.forge_runtime.wake_webhook();
+    }
@@ +3569,0 +3569,18 @@
+fn summarize_webhook_ref_updates(payload: &[u8]) -> (usize, usize) {
+    let mut update_count = 0usize;
+    let mut branch_ref_update_count = 0usize;
+    for line in String::from_utf8_lossy(payload).lines() {
+        let line = line.trim();
+        if line.is_empty() {
+            continue;
+        }
+        update_count += 1;
+        if line
+            .split_whitespace()
+            .nth(2)
+            .is_some_and(|ref_name| ref_name.starts_with("refs/heads/"))
+        {
+            branch_ref_update_count += 1;
+        }
+    }

The webhook_handler function is updated to call a new helper summarize_webhook_ref_updates which parses the line-oriented webhook payload. Each non-empty line is expected to have the format <old-sha> <new-sha> <ref-name>. The helper counts total updates and branch-specific updates (where the third whitespace-delimited field starts with refs/heads/).

If any branch ref updates are present, the handler calls request_mirror_from_webhook (setting the force flag). Otherwise it falls back to wake_webhook (no mirror). The log message is also enriched to report both counts.

Add unit and integration tests for the new mirror-request semantics

Intent: Verify the behavioral contract: `wake_webhook` must not set the mirror flag, `request_mirror_from_webhook` must set it with `Webhook` reason, `request_mirror` must use `MirrorRequested` reason, and the webhook handler must conditionally request mirrors based on ref type.

Affected files: crates/pika-news/src/forge_runtime.rs, crates/pika-news/src/web.rs

Evidence
@@ -780,3 +826,43 @@
+    #[test]
+    fn wake_webhook_does_not_force_a_mirror_pass() {
+    #[test]
+    fn webhook_mirror_request_sets_force_flag() {
+    #[test]
+    fn explicit_mirror_request_uses_mirror_requested_wake_reason() {
@@ -4832,6 +4858,89 @@
+    #[test]
+    fn summarize_webhook_ref_updates_counts_branch_refs() {
+    #[test]
+    fn summarize_webhook_ref_updates_ignores_blank_and_malformed_lines() {
+    #[tokio::test]
+    async fn webhook_branch_ref_updates_request_forced_mirror() {
+    #[tokio::test]
+    async fn webhook_non_branch_updates_do_not_request_forced_mirror() {

Seven new tests are added across the two files:

forge_runtime::tests (3 unit tests):

  • wake_webhook_does_not_force_a_mirror_pass — confirms wake_webhook() does not set mirror_requested.
  • webhook_mirror_request_sets_force_flag — confirms request_mirror_from_webhook() sets the flag and uses WakeReason::Webhook.
  • explicit_mirror_request_uses_mirror_requested_wake_reason — confirms request_mirror() sets the flag and uses WakeReason::MirrorRequested.

web::tests (4 tests):

  • summarize_webhook_ref_updates_counts_branch_refs — mixed refs/heads and refs/tags payload yields correct counts (3, 2).
  • summarize_webhook_ref_updates_ignores_blank_and_malformed_lines — lines without a third field or that are blank are handled gracefully.
  • webhook_branch_ref_updates_request_forced_mirror — full integration test: sends a signed webhook with a refs/heads/master update and asserts mirror_requested is true.
  • webhook_non_branch_updates_do_not_request_forced_mirror — sends a signed webhook with only a refs/tags/v1 update and asserts mirror_requested is false.

Diff