Back to feed

sledtools/pika branch #96

pika-git-legacy-pr-1

Extract legacy PR store

Target branch: master

Merge Commit: b9944cb27e8f7f408cd3d2e3b24e67075ae2bd64

branch: merged tutorial: ready ci: success
Open CI Details

Continuous Integration

CI: success

Compact status on the review page, with full logs on the CI page.

Open CI Details

Latest run #122 success

2 passed

head 9043122a8265c68df9a298c0d46284c3276b0819 · queued 2026-03-26 01:37:57 · 2 lane(s)

queued 4m 02s · ran 28s

check-notifications · success check-agent-contracts · success

Summary

This branch extracts all legacy pull-request storage logic from the monolithic storage.rs module into a dedicated legacy_pr_store.rs file within the pika-news crate. Six data structs (PrUpsertInput, UpsertOutcome, GenerationJob, FeedItem, PrDetailRecord, PrSummaryRecord), ten impl Store methods (upsert, poll markers, claim jobs, mark ready/failed, close stale PRs, list feeds/summaries, get detail, recover stale, queue regeneration), and five private helper functions (ensure_repo, upsert_pr_row, upsert_artifact_row, insert_pending_artifact_version, supersede_non_current_generations, has_inflight_generation_for_head) are moved verbatim. The original storage.rs is slimmed down and re-exports the moved types via pub use to preserve the public API. A minor behavioral improvement upgrades several transaction starts from unchecked_transaction to transaction_with_behavior(Immediate) and adds a PR-state filter (state IN ('open', 'merged')) to the pending-jobs query. The update_poll_markers implementation is also rewritten to use direct column updates on the repos table instead of a separate poll_markers table.

Tutorial Steps

Register the new module in main.rs

Intent: Declare the `legacy_pr_store` module so the Rust compiler includes the new file in the crate.

Affected files: crates/pika-news/src/main.rs

Evidence
@@ -12,6 +12,7 @@ mod forge_runtime;
 mod forge_service;
 mod github;
 mod inbox_store;
+mod legacy_pr_store;
 mod live;
 mod local;
 mod mirror;

A single line is added to main.rs to register the new module:

mod legacy_pr_store;

This is placed alphabetically between inbox_store and live, following the existing convention. No other changes to main.rs are required because all public symbols are re-exported through storage.rs.

Create legacy_pr_store.rs with data types

Intent: Move the six PR-related data structs out of `storage.rs` into their own file, giving the legacy PR persistence layer a clear module boundary.

Affected files: crates/pika-news/src/legacy_pr_store.rs

Evidence
@@ -0,0 +1,811 @@
+#![allow(dead_code)]
+
+use anyhow::Context;
+use chrono::Utc;
+use rusqlite::{params, Connection, OptionalExtension, TransactionBehavior};
+
+use crate::storage::Store;
+
+#[derive(Debug, Clone)]
+pub struct PrUpsertInput {
+    pub repo: String,
+    pub pr_number: i64,
+    ...
@@ +23,12 +23,12 @@
+#[derive(Debug, Clone, Default)]
+pub struct UpsertOutcome {
+    pub inserted: bool,
+    pub head_changed: bool,
+    pub queued: bool,
+}
@@ +30,10 +30,10 @@
+pub struct GenerationJob {
+    pub artifact_id: i64,
+    pub pr_id: i64,
+    pub repo: String,
@@ +45,10 +45,10 @@
+pub struct FeedItem {
+    pub pr_id: i64,
+    pub repo: String,
@@ +57,17 +57,17 @@
+pub struct PrDetailRecord {
+    pub repo: String,
+    ...
@@ +76,9 +76,9 @@
+pub struct PrSummaryRecord {
+    pub repo: String,

The new file begins with #![allow(dead_code)] at the module level since several structs and methods are not yet called from production code paths. It imports crate::storage::Store so it can add impl Store blocks—Rust allows trait-free impl blocks on a type from another module within the same crate.

All six data structs are moved without modification:

StructPurpose
PrUpsertInputInput payload for upserting a PR row
UpsertOutcomeReturn value describing what changed during upsert
GenerationJobRow returned when claiming pending generation work
FeedItemLightweight PR record for the activity feed
PrDetailRecordFull PR detail including tutorial JSON and diff
PrSummaryRecordSummary record with optional tutorial payload

Move impl Store methods for PR lifecycle management

Intent: Relocate all ten PR-related methods from the `Store` impl block in `storage.rs` into the new module, keeping the same method signatures and public visibility.

Affected files: crates/pika-news/src/legacy_pr_store.rs, crates/pika-news/src/storage.rs

Evidence
@@ +86,6 +86,6 @@
+impl Store {
+    pub fn upsert_pull_request(&self, input: &PrUpsertInput) -> anyhow::Result<UpsertOutcome> {
@@ +104,6 +104,6 @@
+    pub fn update_poll_markers(
+        &self,
+        repo: &str,
+        last_open_polled_at: Option<&str>,
+        last_merged_polled_at: Option<&str>,
@@ +133,6 +133,6 @@
+    pub fn claim_pending_generation_jobs(
+        &self,
+        limit: usize,
+    ) -> anyhow::Result<Vec<GenerationJob>> {
@@ +205,6 +205,6 @@
+    pub fn mark_generation_ready(
@@ +281,6 +281,6 @@
+    pub fn mark_generation_failed(
@@ -114,529 +41,6 @@ impl Store {

The moved methods and their responsibilities:

  1. upsert_pull_request — Inserts or updates a PR row and its associated artifact version within an Immediate transaction.
  2. update_poll_markers — Updates repo-level polling timestamps. The implementation is rewritten: instead of upserting into a poll_markers table, it now updates last_open_polled_at and last_merged_polled_at columns directly on the repos table.
  3. claim_pending_generation_jobs — Selects pending artifact versions, atomically sets them to generating, and returns GenerationJob records. The new version uses transaction_with_behavior(Immediate) instead of unchecked_transaction and adds a pr.state IN ('open', 'merged') filter to avoid claiming jobs for closed PRs.
  4. mark_generation_ready — Completes a generation by storing tutorial JSON/HTML/diff, handling the superseded-vs-activated logic.
  5. mark_generation_failed — Records failure with retry-backoff logic (up to 5 retries).
  6. close_stale_open_prs — Marks PRs as closed if they no longer appear in the current open set.
  7. list_feed_items — Queries the activity feed with latest artifact status.
  8. list_pr_summaries — Dynamically-filtered query returning PR summaries with optional tutorial JSON.
  9. get_pr_detail — Full detail lookup for a single PR.
  10. recover_stale_generating — Startup recovery that resets generating artifacts back to pending.
  11. queue_regeneration — Creates a new pending artifact version for re-generation.

All 529 lines of method bodies and 162 lines of helper functions are removed from storage.rs.

Move private helper functions

Intent: Extract the six private helper functions that support the PR store methods into the new module.

Affected files: crates/pika-news/src/legacy_pr_store.rs

Evidence
@@ +688,20 +688,20 @@
+fn ensure_repo(conn: &Connection, repo: &str) -> anyhow::Result<i64> {
@@ +710,30 +710,30 @@
+fn upsert_pr_row(
+    conn: &Connection,
+    repo_id: i64,
+    input: &PrUpsertInput,
+) -> anyhow::Result<(i64, Option<String>, bool)> {
@@ +755,30 +755,30 @@
+fn upsert_artifact_row(
+    conn: &Connection,
+    pr_id: i64,
+    head_sha: &str,
+) -> anyhow::Result<UpsertOutcome> {
@@ +792,15 +792,15 @@
+fn insert_pending_artifact_version(
@@ +800,8 +800,8 @@
+fn supersede_non_current_generations(conn: &Connection, pr_id: i64) -> anyhow::Result<usize> {
@@ +808,11 +808,11 @@
+fn has_inflight_generation_for_head(

Six private (crate-local) functions are placed at the bottom of the new file:

  • ensure_repo — Upserts a repos row and returns its id.
  • upsert_pr_row — Handles the insert-or-update logic for pull_requests, returning the row id, previous head SHA, and whether an insert occurred.
  • upsert_artifact_row — Decides whether to create a new pending artifact version or leave the existing one alone based on head SHA matching and current status.
  • insert_pending_artifact_version — Computes the next version number and inserts a pending artifact row.
  • supersede_non_current_generations — Marks all non-current pending/generating/failed artifacts as superseded.
  • has_inflight_generation_for_head — Checks if a pending or generating artifact already exists for a given head SHA.

These remain private to the crate and are only called by the impl Store methods in the same file.

Update storage.rs to re-export moved types

Intent: Preserve the public API surface so that existing callers importing from `crate::storage` continue to compile without changes.

Affected files: crates/pika-news/src/storage.rs

Evidence
@@ -5,95 +5,22 @@
 use anyhow::Context;
-use chrono::Utc;
 use nostr::key::PublicKey;
 use nostr::nips::nip19::ToBech32;
-use rusqlite::{params, Connection, OptionalExtension, TransactionBehavior};
+use rusqlite::{params, Connection, OptionalExtension};
 use serde::{Deserialize, Serialize};
 
 use crate::inbox_store::merge_branch_inbox_state_alias;
+#[allow(unused_imports)]
+pub use crate::legacy_pr_store::{
+    FeedItem, GenerationJob, PrDetailRecord, PrSummaryRecord, PrUpsertInput, UpsertOutcome,
+};

The import section of storage.rs is cleaned up:

  • chrono::Utc is removed (no longer used here).
  • TransactionBehavior is removed from the rusqlite import.
  • A pub use re-export block is added:
#[allow(unused_imports)]
pub use crate::legacy_pr_store::{
    FeedItem, GenerationJob, PrDetailRecord, PrSummaryRecord,
    PrUpsertInput, UpsertOutcome,
};

This ensures any module that previously imported these types via crate::storage::PrUpsertInput (etc.) continues to work. The #[allow(unused_imports)] suppresses warnings for types that may not currently be referenced through this re-export path.

The Store struct definition and its core infrastructure methods (open, with_connection) remain in storage.rs alongside all non-PR-related logic (inbox methods, chat allowlist, etc.).

Behavioral changes in the moved code

Intent: Document the subtle behavioral differences introduced during the extraction, beyond the pure code-move.

Affected files: crates/pika-news/src/legacy_pr_store.rs

Evidence
@@ +133,6 +133,6 @@
+            let tx = conn
+                .transaction_with_behavior(TransactionBehavior::Immediate)
+                .context("start claim generation jobs transaction");
+...AND pr.state IN ('open', 'merged')
@@ +104,6 +104,6 @@
+            tx.execute(
+                "UPDATE repos
+                 SET last_open_polled_at = COALESCE(?1, last_open_polled_at),
+                     last_merged_polled_at = COALESCE(?2, last_merged_polled_at),

While this branch is primarily a code-move refactor, three behavioral changes are introduced:

  1. Transaction isolation for claim_pending_generation_jobs: Changed from unchecked_transaction() to transaction_with_behavior(TransactionBehavior::Immediate). This acquires a write lock immediately rather than deferring it, preventing potential SQLITE_BUSY errors when the UPDATE runs inside the transaction. The claim loop is also restructured to first collect candidate IDs, then claim-and-load each one individually with a lost-update check (AND status = 'pending').

  2. PR state filter on pending jobs: The generation jobs query now includes AND pr.state IN ('open', 'merged'), preventing artifact generation from being claimed for PRs that have been closed. Previously, a closed PR with a pending artifact would still be picked up.

  3. Poll markers storage model: update_poll_markers no longer upserts into a poll_markers table. Instead, it updates last_open_polled_at and last_merged_polled_at columns directly on the repos table. The method signature also changes from (open_cursor, merged_cursor) to (last_open_polled_at, last_merged_polled_at), reflecting the shift from opaque cursor strings to timestamp-based polling.

Diff