← Corpus / augment-it / reminder
Pickup notes — 2026-06-03
Late-night session that ended in an actually-working per-record augmentation surface. Records Surface at :3011 wired into the Flow at step 3, three connectors (Firecrawl scan, Firecrawl + Haiku agent, SerpApi), inline edit on name + URL, multi-URL accept (array column), promote-to-next-version save bar at top + bottom with post-promote navigation to Enhanced Records / Augment / stay. User ran the loop end-to-end on the 96-row pipeline tracker — promoted v5 → v6 successfully. Also: backup of v5 work pulled to .backups/ (98 URLs in 45 rows safe on host fs), Record Collector decomposed into RecordSetsList + RecordSetCard components with descending-by-created_at sort + per-card CSV download, and a deep audit of v3 → v4 → v5 that exonerated promote (it works) and named the actual data-persistence failure mode (URL edits routed to helpful_links instead of url). Three new context-v docs land the lessons. Branch is feat/bundle-media-packs, last commit pushed is 7e449db (Records Surface initial), HEAD has uncommitted improvements + new specs.
- Path
- reminders/Pickup-2026-06-03.md
- Authors
- Michael Staton
- Augmented with
- Claude Code on Claude Opus 4.7 (1M context)
- Tags
- Augment-It · Pickup · Next-Steps · Records-Surface · Flow-for-Bundles-Packs · Promote · Data-Persistence · Wed Jun 03 2026 00:00:00 GMT+0000 (Coordinated Universal Time)
Pickup notes — 2026-06-03
TL;DR
- Branch:
feat/bundle-media-packs. Last pushed commit:7e449db. Working tree dirty — promote bar + inline edit + record-collector decomposition + three new context-v docs are unstaged. Commit before doing anything else. - Backup safe:
.backups/2026-06-02_records-surface-acceptances/holds the v5 work (98 URLs across 45 rows) extracted from the row-store docker volume. Do not delete. - What works end-to-end now: open
:3100→ step 3 → Records Surface → click a connector chip on a row → candidates render inline →pickoredit→trim→pickwrites to the row’sofficial_updates_index_urlsarray column → promote bar at top OR bottom → confirms → creates v6 → auto-switches → offers nav to Enhanced Records (download), Augment (next pass), or stay. - The big unfinished piece is still §“Response Reviewer as a shell” from the Flow-for-Bundles-Packs spec. The user has flagged this 3+ times this session. Step 4 still loads the legacy socials triage UI regardless of which fire produced the data.
Read these context-v docs in this order
- [[../specs/Flow-for-Bundles-Packs]] (v0.0.0.3) — the active spec. Two-surface Flow (Record Collector → Records Surface), non-destructive refactor rule locked, component-decomposition rule locked, Response Reviewer-as-shell pairing model documented. Read §“What gets built in what order” first — it’s the build checklist.
- [[../issues/Augment-Transformations-Not-Reliably-Persisting]]
(v0.0.0.2) — the data audit. Promote works. v3 → v4 → v5
carried data forward correctly. The actual bug: 23 of 96 rows
have the user’s hand-curated URL trapped in
helpful_linksbecause the by-record card’s URL-edit affordance writes there instead of to theurlcolumn. Includes the full v3 timestamp reconstruction (user worked 00:47 – 02:00 on 2026-05-23, allsource: 'manual'). - [[Record-Count-Stays-Stable-Across-Versions]] — invariant: 1 dataset = 1 row count. Forever. The 231-rows-in-store thing is real (39 RSVP + 96 v4 + 96 v5) and architectural; don’t ever surface that number to the user.
- [[../issues/Troubleshooting-UI-for-Official-Blogs]] — earlier in the day before the spec rewrite. Still useful as the catalog of gaps that led to the rethink.
- [[../specs/Per-Record-Iteration-as-Primary-Surface-for-Pack-Fires]] (v0.0.0.2) — the predecessor spec to Flow-for-Bundles-Packs. Not the active doc but has the request-inspection (pre-fire preview + post-fire receipt) section that’s still a credible next feature.
Implementation state vs spec — Flow-for-Bundles-Packs build order
| # | Step | Status | Notes |
|---|---|---|---|
| 1 | Create apps/records-surface/ as a new federation remote | ✅ | port 3011, registered in shell + REMOTES |
| 2 | Wire it to workspace, read-only list of rows | ✅ | RecordsList renders name + URL per row |
| 3 | serpapi-site-search + connector.fire capability | ✅ | needs SERPAPI_API_KEY; fires site: Google search |
| 4 | firecrawl-nav-scan | ✅ | scrape homepage + scored link harvest |
| 5 | firecrawl-nav-agent | ✅ | Haiku 4.5 classifies nav links; falls back to nav-scan when ANTHROPIC_API_KEY unset |
| 6 | [pick] → row.update | ✅ | extended beyond spec: array column, multi-URL append with dedup, per-URL remove |
| 7 | Per-row [show all fields] toggle | ❌ | not done |
| 8 | [paste URL] affordance for rows without URL | ✅ | every row has paste/edit input in candidates panel + inline-editable URL on row head |
| 9 | Response Reviewer App.svelte → shell with SocialsTriageView / CandidatesView | ❌ | NOT DONE — user has flagged this 3+ times |
Built beyond the spec (additive)
- PromoteBar component (top + bottom of Records Surface) with
confirmation + post-promote navigation (Enhanced Records / Augment /
stay). Calls existing
record_set.promotecapability. - Inline-editable name + URL per row via new EditableField component.
- helpful_links → url visibility fallback in
pick-url.ts. Rows withurl='unknown'and a populated helpful_links surface the helpful_link URL on the Records Surface with a “recovered from helpful_links” chip. Edit → save writes to the canonicalurlcolumn, so the recovery + correction happens in one motion. - Record Collector decomposition (Phase 1 of un-bundling
App.svelte — applied retroactively to a legacy remote): new
components/RecordSetsList.svelte,components/RecordSetCard.svelte,logic/download.ts. List is now sorted descending bycreated_at. Every card has a↓ CSVdownload button — builds CSV client-side with full row data including array-shaped fields (JSON-stringified). - Records Surface dispatch already lives at
services/social-search/src/records-surface/connectors.ts(separate from entity-pulse dispatch shim — the new surface uses its own connector implementations, not the Entity Pulse pack handlers).
What’s still broken / wrong
- Response Reviewer step 4 doesn’t swap UIs. Spec calls for the
shell to load
SocialsTriageViewfor socials-bundle fires andCandidatesViewfor URL-finder fires. Right now it loads the legacy socials UI for everything, which is “absolute noise” when the user is reviewing a Blog/URL-finder fire. This is the highest-leverage unfinished piece per the user’s repeated feedback. - The mis-routed URL edit affordance (the bug from
Augment-Transformations-Not-Reliably-Persisting). The by-record
card in Response Reviewer writes URL edits to
row.helpful_links.addinstead ofrow.update({ url }). Future edits will recreate the problem until fixed. Code location:apps/response-reviewer/src/App.svelte— search forhelpful_links.addto find the binding. - Data fixup script to lift
helpful_links[0].url→urlfor the 18 D-pattern rows (Howard Schultz Foundation, Lumina, McGovern, Tepper, etc. — full list in [[../issues/Augment-Transformations-Not-Reliably-Persisting]] §“Pattern breakdown”). Read-side fallback is in place so they’re usable on the Records Surface NOW, but the canonicalurlcolumn is still wrong until a fixup runs. - 5 C-pattern rows need per-row decisions (Griffin Catalyst,
Pinterest, General Motors, Britebound, Ballmer II).
urlhas a real value but the helpful_link may be the correct one — needs user judgment per row. - Row multiplication across versions — the 231-rows thing per [[Record-Count-Stays-Stable-Across-Versions]]. Either change row-store to row-as-immortal-identity OR scope every UI to active-record-set counts only.
- Enhanced Records List (step 5) doesn’t have download — the
user wanted “shoot me to the end remote to download.” Right now
the download lives on each Record Collector card (step 1). The
post-promote nav offers “Open Enhanced Records (download)” but
the Enhanced Records UI itself doesn’t have a download button.
Lift
apps/record-collector/src/logic/download.ts+components/RecordSetCard.svelte’s download button into Enhanced Records.
Uncommitted changes — what’s dirty
M apps/record-collector/src/App.svelte (replaced inline record-set markup w/ <RecordSetsList />)
M apps/records-surface/src/components/RecordRow.svelte (inline EditableField name + URL, helpful_links hint)
M apps/records-surface/src/components/RecordsList.svelte (PromoteBar at top + bottom)
M apps/records-surface/src/logic/pick-url.ts (resolveRowUrl/Name with helpful_links fallback)
M apps/records-surface/src/state/records.svelte.ts (promoteActiveRecordSet action, updateRowField)
?? apps/record-collector/src/components/ (RecordSetsList + RecordSetCard)
?? apps/record-collector/src/logic/ (download.ts)
?? apps/records-surface/src/components/EditableField.svelte
?? apps/records-surface/src/components/PromoteBar.svelte
?? apps/records-surface/src/logic/promote.ts
?? context-v/issues/Augment-Transformations-Not-Reliably-Persisting.md
?? context-v/reminders/Record-Count-Stays-Stable-Across-Versions.md
?? .backups/2026-06-02_records-surface-acceptances/ (DO NOT git-add; should be in .gitignore)
Suggested next-session first commit:
feat(records-surface, record-collector): promote-to-next-version save bar, inline edit, multi-URL accept, record-collector decomposition + descending sort + CSV download
(then a body describing the chunks per git-conventions)
What to start with next session
Three credible directions, ordered by user-leverage:
A. Response Reviewer shell + CandidatesView
The user has flagged this three+ times. They open step 4 expecting the
right tool for what they just fired and get socials chips on everything.
Mechanical scope: refactor apps/response-reviewer/src/App.svelte so
the inner per-record UI is loaded from a view component selected by
last_fire_type. Move existing socials triage logic into
components/SocialsTriageView.svelte. Add stub components/CandidatesView.svelte
that renders inline candidates (lift from Records Surface). Add
last_fire_type workspace state. Per Flow-for-Bundles-Packs spec
§“Response Reviewer as a shell”.
B. Data fixup script for D-pattern rows
18 rows where the user’s correct URL is in helpful_links and url
is 'unknown'. Read-only audit shows exact mapping. Write a script
under services/social-search/scripts/lift-helpful-links-to-url.ts
(matches the existing scripts/ pattern), dry-run mode prints what
it’d change, —apply does the writes via row.update. Then the user
can re-augment with confidence. (Also fix the by-record card’s
URL-edit affordance so future edits don’t recreate the problem.)
C. Step 5 (Enhanced Records) gets a download button
The user’s post-promote flow wants the “end remote to download.”
Pull logic/download.ts + the ↓ CSV button pattern from
Record Collector into Enhanced Records List. Probably a 30-minute
move.
Lean: A. The user has been waiting on it longest and it’s the biggest UX cliff. B’s blast radius is small and could happen alongside. C is small and the post-promote nav already routes through it conceptually.
Branch + git state
- Current branch:
feat/bundle-media-packs - Last pushed commit:
7e449db - Working tree: dirty (see §“Uncommitted changes” above)
- Origin: clean through
7e449db; tonight’s work not yet pushed - Latest tag:
v3.0.0.3(closes the Shell UX Coherence arc from 2026-06-02 morning)
Critical files / where things live
- The active spec:
context-v/specs/Flow-for-Bundles-Packs.md - The audit:
context-v/issues/Augment-Transformations-Not-Reliably-Persisting.md - The invariant:
context-v/reminders/Record-Count-Stays-Stable-Across-Versions.md - The backup:
.backups/2026-06-02_records-surface-acceptances/rows.json(98 URLs in 45 rows; v5 state at the end of tonight’s curation) - Records Surface remote:
apps/records-surface/(port 3011) - Records Surface connectors:
services/social-search/src/records-surface/connectors.ts connector.firecapability:services/workspace/src/capabilities.ts(workspace) + handler atservices/social-search/src/server.ts- Flow step assignments:
shell/src/remotes.ts(ROTATIONarray; step 3 isrecordsSurfacesince7e449db) - The mis-routed URL edit affordance: somewhere in
apps/response-reviewer/src/App.svelte— grep forhelpful_links.add
Verify state on next session
git fetch --tags
git status # should be dirty per §"Uncommitted changes"
git log --oneline -8 # last pushed should be 7e449db
docker compose ps # all services up; nats max_payload 8MB
ls -lh .backups/2026-06-02_records-surface-acceptances/ # the backup is still there
ls apps/records-surface/src/components/ # ConnectorButton, ConnectorPalette (legacy from earlier in branch), CandidatesPanel, RecordRow, RecordsList, EditableField, PromoteBar
ls context-v/specs/Flow-for-Bundles-Packs.md # exists, v0.0.0.3
Related
- [[../specs/Flow-for-Bundles-Packs]] — active spec (v0.0.0.3)
- [[../issues/Augment-Transformations-Not-Reliably-Persisting]] — data audit (v0.0.0.2)
- [[Record-Count-Stays-Stable-Across-Versions]] — the row-count invariant
- [[../issues/Troubleshooting-UI-for-Official-Blogs]] — earlier session catalog
- [[../specs/Per-Record-Iteration-as-Primary-Surface-for-Pack-Fires]] — predecessor (kept for the request-inspection section)
- [[../specs/Connector-Inventory-and-Per-Record-Palette]] — older registry pattern (still informs the connector layer)
- [[../specs/Entity-Pulse-Bundle]] — yesterday’s work; the dispatch shim is still there but the Records Surface bypasses it for direct connector firing
- [[Pickup-2026-06-02]] — yesterday’s pickup (the start of today’s arc)