Context Vigilance
The why beneath the code
Specs, habits, and reflections that shape how augment-it gets built. Versioned context, intentionally legible.
specs
-
Strategy Curator — An Entry-Point App for augment-it
Pick or create a strategy, gather sources for it (metadata first, full content on demand, PDFs preserved), and pull extracts — without the strategy↔source↔funder↔person graph ever decoupling. The trick: a canonical source registry in SurrealDB owns identity; the client's filesystem owns usage; a shared UUID is the only bond.
-
Record ↔ DB Resolver — operator-driven match/create bridge from row-store records to canonical organizations
The row-store records and the SurrealDB canonical orgs never got bridged. The resolver is the per-record surface that closes the gap: for each record, confirm which canonical org it is (→ additive enrich) or create a new one — one at a time, operator in the driver's seat.
-
Workspaces as Tenant Primitive — toggling, tenant-aware microservices, per-tenant env-var pickup, and the seam that lets MCPs and connectors vary per client
Augment-It (and, by inheritance, its sibling pillar apps memopop-ai and dididecks-ai) needs a tenant primitive named workspace — the boundary that says 'humain-vc' vs 'reach-edu' vs whoever comes next. The terminal state is rich: per-workspace branded theme, team membership, roles + permissions, registration flow, auth, per-tenant choice of LLM provider / CRM / MCP server / search connector / storage destination. We are filesystem-backed and local right now, so this spec lays out the vision once and then scopes baby step 1 down to its bones: a workspace toggle, a workspace-aware envelope on every microservice request, I/O routing to `clients/<slug>/`, and a per-workspace `.env` pickup that resolves through a connector-config seam designed to absorb Decile-shaped (and future MCP-shaped, future LLM-shaped) per-tenant integrations without re-design. Inspired by [[Cloud-Variant-of-Dididecks-AI-Workspace]] but diverges on storage (filesystem-now vs cloud) and on cross-cutting reach (this is the contract sibling pillar apps inherit, not a dididecks-only artifact).
-
Client tagging on canonical writes — every observation carries the client that produced it, and every canonical entity carries the materialized set of clients that can see it
The canonical layer (persons, organizations, events, locations on SurrealDB main/main) is cross-client by design — one row per real-world entity, deliberately shareable across reach-edu, humain-vc, and any future client. That sharing is the point. But the operator still needs to know, at a glance, *which clients have touched this entity*: who first surfaced them, who's edited them since, who currently has them visible in their workspace. The pattern: every observation carries a `client` field; every entity carries a materialized `client_access` array (sorted by first-touch time). Reads are filtered by `WHERE client_access CONTAINS '<current-workspace-slug>'`. Writes always stamp the current workspace into the observation. Three forms of 'touched' (entered, updated, currently available) all collapse into the observation log — no separate access table, no separate audit table.
-
Pulse pattern — one operator burst against one entity, expressed as N independent pulse-dimensions composed in a pulse-surface, each dimension potentially owning its own microservice
Building the first version of the person-enrichment remote surfaced a pattern hiding inside the existing per-record / per-flow vocabulary. Operators don't think in 'steps' when they're enriching one person — they think in *bursts*. One Google search reveals a name, a LinkedIn URL, an X handle, two alt emails, and which org the person works for. That's not five sequential UI steps; it's one operator-attention burst against one entity, and the UI should let the operator commit all of it in one save. The pattern: a `pulse` is the burst itself; a `pulse-dimension` is one independent editable concern within the burst (name, socials, emails, org); a `pulse-surface` is the parent that loads the entity, hosts its dimensions, collects them, and writes them as one observation batch. Dimensions can be nested (OrgPicker contains OrgCreate). Dimensions are reusable across pulse-surfaces (NameFields is the same in person-enrichment as it will be in org-enrichment). Each dimension can grow its own microservice when it earns one — matching the existing pack / surface pattern.
-
Sparse-Person Enrichment Surface — per-email triage where the operator finds the human behind the address, picks or creates the org they belong to, and writes both back to the canonical layer
The 177 Turning-Jobs-Into-Degrees attendees just landed in SurrealDB as 177 emails with almost nothing else — no names, no titles, no LinkedIn URLs, and (for the 63 with personal-email domains) no org signal either. The default augment-it flow assumes records are URL-bearing and scrape-targetable; this set is the opposite shape. What the operator needs is a per-row enrichment surface: pick a sparse person, search for who they are (web search → LinkedIn → SerpApi), match to an existing `persons` row or create a new one, then pick or create the `organizations` row they belong to (with multi-domain handling for orgs like Sterling Foundations that spell themselves three ways), and save the result as observations on both sides. This is a *sibling* to [[Per-Record-Iteration-as-Primary-Surface-for-Pack-Fires]] — same chrome pattern, same shell integration, different verbs and a two-entity save target. Bulk fan-out stays *secondary* (per that spec's hard-won lesson) until manual enrichment has surfaced what works.
-
Records Surface sort step and UI — sort ships as part of the **Sort & Filter Lens**, the second lens in step 2's default set (alongside the existing **Pack Firing Lens**); `lens` is the architectural primitive this spec introduces and lenses are *swappable* (one active per step at a time), not stackable; each step ships with a small curated default lens set and the registry stays *open* so new lenses can join without architecture changes
Today's Records Surface renders 96 records in CSV order. The 2026-06-09 ship made augmentation state visible (the `corpus_count` chip + four system columns in v9), but the operator can't *re-order* the list to focus their attention on the tier-2 middle-band records the [[../explorations/Operator-Built-Flows-Beyond-The-Universal-Pipeline]] explored. Sort is the cheapest move that unlocks that scenario. This spec ships sort inside the **Sort & Filter Lens** — the second lens in step 2's default set. The first lens, the existing pack-firing UI (bundle picker + roster + 'fire on 96 rows' button), is named in this spec as the **Pack Firing Lens** so the architecture has language for it. Lenses are *swappable* — one active per step at any moment; the step counter in the FLOW header shows which lens is active and right-clicking it opens the lens menu. The agent has the same authority: `/lens sort` (or 'let me sort these records') swaps step 2's active lens. State persists across swaps so the operator can toggle between Pack Firing and Sort & Filter without losing their bundle selection or their sort. The lens registry is **open** — new lenses appear in a step's menu by dropping a manifest, not by editing a closed enum; the default set per step is the *curated opinion the app ships with*, not the *closed universe of possible lenses*.
-
Chat Context-Awareness Architecture — three layers (workspace as context broker, slab-assembly contract, verb registry) that grow the in-app chat from prompt-only to surface-aware verb router
The agent-chat surface (`apps/chat/`) already has working scaffolding — Anthropic SDK in `services/prompt-runner/src/chat-turn.ts`, four cache-eligible system slabs assembled in `services/workspace/src/chat.ts`, three response modes (chat_answer / chat_propose / chat_invoke), the Svelte transcript + proposal-card surface. What makes it feel narrow is two design holes the v0.0.1 implementation deliberately left open: the verb roster is hand-written and prompt-only, and the context slab inlines `record_set_id` and nothing else. This spec defines the architecture for closing both — workspace as the single source of truth for *what the user is looking at right now and what client they're in*, a slab-assembly contract that names every dimension of context the chat can use, and a verb registry that derives the model's tool roster from `CAPABILITY_TO_SUBJECT` + per-verb args schemas rather than from a hand-edited prose block. The corpus-inbox `/inbox` verb (shipped 2026-06-08 as the first non-prompt verb) is the load-bearing demo: it works because the context slab now carries `active_client_id`, and it points at the seams the spec is here to formalize.
-
Corpus Inbox — capture first, triage later; a zero-friction save destination for URLs without a home yet, with a future triage layer that sorts inbox content into the right places
While researching funders the operator finds URLs that don't yet have a home — articles about the right funder but at a different angle than the active record, cross-funder pieces mentioning several records at once, sector reports, regulatory documents, downloadable PDFs, destination sites worth remembering. The existing Content Reader manual-add affordance requires an active record context; without one the operator either makes a premature filing decision or loses the URL to a tab graveyard. Corpus Inbox is the missing third path: a zero-friction `clients/<client>/corpus/inbox/` destination that captures the URL + Jina-fetched body + operator's drive-by note, holds it in pending state, and waits for triage. Capture vectors at v1: a dedicated microfrontend (`apps/corpus-inbox/`), a `/inbox <url>` chat verb, AND a verb-less conversational path where pasting a bare URL into the agent-chat triggers an 'inbox this?' confirmation (the friction-minimum path for operators who don't want to remember a command); a future browser-plugin capture vector slots in cleanly per the [[../explorations/In-App-Browser-Or-Plugin-For-Corpus-Add]] sketch. The triage layer — direct UI or agent-chat harness that routes inbox items to `corpus/<funder-slug>/`, `corpus/reference/<topic>/`, or `discard` — is scoped here but specced separately. This spec is the *prerequisite* for healthy manual-add work: without an inbox the operator can't research freely without paying a filing tax on every discovery.
-
Funder Content Corpus Workflow — what the system has to do, ranked by quality bar, with no implementation prescribed
Tonight's session went sideways because the implementation got ahead of an explicit shared statement of goals. This spec captures the goals — what the operator wants the funder-content augmentation workflow to actually produce, in what shape, with what quality bar — without prescribing how. Any future implementation choices (refactoring the pack, evolving Content Reader, swapping connectors, adding curation surfaces) must measure themselves against these goals. The deeper failure mode tonight was a class of fixes that filtered yesterday's bad data tighter at display time instead of producing today's good data via a fresh fire; the goals here are written so a future agent can resist that pull. Treat the prior specs ([[Response-Reviewer-Shell-and-Content-Reader-Mode]], [[Flow-for-Bundles-Packs]], [[Entity-Pulse-Bundle]]) as compatible siblings — this spec sits at the level of 'what we want,' those describe pieces of the 'how.'
-
Record-Set Family Grouping — internal lineage vs. external-variant families in the Record Collector sidebar
The Record Collector sidebar lists every record set as a peer, which makes a tracker uploaded as v4 / v5 / v6 / v7 / v8 look like five unrelated CSVs and an in-app promotion chain look the same as a fresh upload. Two distinct family signals deserve first-class representation: the **internal-lineage family** (already in the data model via `promoted_from`) and the **external-variant family** (not modeled — separate uploads the user explicitly groups, or whose filenames share a stem). This spec defines the field, the grouping mechanism, the sidebar rendering, and the interaction with [[Enhanced-Records-List-and-Promotion-Checkpoint]] and the new two-CTA `Augment this Set` panel.
-
Response Reviewer Shell and Content-Reader Mode — Response Reviewer becomes a wrapper whose inner review UI swaps based on what the bundle that fired actually produced
Response Reviewer was built to answer one question, well: 'is this URL the right one?' That's the right question for socials packs (linkedin, x, wikipedia, bluesky) — the candidate is a single URL, the action is accept-or-reject. It's the wrong question for content-shaped packs (official-blog, official-pressrelease, official-social-posts, future YouTube / podcast / RSS packs). The unit of work there is an *article* the operator wants to *read*, not a URL to triage — and the recently-shipped audit (1,109 OfficialPulse responses, 3 accepts, 99.7% reject rate) is what that mismatch looks like at scale. This spec splits Response Reviewer into a **shell** (entity selector, pack-filter chips, navigation, response-store connection) and a swappable **review mode** mounted inside the shell. The bundle that fired the responses declares which mode it pairs with — `profile-builder` → `candidate-triage` (today's UI), `entity-pulse` → `content-reader` (new). The content-reader's job is to show the operator what was found and let them include / exclude items from a downstream Jina ingest queue, not to accept individual URLs into row columns. First slice ships the shell pattern + the content-reader mode for `official-blog-pack` + `official-pressrelease-pack` against funder blog / press / RSS sources — articles only, no video / no social posts yet, expanding to those modes in follow-up. Composes with [[Per-Client-Privacy-and-the-Path-Off-Local]] (the Jina-ingested content lands in `clients/reach-edu/corpus/`), [[OfficialPulse-URLs-Appear-as-Junk-in-Promoted-Versions]] (this is the audit's prescribed replacement for per-URL triage on the OfficialPulse packs), and [[Packs-and-Bundles-Pattern]] (the bundle definition grows a `review_mode` field). Treats [[Response-Reviewer-and-Response-Store]] as the prior art whose shell scaffolding gets refactored, not replaced.
-
Connector Inventory & Per-Record Palette — Hot-Swap Providers, Re-Fire One Record at a Time
Today's connectors are hardcoded into a small ProviderId union and bound to specific packs at compile time. The reality of LLM web research is that providers churn — new ones launch, old ones rate-limit, costs shift, a specific entity needs a specific connector to crack a stubborn search. The system needs to (a) accept any connector at runtime via a registry that resolves by *capability* not by hardcoded id, and (b) give the human, at triage time, a per-record palette of short-labeled buttons — click `f` to re-fire just this record through the next Facebook connector in priority order, or long-press to pick a specific one. The killer UX from prior live attempts: per-row, per-intent, per-connector, one click. This spec scopes the registry, the pack-intent declarations, the bundle-level chain config, the per-record palette, and the integration with the Pulse Curation Layer.
-
Entity-Pulse Bundle — Press Releases, News Mentions, and the Social Voice of a Record
Profile Builder finds the canonical accounts an entity lives at. Entity Pulse finds what that entity has been *saying* and what's been said *about* it lately. Three categories — **Official Updates** (blog + press release + entity's own social posts), **Media Mentions** (news coverage + thematic inclusion + deep analysis), **Socials Mentions** (third-party mentions across platforms) — fire as a foundation-first four-phase DAG: OfficialUpdates first (identifying the entity's own voice), then a rollup-agent synthesizing that voice into a grounding context, then MediaMentions and SocialsMentions on top (with the OfficialUpdates rollup as relevance prior + cross-category dedup), then their rollup-agents. Each item carries two 0-100 scores (confidence + relevance against a fundraise brief); each category lands as a three-layer Pulse Curation state (immutable raw_output / live curated_output / immutable finalized_output snapshot) so the human gates every item with full audit trail. Augmenting a record set with this bundle answers 'what's the current shape of this entity's public conversation, and what survives our curation?' in one fire.
-
Flow for Bundles & Packs — The Simplest Possible Thing
Two surfaces. Record Collector (existing) and a Records Surface (new). For each row, see the entity's name and URL. Click a connector. See candidate links inline below the row. Pick one. Move on. That's the whole flow. No Request Reviewer. No Response Reviewer. No bundle picker. No fan-out preview. No multi-step pipeline. The previous specs (Entity-Pulse-Bundle, Connector-Inventory, Pulse-Curation, Per-Record-Iteration, In-App-Chat) collectively overengineered this problem; this spec deliberately ignores all of them and starts over. The job is: for 67 rows that have a website URL, find each one's blog/news/press index. That's it.
-
Per-Record Iteration as the Primary Surface for Pack and Bundle Fires — The Flow Rearranges Per Fire Type
Pack and bundle fires deserve a fundamentally different primary surface than prompt-template fires deserve. For prompt templates the existing Augment → Request Reviewer → Response Reviewer → Promote sequence works, and the Response Reviewer for socials eventually became lovely. For pack/bundle fires that sequence is wrong from step 2 forward: the Request Reviewer surface is empty, Response Reviewer mixes the current fire's results with every prior fire's results, and bulk fan-out across 96 rows wastes time and credits when the connector chain fails 96 ways before it ever works once. The right primary surface is a **per-record list scoped to the active pack/bundle**, where each row shows only the relevant/visible fields, and a connector palette runs in-place with results landing inline on the row. Bulk fan-out becomes the *secondary* mode, available only after the user has validated a chain on 4–8 records by hand. The Flow chrome adapts to the fire type — same step numbering, different active members per step. This spec describes both the new remote and the Flow rearrangement.
-
Pulse Curation Layer & UI — Three Layers per Category, Triage Per Item, Finalize on Demand
Pulse-shaped bundles (Entity Pulse and its siblings) produce multi-item structured rollups that the human triages item-by-item. The data model needs three layers per category — an immutable `raw_output` from the LLM fan-out (audit trail), a live `curated_output` that updates as the user accepts to canonical, accepts to additional context, or discards each item, and an immutable `finalized_output` snapshot taken when the user marks the triage done. Re-firing a bundle later creates new raw + curated layers; the previous finalized is sticky. This spec codifies the three-layer pattern, the triage actions, the row schema, and the UI shape — the layer the Response Reviewer (per Decision §10) renders for pulse-shaped responses.
-
API-First In-App Documentation — Each Surface Reveals Its Own Docs Inline
Every remote, the shell, and every service ships its documentation inside itself — toggled by a small icon-CTA, revealed inline within the surface rather than in a separate tab. Not generic API docs but relevant ones: the data flowing in, the services called with which payloads, the response shapes, the diagrams. Because augment-it is also a demonstration of API-first architecture, the docs are part of the product, not adjacent to it.
-
Initial User Experience — From Landing to a Productive First Action
What does the first thirty seconds of augment-it feel like? This spec covers the surface a brand-new user lands on, the lightweight onboarding that gets them to a productive first action without lecturing, and the seams where the working-app and architecture-demonstration identities first meet the visitor. Designed to be skipped by returning users, never re-shown unnecessarily, and never required to dismiss to do work.
-
Shell & Micro-Frontend UX Coherence — Affordances That Are Found, Consistent, and Talk Back
One demo-prep session turned into a chain of 'I can't find / reach / trigger X' failures across Pack Runner, Enhanced Records, and Response Reviewer. Individually each was a one-line patch; together they're a verdict — the augment-it shell's affordances hide, die, or mismatch. This spec audits the whole shell + its micro-frontends through that lens and fixes the pattern, not just the instances.
-
Tooltip System — A Versatile First-Class Family, Plus a Walkthrough for New and Returning Users
Most apps don't succeed or fail at functionality. They succeed or fail at onboarding, orientation, and re-orientation when a returning user comes back to find things have moved. Tooltips are the load-bearing affordance for all three — and they have to be a versatile first-class family, not a single component bolted onto the browser's half-working `title=`. This spec scopes a family of tooltip primitives (Simple, Rich, Popover), the geometry they share (directional `from` placement with arrow connector, configurable distance, dismissal discipline, accessibility), and the **Tooltip-Walkthrough** surface that sits on top — a guided series of tooltips that fire for users who haven't logged in since a given milestone, so they discover what changed without reading release notes.
-
Enhanced Records List — the Triage Checkpoint and the Promotion Loop
A new microfrontend that gives the human the first view of the whole enrichment outcome at once — a sortable list of every record across the parent set + its derived sets, deduplicated by a stable record identity, with each record's current triage state, edits, and helpful_links surfaced inline. From that surface the human selects records (typically all accepted + all good) and promotes them: those selected versions become the canonical record set, the predecessors archive, the same record identity carries forward. The next enrichment prompt fires against the canonical set, and the loop continues. The end-game is rich CRM-grade donor / grantmaker profiles built through repeated tight loops of LLM enrichment + human checkpointing.
-
Request Reviewer — the Pre-Flight Surface
Before a prompt fires, you should be able to see exactly what is about to leave the building. request-reviewer is the augment-it stage that shows the resolved request — the prompt template with every {{token}} replaced by a real row's property value — lets you pick which model it goes to, optionally shows the literal JSON request body, and then fires it. The hard part is not the UI; it is guaranteeing the request you reviewed is byte-identical to the request that sends. That guarantee is the spec.
-
Response Reviewer and Response Store — the Post-Flight Surface
Once a request has fired, the verbose prose the model returns has to be inspected before any of it reaches a CRM cell. response-reviewer is that inspection surface — and it cannot exist until a fired response becomes a first-class stored object instead of a bare cell value. So this spec defines two things at once: a new response-store service that records every fired response with its request, model, and review flag; and the response-reviewer remote that reads it, lets a human triage good/partial/wrong, and either accepts a whole response straight into a cell or sends the row back to be re-run.
-
Walking-Skeleton Pre-Flight Decisions — Augment-It Rewrite
Five decisions that have to be settled before we can write code for the augment-it rewrite. Captured here as a single load-bearing doc so a future session can pick up the build without re-walking the decision tree in chat. State management is hand-rolled useSyncExternalStore. Transport is HTTP fetch to a bun sidecar at localhost. Persistence is JSON files written by the sidecar; graduate to libSQL when multi-device demands it. Auth is opaque session tokens auto-minted by the sidecar on first contact — real user_ids and sessions without an OAuth detour. The @lossless/in-app-agent scaffolding question remains open pending a final call on whether augment-it drives the package's first implementation.
-
Augment-It as a CRM-Augmentation Pipeline (Microfrontends + Microservices)
First-pass spec capturing what Augment-It actually is, where it sits in the family of Applied AI Labs apps (memopop generates memos, dididecks generates slides, augment-it generates information that goes back into a CRM), and what the minimum viable path looks like given a paying client needs a fundraising lead list augmented this week against a repo that is currently a clean rebuild on rsbuild + Module Federation + Turbo. Journey-mode: the destination isn't pinned, the team's understanding is.
prompts
-
Common-Six Social Packs — First Real Packs on the Structured-Output Surface
Six pack identities (linkedin, x, bluesky, youtube, facebook, wikipedia) registered against one consolidated backend service that routes by pack_id, with Tavily as the underlying search and URL-shape verification driving the confidence score. No new microfrontend per pack — Response Reviewer's generic candidate card already renders every pack response. A small new Pack Runner remote provides the per-row × per-pack fan-out affordance so the foundation dataset can be augmented and reviewed today, before the bundle abstraction lands in a later session.
-
Helpful Links on Records — Captured During Response-Reviewer Triage
While a human-in-the-loop is triaging LLM responses in Response Reviewer — google-searching the unclear ones, fixing wrong URLs, deciding whether an org even exists — they routinely stumble onto adjacent links worth keeping with the record (a foundation page, a personal LinkedIn, a press release, a related grantee site). Today those links die in the human's clipboard. This prompt scopes a `helpful_links` array field per row, attachable inline from Response Reviewer with one click, surviving across record-set derivations so the next enrichment round can read them.
-
Response Reviewer — Structured-Output Extension (Packs-and-Bundles Foundation)
Extend the shipped response-store schema and the shipped Response Reviewer remote to support pack-shaped responses — structured candidates with confidence-band pills, a five-value outcome enum, and a sibling-payload + archival-markdown shape — without breaking the free-form text responses that today's prompt-runs produce. Zero packs ship in this work. Zero bundles ship. The deliverable is the surface that every pack will land on, exercised by a small fixture of mock pack responses so the renderer can be felt in all three theme modes before any real pack exists.
-
Run /speckit-specify — Response Reviewer Structured-Output Extension (Packs-and-Bundles Feature 1)
The first Spec-Kit feature in the packs-and-bundles series. Extends the shipped response-store schema with sibling-payload structured output, the five-value outcome enum, archival markdown, and the pack/bundle correlation fields — and extends the shipped Response Reviewer remote to render the structured payload as a candidate card with a confidence pill. Zero packs are implemented in this feature. Zero bundles are implemented. The work is the surface that every pack will land on; the next three features (linkedin-pack, profile-builder.common bundle, then two-pass + entity-typed bundles) ride this foundation.
-
Build the Shell Tiling & Peek-Deck — Window the Federated Frontends
The augment-it shell stops being a one-at-a-time tab switcher and becomes a desktop-window-tiling surface. Two layout modes: a peek-deck where the focused frontend sits at 90% width with its sequence neighbours peeking from the edges and hover-expanding, and a co-existence split where two frontends share the viewport at a configurable ratio with a draggable seam — entered when a prompt is run against a single record. Full-width single-frontend stays available as a third option. Layout state is User-scoped preference, persisted to localStorage now against a declared shape that the forthcoming shared-auth user store will back later.
-
Run /speckit-constitution — Seed Augment-It's Constitution from Context-V
The directive to paste alongside /speckit-constitution to fill .specify/memory/constitution.md from augment-it's already-documented thinking. Five principles, two constraint sections, governance — all grounded in the context-v blueprints and specs so spec-kit synthesizes from what we've actually decided rather than re-deriving from training data.
-
Run /speckit-specify — Workspace + Sidecar Foundation (Feature 001)
The first feature spec for augment-it after the constitution lands. Scope: the workspace package and bun sidecar that everything else depends on, plus exactly one read-only capability (records.import) end-to-end to prove the pipe. No federation yet. No research agents yet. No chat surface implementation yet. Just the foundation that makes every subsequent feature possible — and a CSV import that lands rows in the workspace and persists them to disk.
reminders
-
Pickup notes — 2026-06-03
Late-night session that ended in an actually-working per-record augmentation surface. Records Surface at :3011 wired into the Flow at step 3, three connectors (Firecrawl scan, Firecrawl + Haiku agent, SerpApi), inline edit on name + URL, multi-URL accept (array column), promote-to-next-version save bar at top + bottom with post-promote navigation to Enhanced Records / Augment / stay. User ran the loop end-to-end on the 96-row pipeline tracker — promoted v5 → v6 successfully. Also: backup of v5 work pulled to .backups/ (98 URLs in 45 rows safe on host fs), Record Collector decomposed into RecordSetsList + RecordSetCard components with descending-by-created_at sort + per-card CSV download, and a deep audit of v3 → v4 → v5 that exonerated promote (it works) and named the actual data-persistence failure mode (URL edits routed to helpful_links instead of url). Three new context-v docs land the lessons. Branch is feat/bundle-media-packs, last commit pushed is 7e449db (Records Surface initial), HEAD has uncommitted improvements + new specs.
-
Pickup notes — 2026-06-02
Today's session closed the Shell UX Coherence refactor's last threads (Tooltip-System sharpened to a family + Walkthrough, bundle picker dropdown, tag v3.0.0.3 cut and pushed) and then went deep on a NEW bundle: **Entity Pulse**. By session end, four interlocking specs cover the bundle's design — Entity-Pulse-Bundle itself (foundation-first 4-phase DAG with three categories, seven source-bound packs, three rollup-agents), Pulse-Curation-Layer-and-UI (three layers raw/curated/finalized + four triage actions), Connector-Inventory-and-Per-Record-Palette (registry + per-record per-intent button palette), plus the prior Tooltip-System for the chip menus. Nothing implemented past the existing Phase 0-6 UX refactor; all four new specs are Draft. Migration plans cross-reference each other; foundation-first sequencing means OfficialUpdates ships first, gates the rest. The user said start fresh; this is your map.
-
Record Count Stays Stable Across Versions — Augmenting Adds Columns, Never Rows
1 dataset = 1 row count. Forever. From ingest through every promote, every augment, every triage. If the user has 96 records, every surface that says a number should say 96. Adding properties + values to records becomes new COLUMNS on those records, not new rows in the store. Discovering 231 rows in the row-store for what the user knows is a 96-record dataset is a symptom — of either ghost data from earlier record sets, version-multiplication of the same 96 records across v3/v4/v5, or a leak in the ingest/promote path. Either way: the user never wants to see a number other than their dataset's record count, and the architecture must reflect that.
-
Pickup notes — 2026-06-01
Today's session pivoted from reactive demo-prep patching into spec work. The user named the pattern — affordances that hide, die, mismatch, or misname — and asked us to stop patching and develop a spec. We did. The whole-shell UX-coherence spec is now Draft with 8 locked decisions, 12 evidence items, 3 confirmed failure shapes + 1 emerging, 4 surfaces audited (Pack Runner, PTM, Record Collector, Shell), 3 open architectural questions, and a 'wish list' that already spawned four sibling context-v stubs (two specs, two blueprints). Nothing was implemented past the live demo-prep patches; everything else is held at the sign-off gate. The user explicitly said: don't start implementation before sign-off.
-
Pickup notes — 2026-05-27
Where we left off the night of 2026-05-26: the two-day packs-and-bundles arc closed cleanly. The structured-output extension is shipped, the social-search microservice is in the stack, pack-runner is live at :3009 paired with prompt-template-manager, the response-reviewer (:3005) by-record triage cockpit is in a good place with inline URL / display_name / entity-name editing + stale-companion-field discipline in the store, and the Packs-and-Bundles-Pattern blueprint absorbed eleven emergent UX requirements from the foundation-dataset smoke. Three docs got written tonight that name the next several arcs without committing to which one fires first: the Run-as-First-Class-Operation plan (Parts 2–6 sequenced, only Part 1 shipped), the Search-Providers-as-First-Class issue (filed, reframed 2026-05-27 from a Tavily→SearXNG swap to a provider-plurality architectural concern with SearXNG joining as a peer + new default for social packs, Tavily staying as a peer for content-RAG packs, and per-row provider iteration named as the workflow we're building toward), and a new Agent-Chat-Skills-and-Commands-Candidates exploration that lists ~20 verbs the chat could route to but currently can't. Plus the README finally got rewritten to match reality (Svelte 5, not React+Next.js). These are the threads to pick up — roughly in priority order — for tomorrow.
-
Pickup notes — 2026-05-23
Where we left off the night of 2026-05-22: the four-stage enrichment pipeline (Prompt Templates → Request Reviewer → fire → Response Reviewer) has all four remotes built with working UIs, but nothing has been run end to end. These are the threads to pick up — roughly in priority order — as seeds for tomorrow's prompts.
-
Pickup — Build the Enhanced Records List + Promotion Checkpoint
Next session opens with the full spec already in hand: enhanced-records-list, a record-grained checkpoint surface that snapshots the whole triage state per record_uuid into a new canonical record set on promotion. This pickup points at that spec, summarizes the uncommitted-but-working state from the 2026-05-22 session (Save/Apply on Prompt Templates, autosave on response edits, helpful_links on rows, runner observability/cancel, response triage state ergonomics, the needs-human flag, blank-row cleanup), and queues the concrete first build step: the record_uuid plumbing + backfill.
explorations
-
JuiceFS — POSIX-Over-Object-Storage (Explored, Then Set Aside for rclone-to-R2)
Explored JuiceFS to give the corpus a 'path off local.' Verdict: wrong shape for one-person, local-first content work — it's a network drive needing a macOS kernel extension, not Dropbox. Superseded by automated rclone sync to Cloudflare R2. Kept for the hard-won R2-credential recipe (which still works).
-
The Moat Is Grounded Deliverable Production, Not Chat
Chat-over-documents is commoditizing fast; the durable value is the last mile — turning a client's knowledge into the deck or memo they actually ship, with every claim traceable to source. dididecks and memopop are two renderers of one grounded content model.
-
Funder-Fit Engine — Org-Anchored Corpora and the Bidirectional Story↔Funder Cycle
Fundraising is a two-way match: understand a funder and build a story that answers them, or take a committed story and find the funders it unlocks. Both directions run on one substrate — a trustworthy, per-organization corpus. This maps the path to that engine, and argues the differentiator is KAG (graph-grounded), not plain RAG, because we just built the graph.
-
Forced One-By-One Tag Selector — a pulse-dimension pattern for deliberate triage instead of lazy bulk-tick
Bulk-checkbox UX produces uniform half-considered tags. We want a reusable pulse-dimension component that presents one tag at a time, with apply / skip / skip-rest controls, so each tag forces a deliberate per-entity decision. Surfaces first as a 'flag high-value under-the-radar lead' affordance on the person-enrichment worklist; reusable across orgs (sector / region / funder-priority), corpus content (quality tier, sensitivity, voice-fit), and any triage flow where deliberation matters more than throughput. Pattern is abstracted: instantiate with a tag set + an entity ref + an onApply callback; the component owns the one-at-a-time stepping and the keyboard rhythm.
-
Joined People UI and the Network-First Pivot — augment-it's sibling-flow to org-first augmentation, and the canonical/proprietary layer split underneath it
Org-first and people-first aren't two flows but two pivot points over the same org↔people join. A canonical layer (LinkedIn truth, refreshable, shareable across clients) split from a proprietary layer (this client's commentary, engagement-scoped, never shareable) — reads it against the CRM platforms that almost got it right (Attio, Affinity, Gong, SalesQL, Crawlbase), and lands on sub-scale plus domain-opinionated -- as the competitive position the big platforms structurally can't occupy.
-
LinkedIn Network Explorer — slicing your own connection graph by geography for curated invites, when LinkedIn won't let you query it directly
A client is hosting a dinner in Manhattan and the operator wants to invite their LinkedIn contacts who actually live in NYC. LinkedIn's public API removed the connections endpoint years ago, their Terms of Service explicitly forbid scraping, and they ban accounts that get caught. So the question isn't 'how do I scrape LinkedIn' — it's 'given that LinkedIn is hostile to programmatic querying of your own network, which legitimate paths let you produce a geo-filtered slice of your connections in time for next week's dinner, and which of those compose with augment-it's existing pack-runner / record-set / response-reviewer stack so the same pattern works for the next client dinner and the one after that.' This exploration walks the four paths (data export + enrichment cascade, Sales Navigator subscription, third-party scraping services, direct careful scraping), names the legal/ban posture of each, and lands a recommendation that dogfoods augment-it — because the operator has literally built the tool for 'augment a list of contacts with metadata you don't have yet,' and this use case is the canonical instance of that pattern.
-
Best Way to RAG Over the Corpus — design space for retrieval-augmented operations on augment-it's per-client corpus, given the structured frontmatter we just earned
245 .md files now carry first-class `published_at`, `record_id`, `record_uuid`, `funder_slug`, `pack_id`, `tags`, `binary_asset.sha256`, and `extra_metadata.{language,description,content_length_bytes}`. That metadata is the differentiator — RAG over augment-it's corpus is not 'embed-and-cosine-search'; it's hybrid retrieval where the structured facets (date, funder, source, document type) do most of the filtering and dense embeddings handle the residual 'about-ness.' This doc walks the design space: vector store choice (local Chroma is the right starting point, given the wider Lossless tree's existing Chroma MCP and the `chroma-local` / `search-lossless-corpus` skills), chunking strategy (markdown H2-aware with overlap, lineage-stable doc IDs keyed by `record_uuid`), the per-funder collection vs single-with-filter trade-off, where the retrieval lives in the runtime (a `corpus.retrieve.requested` NATS capability mirroring the existing `corpus.list_for_record` shape), how the inbox composes (indexed separately under its triage discipline), and what the operator-facing surface looks like (cited answers in chat; retrieval-backed chips on the Sort & Filter Lens; an MCP server export so other tools in the tree can query the same corpus). A phased plan ends the doc — Phase 1 ingest to local Chroma + the retrieve capability; Phase 2 chat-surface citations and lens-side filter-by-retrieval; Phase 3 graduate to Chroma Cloud when multi-user reach matters or local resources become the bottleneck.
-
Inbox Sort by Agent Tasks — moving the inbox from a flat 86-item pending queue into a task-typed work surface where the operator sees what's next and the agent does the safe parts automatically
The inbox today is a flat list — 86 .md files with `inbox_status: "pending"`, all rendered identically in the lens regardless of whether they're (a) a fresh capture that obviously belongs in `corpus/hewlett-foundation/` based on title alone, (b) a PDF the operator dropped that needs body extraction before it's useful, (c) an inbox file that's a near-duplicate of one already in an assigned funder dir, or (d) a low-signal scrape that should probably be discarded. The operator sits down at the lens, sees 86 items, and has to context-switch between four different task types as they scroll. Treating those as separate sortable buckets — each with its own affordance, its own agent-assist scope, and its own confidence band — turns the inbox from a backlog into a triage console. This doc maps the task taxonomy (TRIAGE, EXTRACT, ENRICH, DEDUPE, FLAG, DISCARD), what each task can be done by an agent vs requires a human, how to bake the suggested-task into the file's frontmatter at capture time (an agent-proposed `triage_suggestion:` block) so the lens has something to group by without recomputing every load, the confidence bands that gate auto-action vs human-confirm vs human-decides, and how this composes with the retrieval doc's idea of corpus-level semantic search. A phased plan ends with the simplest move: stamp `triage_suggestion:` at capture time using the title + URL + funder roster, group the lens by task type, and let agent-doable tasks ship a 'do it for me' button next to the operator-decides ones.
-
Operator-built flows beyond the universal pipeline — the default augmentation path is a dud for most edge cases, and the cheaper-than-microfrontend escape is a view-spec the agent-chat composes from natural language and the UI renders generically
Today's `augment-it` assumes a linear flow: fire packs against records, triage responses, save to corpus, repeat. That works for the records where SerpApi + Firecrawl + Jina hit cleanly — about 17 of 96 in the reach-edu working set. For the rest, the flow is a dud: the bottom ~25% are deliberately private (high-net-worth individuals, small foundations operating by reputation) and have no digital footprint to scrape; the middle ~60% have URLs and socials but our automated fan-out misses what a human eye catches in two minutes of search-engine browsing. Going to the client with content informed by 17 of 96 funders would look like we didn't really try. The fix is *not* a new microfrontend per scenario — there will always be a next scenario — but a *generic spec-renderer* the operator and the agent both compose against. Sort + filter + named views first, sequenced playbooks second, agent-generated view-specs third; the formal flow-editor that competes with n8n is the never-shipped fourth tier unless and until the prior three prove insufficient. The differentiator from n8n / Flowise isn't UX polish — it's that augment-it knows about *records as the implicit data spine*, which means a flow's verbs can pre-bind to a record subset and skip the data-plumbing primitives that make general-purpose flow builders feel like overkill.
-
In-App Browser vs Browser Plugin for Corpus Add — bridging the operator's own search into the per-record corpus without leaving the cockpit
The Content Reader now has a manual-URL field — paste a URL the operator found via their own Google search and it lands in the funder's corpus. That closes the immediate gap but leaves the friction of leaving Augment-It to do the search, finding the URL, then copying it back into the right card. This exploration looks at two ways to collapse that round-trip: (1) an in-app browser/search surface — embed a search experience inside Augment-It so the operator never context-switches; (2) a browser plugin — let the operator search wherever they normally do (Chrome, Safari, Arc) and post the active tab's URL into the active record's corpus with a single keystroke. Each path has real friction-cost / engineering-cost / security-cost trade-offs. The doc captures the trade-offs, the unknowns to resolve before committing, and a recommendation.
-
Per-Client Privacy and the Path Off Local — when does single-operator-local-only stop scaling, what stack do we reach for, and how do we architect today so the move is cheap when it comes
Two forcing functions converged in the same week. (1) The funder-content corpus the operator wants to build for reach-edu — Jina-pulled markdown from each funder's blog / press / RSS — must be private per client; reach-edu's corpus and Laerdal's would share little, and both treat the material as highly sensitive. (2) Across all ai-labs projects, the single-operator-on-localhost posture is bumping up against the seam where typical SaaS concerns (auth, workspaces, per-client isolation, admin controls) would normally live. An active collaborator is joining; a client wants login soon. The exploration's job is NOT to commit to a stack now — the operator's instinct that 'we're not quite there in terms of actually needing it, maybe in a few weeks' is the right call — but to (a) map the option space across repo topology, storage substrate, identity, and multi-tenant data model, (b) identify which architecture choices made TODAY (about the funder corpus, the augment-it record store, the chroma collections) preserve cheap optionality for the move off local, and which lock us in, (c) name the decision-forcing functions that would flip 'not yet' to 'now'. Anchor reference: the dididecks-ai client-private-repos pattern (calmstorm-decks et al.). Candidate stacks include all-local-private-repos, Railway-Postgres-per-client (single-tenant), Railway-Postgres-multi-tenant with auth, hybrid markdown-in-repos + Chroma-for-retrieval, and managed BaaS (Powabase.ai surveyed, flagged expensive). The exploration ends with a recommended posture, not a decision.
-
Agent-Chat Skills and Commands — Candidates List
The in-app chat surface (apps/chat) is augment-it's verb-routing layer — the place a user states a goal in prose and a capability handler executes it. Today's roster is small and prompt-focused: `prompts.draft`, `prompts.improve`, `prompts.apply`, `records.list`. The product needs many more verbs to be useful, and several scripts that already exist under `scripts/` and `services/*/scripts/` are the natural first candidates to graduate from shell-only into chat-callable capabilities. This doc maintains the running list of candidates — what they do, what shape they need to take (TS handler / ScriptCapability / McpCapability / SkillCapability), what gating they require (destructive vs read-only, scope-aware vs global), and what the chat-side phrasing should look like. Treated as a living roster — the next session adds entries as they get nominated; promotion out of this doc happens when a candidate becomes a spec or lands in `packages/workspace/src/capabilities/`.
-
Augment-It Prior Art Survey — What's Already Been Built, and What It Tells Us
Before we rewrite Augment-It on the new stack, we owe ourselves a clear-eyed look at what's already been built. Two parallel attempts exist: Tanuj's split-into-microfrontends approach across three Next.js / Vite repos (record-collector, prompt-manager, request-reviewer — the only one that actually got Module Federation working) and an earlier monolith by Michael on the archive/bolt-code branch that covers more pipeline stages with multi-provider LLM handling and Supabase auth. Both capture most of the core functionality. Neither is what we'll ship. This survey is the map.
-
Bolt-Era API Provider Widget Analysis
Lifted from `augment-it-bolt/specs/APIProviderWidget-Analysis.md`. Per-feature breakdown of the bolt monolith's API provider configuration widget — the surface that let users choose between Claude, GPT, and Perplexity per prompt section, with per-call options. The widget itself won't survive the rewrite verbatim, but the multi-provider configuration *shape* is the right starting point for the rewrite's model-routing UI.
-
Bolt-Era Codebase Analysis
Lifted from `augment-it-bolt/specs/Bolt-Codebase-Analysis.md` (the archive/bolt-code branch of lossless-group/augment-it). A comprehensive architectural analysis of the bolt monolith — React + TS + Vite + Supabase + zustand. Stale relative to the codebase we're rewriting toward, but a faithful map of the prior state and worth keeping as a reference for what the entity model and component graph looked like when it was working.
-
Bolt-Era Highlight Collector Analysis
Lifted from `augment-it-bolt/specs/HighlightCollector-Analysis.md`. Per-feature breakdown of the highlight-collector module — the pipeline stage where users mark up LLM responses with section_title + color spans and persist them. One of the two stages that never got split into Tanuj's microfrontend repos, which means this document is the only existing description of how that stage works. Load-bearing reference for the rewrite.
-
Bolt-Era Main Container UI Analysis
Lifted from `augment-it-bolt/specs/MainContainerUI-Analysis.md`. The longest of the bolt-era analyses (~750 lines). Describes the app shell, authentication gate, routing, and layout structure of the bolt monolith. Most directly relevant to the rewrite's shell decision — does augment-it have one host shell (with mounted microfrontends), or is each app its own host?
-
Bolt Monolith As Built — The archive/bolt-code Branch
The earliest working version of Augment-It is a Vite + React + TypeScript monolith on the archive/bolt-code branch of lossless-group/augment-it, built (probably with Bolt.new, hence the branch name) by Michael before Tanuj split things into microfrontends. It is feature-richer than any of the federated repos — it covers five of the six pipeline stages, has multi-provider LLM response handling, Supabase auth, and Tanuj's own per-feature analysis specs already sitting in a specs/ folder. It is also a monolith and uses an auth substrate we won't keep. This doc captures what's there so the rewrite can lift the ideas and discard the architecture.
-
Bolt-Era Prompt Section Analysis
Lifted from `augment-it-bolt/specs/PromptSection-Analysis.md`. The shortest of the bolt-era analyses (~200 lines). Describes the editable+previewable prompt section feature — prompt templates as MDX-rich, variable-aware, edit-toggle-preview blocks. The pattern that the rewrite's prompt-template-manager microfrontend should likely adopt.
-
Bolt-Era Record Collector Analysis
Lifted from `augment-it-bolt/specs/RecordCollector-Analysis.md`. Per-feature breakdown of the record-collector module in the bolt monolith. Worth cross-referencing against `Tanuj-Record-Collector-As-Built.md` — Tanuj later forked record-collector into its own Next.js repo, and the differences between this bolt-era version and Tanuj's split version are instructive about which patterns survived the federation experiment.
-
Entity-Profile Augmentation Workflow — Common Social + Vertical-Specific Sources, Per Entity Type
A common workflow in augment-it is filling in 'who is this' and 'where are they on the web' for every row in a dataset — LinkedIn, X, BlueSky, YouTube for everyone, plus type-specific sources (ProPublica Nonprofit Explorer + Candid for philanthropic orgs, SEC EDGAR for public companies, Crunchbase for startups, etc.). Today this work happens prompt-by-prompt, one column at a time. This doc converges on a two-tier abstraction: **packs** as atomic, source-bound microfrontend+microservice units (one per data source), and **bundles** as workflow-shaped compositions of packs (a `profile-builder` for philanthropic orgs vs for VC firms). Verification rides the existing Response Reviewer surface with a structured-output extension.
-
Federation and Bundler Decision — Bun + Rsbuild + Module Federation, Workspace State, Optional Chat
The single architectural decision the augment-it rewrite has been gated on. We federate. The bundler is rsbuild (not Vite). The workspace orchestrator is bun (not turbo). State of truth lives in @augment-it/workspace — a per-app package following the shared workspace conventions — not inside the chat package. The @lossless/in-app-agent chat surface is mounted as one consumer of that workspace through a typed WorkspaceAdapter; non-chat consumers (Window microfrontends, headless workflow scripts) consume the workspace directly. Tanuj's federation experiment proved Module Federation works but exposed a smoking gun around shared state; the singleton-module idiom on @augment-it/workspace dissolves that gun, because every consumer reads from the same instance.
-
Tanuj's Prompt-Manager As Built
The scaffolding repo. Next.js 15 + React 19 + JavaScript, with a full CRUD-shaped UI for prompts (cards, search, statistics, create/edit modals, import/export) — but no store, no real data flow, and a `samplePrompts` array hardcoded inline. Honest about what it is: the right shape, none of the wiring. Worth keeping the schema, throwing away the implementation.
-
Tanuj's Record-Collector As Built
The most-finished of the three microfrontend repos Tanuj split out. A standalone Next.js 15 app with a working ingest → configure → augment → export flow, a zustand-with-persist store, Perplexity-only LLM substrate, and one genuinely clever idea — the prompt template auto-generates from whatever columns the imported CSV has. Not the architecture we're keeping, but the field-aware prompt-generation idea is.
-
Tanuj's Request-Reviewer As Built — The Module Federation Proof
The smallest of the four sources, and architecturally the most important. Vite + React 19 + Module Federation. The only place Tanuj actually got cross-app code sharing to work — a federated `RecordCard` component imported from `@module-federation-vite/ui`. The augmentation logic, however, is duplicated from record-collector by copy-paste, which exposes the seam that the federation experiment didn't resolve.
-
Multi-Agent Research Fan-Out Per Row — The Real Shape of Augment-It's Capability Runtime
Augment-It's capabilities are not single LLM calls. Each row in a CRM export gets enriched by multiple specialized research agents running in parallel — one fetches social profiles and vanity KPIs (X handle, X followers, LinkedIn), another crawls every linked page on the company's website looking for team changes, press mentions, blog posts, and PDF publications. The capability runtime, the @lossless/in-app-agent transcript shape, and the chat UI all need to handle (rows × agents) fan-out, partial results, heterogeneous latencies, per-agent rate limits, and per-agent caching. The memopop character-cast pattern is the UI surface this maps to.
issues
-
Grilling on the DB Resolver — questions to settle before v0.0.0.2 / v0.0.0.3
Before building the next resolver iterations, a hard pass over the open decisions — where the join key lives, whether a slug can be edited without breaking everything, and whether 'opportunities' reopens the CRM-stays-in-Decile call. Answers get folded back into the spec.
-
Personal-link observations need named query lenses — without them, an accumulating fact log goes uninspected
The `has_personal_link` observation predicate added with the PersonalLinks pulse-dimension stores rich qualifiers (`kind`, `title`, `url_domain`, `org_id`) that we expect to query in specific shapes — per-person presence rollup, per-org corpus rollup, kind-filtered slices for analyst views. None of those queries exist yet. As personal-links accumulate (a single thought leader can easily contribute 50+), the operator will hit 'I have the data but no way to look at it' unless we name the lenses up front and ship them as we hit the friction.
-
Some records show empty corpus in the Sort & Filter Lens despite per-funder directories existing on disk — the lineage join is healthy but the workspace-service capability has no timeout override (defaults to 5000ms), the content-ingest handler processes requests serially, and each call walks every funder directory under clients/<id>/corpus/, so late requests in the 96-row fan-out time out and the lens swallows the error silently; the durable fix is to make corpus_funder_slug (a column the records sheet already populates) the primary join key, which limits each call to one small directory and dissolves the timeout race — this supersedes the originally-proposed corpus-overrides.yaml because the override surface is now the records-sheet cell
End of 2026-06-09 session: corpus content built for ~21 new records (warm 17 → 38 between v9 and v10). Operator saw four rows showing `corpus 0` despite per-funder directories on disk — sobrato-philanthropies, stand-together-trust, steve-and-alexandra-cohen-fnd, todd-fisher. UPDATED 2026-06-10 follow-on session: a hard-refresh cleanly resolved those four but exposed three more — hewlett-foundation (2 files on disk), howard-schultz-foundation (6), kellogg-foundation (6). A second backend probe confirmed `corpus.list_for_record` is returning the correct counts for those three too. The actual bug is in the wire: `corpus.list_for_record` has no `CAPABILITY_TIMEOUTS_MS` entry so it falls through to the default 5000ms, the content-ingest handler is a serial `for await` loop, and each call walks every funder directory. With 96 visible rows firing 96 parallel requests, late ones in the burst time out and the lens `catch` block swallows the error → chip never updates. The simpler, durable fix the operator proposed — join via `corpus_funder_slug`, a column the records sheet already populates per row — dissolves all three causes at once: each request now walks one ≤13-file directory instead of 301 files, the override surface is a sheet cell instead of a new YAML format, and lineage stays as a fallback for rows without a slug. The originally-proposed `corpus-overrides.yaml` mechanism is SUPERSEDED. Backfill of `record_uuid` into 201 unstamped files stands as independent hygiene (script built this session, dry-run found zero stale conflicts).
-
Funder Corpus First Session Failed — 75 markdown files across 15 of 96 funders, 81 records unprocessable, hours spent layering filters on stale data instead of producing clean fresh data, rebuild against the goals spec exists in code but was never validated against a real pack fire because the session ended
After a multi-hour session on 2026-06-05 building toward the per-client funder content corpus for reach-edu, the operator stopped because the workflow was unworkable. Final state: 75 corpus markdown files written to `clients/reach-edu/corpus/` across 15 funders (Annie E. Casey, Arnold Ventures, Arthur Blank Foundation, Ascendium Education, Ballmer Group II, Bridgespan, Carnegie Corporation, Charles and Lynn Schusterman, Charles Koch Foundation, Daniels Fund, ECMC-2, Education First, Gitlab Foundation, Greater Texas Foundation, Heising-Simons Foundation). 81 of 96 funder records have zero corpus content. The operator's stated framing: 'I was only able to process like 1/3 of the plausible records' and 'this is all fucking bullshit, nothing is working the way it should.' This issue captures what failed so morning-self has a target list and doesn't repeat the failure modes.
-
OfficialPulse URLs Appear as Junk in Promoted Versions — operator believes promote v6 → v7 → v8 wrote bad data into the array column; audit shows promote is clean and v6 itself already contained every URL
The operator reports that v7 and v8 of the Master-Pipeline-Tracker carry 'junk' OfficialPulse URLs (news / blog / stories / grants / press indices) that weren't in v6, and proposes a recovery of (a) hand-fixing the v6 → v7 export, (b) deleting v8, (c) hardening the next promote so it doesn't recur. The audit inverts that framing: v5, v6, v7, and v8 all carry IDENTICAL `official_updates_index_urls` arrays — 122 URLs across 57 rows, zero per-row diffs across any consecutive pair. The promote is doing exactly what the 'All data continues' invariant from [[Original-and-Enhanced-Record-Instances]] requires. What the operator is *seeing* for the first time is the JSON content of an array column that used to render as a blank box in Record Collector — today's `apps/record-collector/src/logic/format.ts` formatter surfaced it. The operator's claim 'I had approved the proper links' is consistent with the pick-path code (`RecordRow.svelte:63 pick()` appends one URL at a time, no silent bulk-write), but multiple records-surface sessions accumulated to 122 URLs and the cumulative result is what's now legible. The real gap is a curation surface — Record Collector's structured-value branch is read-only; there is no per-URL remove affordance anywhere in the UI that operates on the canonical column. This issue closes the data question and reframes the next step as adding that affordance, plus a pre-promote audit step in PromoteBar.
-
Augment Transformations Not Reliably Persisting — Hand-Curated Field Edits and Whole-Row Augmentations Are Disappearing Across Record-Set Versions
Two failures observed in the same 2026-06-02 session, escalating from immediate to systemic. (Immediate) The user just hand-curated 98 OfficialUpdate URLs across 45 records on the new Records Surface — there is no save / promote affordance in that UI, so the only thing keeping those acceptances alive is the row-store's auto-persisted JSON on disk. A backup has been pulled to `.backups/2026-06-02_records-surface-acceptances/` against accidental data loss. (Systemic) The active record set v5 is missing the homepage-URL edits the user made in a prior v3→v4 augment pass — those edits ARE in the row-store's earlier record sets but did not carry forward through promote. The product's whole value premise is 'augment a record set, promote it to the next version, and your work survives' — that is currently not true for some field categories.
-
Troubleshooting UI for Official Blogs — The Bundle Fire Path Doesn't Fit the Flow That Was Built for Prompt Templates
Entity Pulse Phase-1 bundles (entity-blog, entity-officials) are now selectable in the Augment remote and fire end-to-end against real rows — but the Flow that wraps Augment was built around prompt-template fires, not bundle fires, and at every step downstream of step 2 the user hits a surface that makes no sense for what they just did. Request Reviewer (step 3) has no preview of what's about to be sent for a bundle fire; the navigation to Response Reviewer is blocked behind it; Response Reviewer (step 4) only renders responses after they exist; and the per-record fire affordance lives in Response Reviewer instead of in Augment where it should be visible from step 2. Plus a NATS payload-limit crash that ate the first fire's responses. The chip palette pattern is sound; what's broken is the flow's assumption that everything routes through a prompt template, and the gap between the bundle picker and the per-row fire surface.
-
Search Providers as First-Class — Stand Up SearXNG as the New Default for Social Packs; Tavily Stays as a Peer; Per-Row Iteration as the Workflow We're Building Toward
The social packs route through Tavily today, but Tavily is a content-RAG index — wrong substrate for sparse-text JS-rendered social-profile pages. The fix is **not** to swap Tavily out. The fix is to make **search provider** a first-class concern in the architecture, stand SearXNG up as a peer, flip the common-seven social packs so SearXNG becomes their new default, and keep Tavily wired in as a peer for the content-RAG packs/bundles that will want it (deep-research, document extraction, annual-report summarization). The deeper goal — the reason this is more than a refactor — is that the product is being shaped around an **iteration loop**: a user finds a `not_found` or low-confidence response in the by-record triage view, suspects the provider is the issue, and wants to re-fire the same pack against the same row through a different provider (SearXNG → Brave → Google CSE → direct-API connector) until the accurate data surfaces with the fewest API calls. That loop only exists if providers are plural and selectable per-fire.
-
Changelog entries duplicated across augment-it/changelog/ and content/changelog--laerdal/
Eleven backfilled augment-it changelog entries currently live in two places. Either location can be the source of truth — but right now both are, and updates have to be made twice.
general
blueprints
-
Connecting To And Using SurrealDB
The one place that codifies how augment-it talks to SurrealDB Cloud — the five env vars, the connect → signin → use dance, the client-tagging write contract, and the dev-only-credentials posture that a proxy service eventually replaces.
-
Augment-It as Working App and Architecture Demonstration
Augment-it lives two lives at once — a working product that augments CRM-style record sets, and a living showcase of microservices + microfrontend + API-first architecture. This blueprint names the dual identity, draws out the tensions it creates, and codifies the discipline of evaluating every architectural and UX decision under both lenses so neither identity quietly wins.
-
Auth Patterns — Following Astro Knots Conventions in a Rsbuild + Module-Federation Shell
Augment-it isn't a pure Astro site — the shell is Rsbuild + Svelte + Module Federation, and the backend is NATS-fronted microservices with a workspace-service holding session tokens. But the Astro Knots auth conventions (session cookie, middleware gate, public allowlist, never-prerender-a-gated-route) still encode the right discipline. This blueprint translates each rule into the augment-it architecture so the audit-trail and the failure modes stay legible across the two stacks.
-
Response–Row Identity Across Promote — Why Responses Outlive Their Rows, and the `record_uuid`-on-Response Fix
Responses live in response-store. Rows live in row-store. When `record_set.promote` folds a parent set into a canonical, the **parent rows are deleted and new canonical rows are minted with fresh row_ids** — but the old responses survive untouched, still pointing at row_ids that no longer exist. Today there's no link from those orphaned responses back to the canonical row that inherited their accepted value, because the stable cross-derivation identity (`record_uuid`) lives only on rows, not on responses. The fix is small and additive: carry `record_uuid` on the ResponseRecord at create time, and the by-record view can resolve orphans by identity instead of by dead row_id.
-
Packs and Bundles — The Two-Tier Pattern for Entity-Profile Augmentation (and Beyond)
When augment-it needs to fan out across many sources to find verified profiles for an entity (LinkedIn + X + BlueSky + YouTube for everyone, then Candid + ProPublica + IRS 990 + Charity Navigator for a nonprofit), the right abstraction is not a single capability and not a single big prompt. It is two tiers. A **pack** is the atomic unit — one source, one microfrontend, one MCP-server microservice, one prompt-snippet, one extraction-schema, one render-config. A **bundle** is the orchestration unit — a named composition of packs with single-pass or two-pass execution, data carry-forward between passes, agent-driven pre-flight dedup against existing `helpful_links`, and a single chat verb that fires the whole thing. This blueprint codifies both.
-
Original and Enhanced Record Instances — the Record-Instance Model
augment-it does not mutate an uploaded record set, and it does not spawn a new derived set per prompt run. It keeps an immutable original import and a single mutable enhanced instance that accumulates every enrichment in a round. The user can promote an enhanced instance to be the source of the next round, so the model is generational — original → enhanced → promote → enhanced' — with an id-map that holds across every generation so the eventual write-back into the system of record always knows its target. Two classes of enhancement — lookup and judgment — flow through this model very differently.
-
Module Federation + Rsbuild — Dev Loop Gotchas
Operational reference for the five things that bite teams adopting Module Federation 2.0 on the rsbuild stack. None of them block adoption; all of them silently cost time if you don't see them coming. Cross-origin HMR, TypeScript across federation boundaries, the rspack-escape-hatch config shape, MF 1.0 vs 2.0 documentation drift, and the smaller-than-Vite plugin ecosystem. Reference for the augment-it rewrite (and anywhere else in the family that picks the same substrate).
-
Spec-Kit and Context-V Coexistence — How They Work Together in Augment-It
On 2026-05-18 augment-it adopted GitHub's Spec Kit (v0.8.11) alongside its existing context-v directory. This blueprint codifies how the two systems coexist: context-v holds the project's living memory (explorations, blueprints, reminders, journey docs, broader specs), and spec-kit drives implementation flow per feature (constitution → specify → plan → tasks → implement, with quality gates). The two are complementary, not competing. The 14 /speckit-* slash commands now available in Claude Code operate on .specify/, while context-v continues to be the source of truth for project memory.
-
Why Response Reviewer and Highlight Collector Exist — The Verbose-Prose-to-Tabular Bridge
The core impedance mismatch Augment-It solves: CRM data is tabular (rows × columns, succinct cells), while LLM and Deep-Research responses are verbose markdown prose (paragraphs, hedging, citations, expansion). Augment-It's purpose is to turn prose back into something that fits in a CRM cell. The response-reviewer and highlight-collector are the two pipeline stages where that bridge gets built — and they exist because no structured-output regime, however good, removes the need for a human-in-the-loop integrity check before tabular data gets written back into a system of record.
plans
-
First-Pass Corpus Quality Scan for reach-edu — a measured before/after baseline of the RAG corpus
Before wiring RAG/grounding over the reach-edu corpus, take a read-only baseline of what's actually there — 517 markdown files across 57 funder dirs (140 still in inbox). A fresh Claude Code session inventories, scores, and reports corpus quality so 'after' is comparable to 'before.' Diagnostic only — it changes nothing.
-
Canonical Entity Registry on SurrealDB Cloud — just write records
SurrealDB Cloud is up at main/main with three SCHEMALESS tables (persons, organizations, affiliations). Goal: write records from tonight's CSV+JSONL into those tables. SCHEMALESS stays. No field discipline up front.
-
Augmentation-state preservation and snapshot promotion — the operator's CSV is the system of record, the filesystem is the truth, and `/promote-snapshot` is the verb that walks the corpus and emits the next CSV with system columns appended so flow-switches don't erase the prior cycle's work
v8 has been at rest in `inputs/` since 2026-06-05 while 22 corpus files landed across two new funders, 12 inbox captures piled up, one chat verb shipped, and zero of that progress is visible in the spine the operator hands to themselves a week from now. The fix is one new verb: `/promote-snapshot` reads the latest CSV in `inputs/`, walks `clients/<client>/corpus/` indexing markdown files by their `record_id` frontmatter, and emits `<date>_<basename>_v<N+1>.csv` with system columns appended — `corpus_count`, `corpus_funder_slug`, `corpus_last_updated`, `corpus_by_pack`. No new infrastructure between the filesystem and the CSV. No register, no write-hooks, no boot-time backfill — just a stateless join at the moment the operator chooses to advance. The CSV in `inputs/` is the system of record (both spine columns and system columns); the filesystem is the truth that system columns are *derived from*. A re-export from Google Sheets (which won't carry system columns) doesn't lose anything — the next `/promote-snapshot` re-derives them from filesystem and emits them back into the CSV. Ships in two phases: a `corpus_count` chip on Records Surface so the 17/96 coverage gap becomes visible in the view (Phase A); the promotion verb itself (Phase B).
-
Download PDFs into Corpus Inbox — preserve the original binary alongside Jina-extracted markdown; wire both inbox vectors (UI and agent-chat) so the operator's PDF discoveries land as commit-able evidence, not just summarized text
Today's inbox flow Jina-fetches every URL and writes the extracted markdown to `clients/<client>/corpus/inbox/<date>_<slug>.md`. For HTML pages this is fine — the markdown IS the content. For PDFs it loses information that matters: the original PDF (with its tables, figures, signatures, page numbering, citation-able URL) is gone, and the operator is left with whatever text Jina could pull out of it. The DOL workforce-strategy PDF captured in the 2026-06-08 milestone ship is exactly this case — useful text, but the binary that needs to be cited downstream isn't preserved. This plan adds a binary-download primitive to `services/content-ingest`, threads PDF detection + filesystem persistence through `corpus.inbox.add`, extends the inbox frontmatter contract with a `binary_asset` block, surfaces 'PDF saved' affordances in both the agent-chat result bubble AND the Content Reader UI (which gets a small 'send to inbox instead' toggle as an interim inbox-UI surface before the dedicated microfrontend ships), and lays down a per-client `.gitattributes` git-lfs discipline so PDFs don't bloat the per-client repo's git history. PDFs first; the same scaffolding extends to docx/pptx/xlsx when those become operator pain points.
-
URL Auto-Detector and Clickable Rendering for List Fields — make socials, helpful_links, and official_updates_index_urls open-in-tab links instead of opaque JSON
Three of the most operationally important Augment-It fields are list-shaped — `socials`, `helpful_links`, `official_updates_index_urls`. Each is either a `string[]` or an `Array<{ url: string, ...metadata }>`. Today's field renderer (shipped earlier today in commit `2004770`) stringifies them as JSON, which is legible but not actionable — the operator can't click through to verify the link works, and the secondary metadata (display_name, confidence, label, note) clutters the visual scan. This plan adds a shape-detecting URL extractor (`formatFieldValue` becomes pluggable) and renders any field whose value is a URL — scalar string, array of strings, or array of objects with a `url` key — as a list of clickable `<a target="_blank" rel="noopener">` links, one per URL, with the auxiliary metadata dropped from the rendered view. The forcing function is the next bundle's hard dependency: it fires only against rows whose `socials`, `helpful_links`, and `official_updates_index_urls` already carry ≥1 URL, so the operator must be able to skim Record Collector and visually confirm 'yes there's a link here, and yes it's the right one' fast. Closes the audit side of [[OfficialPulse-URLs-Appear-as-Junk-in-Promoted-Versions]] — the curation/remove side stays as a sibling follow-up.
-
In-App Chat v0.0.1 for Augment-It — The Prompt-Drafting Triad as the Demo Affordance
Revised. The v0.0.1 demo arc is the gated-enhancement pattern made conversational: the user states a goal, the chat drafts a prompt (`prompts.draft`), refines it across one or two turns (`prompts.improve`), then explicitly binds it to records (`prompts.apply`) — a ScriptCapability with postconditions that actually checks whether the enrichment did what the prompt promised. Plus `records.list` to look at the result. Four capabilities total, all per-app; two adapter shapes exercised (TS handler, ScriptCapability). McpCapability and SkillCapability honestly deferred to v0.0.2 — no corpus exists yet for `corpus.search` to query, and no skill is wrapped yet. The blueprint's full Pattern 1 isn't proven by v0.0.1; the gated-enhancement triad is. That's the right trade for a client-meeting demo on fundraising-pipeline development.
-
Shell & Micro-Frontend UX Coherence — Refactor Plan
Sequence the eight locked decisions from the UX Coherence spec into five phases that land on `refactor/ux-streamlining`. Foundation first (Deck→Flow rename so every later phase speaks the right vocabulary), then mechanical polish that's independently shippable (peek labels, pack-selection helpers, off-mode hide), then the hierarchical Flow widget (bubble progress strip + icon-tooltip layout sub-options), then 'Augment This Set' with the record-set pre-selection unification, then harvest the cross-cutting principles into a living doc. Architectural open questions (enrichment-surface composition, Flow-widget default position) are spiked but not allowed to block the visible UX wins.
-
Augment-It Workspace — Walking Skeleton Plan
First buildable plan for the augment-it rewrite. Scaffold @augment-it/workspace (Svelte 5 singleton, per the pre-flight spec) plus a thin Workspace Service container (Node/Fastify) plus one domain microservice (row-store) plus a NATS message bus. Browser ↔ Workspace Service is WebSocket; Workspace Service ↔ domain services is NATS pub/sub. Two spreadsheets become the proof-of-life payload: load → row appears in workspace state → invoke a capability → round-trip the change back through SSE-equivalent broadcast. Supersedes the Transport decision in [[Walking-Skeleton-Pre-Flight-Decisions]]; the demo audience (a 26-product client) is the reason.
-
Impose the Three-Mode Theme System on augment-it
augment-it adopts the Lossless theme religion — the two-tier token system and the light/dark/vibrant three-mode contract from the Astro Knots blueprint — adapted for a non-Astro, non-Tailwind, federated multi-remote app. A shared packages/theme holds one theme.css (named tokens → semantic tokens, three mode blocks) and one SSR-safe mode-switcher; the shell renders a 3-mode toggle in its chrome; every hardcoded hex across the shell and the two remotes is replaced with a semantic var(). The structure is imposed in this work; the user then iterates the actual colours at the named-token tier without touching component code — which is the entire point of the two-tier system.
-
Prompt-Template-Manager — Walking Skeleton Plan
The second federated app for augment-it: a place to author prompt templates with {{column}} placeholders and run them per-row against a record set, producing a derived record set with the LLM's output as a new column. Two new microservices land — prompt-store (template CRUD, JSON-backed, same shape as row-store) and prompt-runner (the only container that calls the Anthropic API). The proof-of-life is the url-enrichment prompt: the pipeline-tracker CSV has no url column, so the first useful prompt is 'given the organisation name, find its URL' — and its output becomes the url column the next prompt needs.
-
Run-as-First-Class-Operation — Pair Pack-Runner with Prompt-Template-Manager, Make Runs Legible Across the Pipeline
Two coupled fixes that came out of the 2026-05-25 smoke against the foundation dataset. **First**, pair the Pack Runner remote with Prompt-Template-Manager in co-existence mode — they are the two ways to enrich a record set (custom LLM prompt vs source-bound pack), so they belong in one viewport with a draggable seam. **Second**, lift `Run` from a string id buried on each ResponseRecord into a first-class entity in response-store, so Response Reviewer (and Pack Runner, and every future surface) can see what batch of work produced the responses it's looking at — what prompt or packs fired, against which rows, with what aggregates, when. The pair is a one-line PAIRINGS entry. The Run lift is a small new service surface plus four UI touches; it pays for itself the moment a bundle wants live progress without inventing new infrastructure.