← Corpus / perplexed / issue

perplexed/issues/getting-claude-to-respond-with-research

Path
issues/Getting-Claude-to-Respond-With-Research.md
  1. Claude decided not to actually use the tool. The model decides when to search. With tool_choice: { type: “auto” } (default — what I’m using) and Opus 4.7’s training cutoff at January 2026, it already knows about DevDay 2025, AgentKit, Klarna’s two-thirds stat, Salesforce Agentforce, etc. Its narrative “I’ll research this… Let me dig into…” is polite preamble — Claude is performative there. If it judged it could answer from training, it skipped the actual web_search tool call entirely. No tool call → no web_search_tool_result blocks → no citations. The fix is one of:
  • tool_choice: { type: “any” } to force at least one tool call (since web_search is the only tool, that forces a search)
  • Or a system prompt: “ALWAYS use the web_search tool to verify and ground every factual claim. Do not answer from training knowledge.”
  1. Claude DID use the tool but only emitted web_search_tool_result blocks, not per-claim web_search_result_location citations on text blocks. This happens when the model summarizes findings narratively rather than attaching formal citations to specific sentences. My original extractWebCitations only walked text-block citations and missed the result blocks. The fix I shipped this session adds a fallback that also reads URLs+titles from web_search_tool_result blocks. After that fix the only way to get zero citations is if Claude didn’t search at all.

Diagnostic path that’ll tell us which one is happening — I added console logging this session that prints block types: […]; text blocks with citations: N; extracted: N. When you next run Ask Claude with Obsidian’s dev console open (Cmd+Opt+I), the log line tells us:

  • If block types contains “web_search_tool_result” → Claude searched. Citations should now extract via the fallback.
  • If block types has no web_search_tool_result → Claude didn’t search. We need tool_choice: any or a system prompt to force it.

No file edits this turn per your direction. When you’ve run a test and pasted me the [ClaudeService] log line, I’ll know which fix to make. If you want me to just ship the tool_choice: any + system-prompt fix preemptively (it’d handle both failure modes), say the word.

But your question opens the door wider — we could parse everything else in the response object too:

Block type: web_search_tool_result What’s in it: URL + title + encrypted_content + page_age per result Currently parsed?: ✓ URL + title only Could be: + page_age for date-aware refdefs ──────────────────────────────────────── Block type: server_tool_use (web_search) What’s in it: The actual query string Claude searched for Currently parsed?: ✗ Could be: Group references under the query that produced them ──────────────────────────────────────── Block type: code_execution_tool_result What’s in it: Output of dynamic-filter code — likely contains the filtered snippets Claude actually

used

Currently parsed?: ✗ Could be: Pull cited_text-equivalents from here ──────────────────────────────────────── Block type: text blocks (no citations attached) What’s in it: Just the narrative Currently parsed?: ✓ for body Could be: —

Going deeper means we don’t need Claude to attach per-claim citations — we synthesize what we need from the surrounding blocks. Trade-off: it’s heuristic (we can’t know exactly which sentence came from which URL without per-claim attachment), but a Citations section enriched with search query + URL+title + snippet from filter output is genuinely useful research provenance, not just a flat URL list.

But before I go bigger — the gating question is still whether addCitations is actually writing the section to the file. The log proves we extract 60 entries. If they’re not reaching the file, expanding the extractor doesn’t help. The simplest way to find out: I add one more log line right after the editor.replaceRange call in addCitations that prints [ClaudeService] Citations section written: N entries, lastLine=X. Then on next run, the log tells us definitively whether the write happened.

Proposed two-step:

  1. Now (small): instrument addCitations with a post-write log. You re-run, paste the log line. We confirm the write or catch the silent failure.
  2. Then (bigger): extend the parser to pull search queries from server_tool_use blocks and snippets from code_execution_tool_result blocks. Citations section becomes:

Citations

Search: “agent builder market 2025 startups”

[1]: Title. > snippet from filter output… [2]: Title. > snippet…

Search: “OpenAI AgentKit launch DevDay”

[3]: Title. > snippet…

The data IS in the response object. There are two distinct citation surfaces in Anthropic’s web search responses:

┌────────────────┬────────────────────────────┬───────────────────────────────────┬────────────┐ │ │ │ │ Status in │ │ Surface │ What it contains │ Where it lives │ your last │ │ │ │ │ run │ ├────────────────┼────────────────────────────┼───────────────────────────────────┼────────────┤ │ │ URL + title + │ │ ✓ Present │ │ Search-result │ encrypted_content + │ web_search_tool_result.content[] │ — 60 of │ │ blocks │ page_age for every page │ array │ them │ │ │ Claude searched │ │ │ ├────────────────┼────────────────────────────┼───────────────────────────────────┼────────────┤ │ Per-claim │ cited_text quote + URL + │ │ ✗ Zero │ │ citations │ title, attached to a │ TextBlock.citations[] array │ attached │ │ │ specific text block │ │ │ └────────────────┴────────────────────────────┴───────────────────────────────────┴────────────┘

We have surface #1 (the URLs and titles of every searched page). We don’t have surface #2 (the mapping from “this specific sentence” to “this specific URL”). The dynamic-filtering pass on web_search_20260209 is doing its post-processing in a code-execution sandbox and the per-claim attachment isn’t surviving the round-trip — that’s the apparent root cause.

What that means practically:

  • We CAN populate a Citations section at the bottom of the note with all 60 URLs+titles. My fallback already does this. The references are real and verifiable.
  • We CAN’T put inline [N] markers next to specific claims, because the response doesn’t tell us which sentence came from which URL. Building that mapping ourselves would be heuristic guesswork (string-matching titles against text) — fragile and easy to get wrong.

So the real question is: did your test file get a ### Citations section appended at the very end with 60 entries? That tells us where the gap actually is:

  • Yes, it has the section → we have references-but-no-inline-markers. The fix for inline markers is to switch to web_search_20250305 (the older tool version) which reliably attaches per-claim citations.
  • No, the section never appears → my addCitations writer is silently failing for a reason I haven’t pinned yet, and I need to add another log line that fires after the editor write.

Could you scroll to the bottom of the file you tested on (looks like OpenClaw.md from the log) and tell me which of those two it is? If it’s “no section” I’ll instrument addCitations to log success/failure of the write itself and we’ll catch it on the next run.

┌─────────────────────────────────┬──────────────────────────────────┬────────────────────────┐ │ Step │ Who │ What │ ├─────────────────────────────────┼──────────────────────────────────┼────────────────────────┤ │ 1. Search the web │ Claude │ Issued 6+ search │ │ │ │ queries, got results │ ├─────────────────────────────────┼──────────────────────────────────┼────────────────────────┤ │ │ Claude (via │ Post-processed results │ │ 2. Filter results in code │ web_search_20260209’s dynamic │ in its sandbox │ │ │ filter) │ │ ├─────────────────────────────────┼──────────────────────────────────┼────────────────────────┤ │ 3. Emit web_search_tool_result │ │ 125 entries in the │ │ blocks with URL+title for every │ Claude │ response object │ │ searched page │ │ │ ├─────────────────────────────────┼──────────────────────────────────┼────────────────────────┤ │ 4. Emit │ │ │ │ web_search_result_location │ Claude │ 0 entries — Claude │ │ per-claim citations on text │ │ didn’t do this │ │ blocks │ │ │ ├─────────────────────────────────┼──────────────────────────────────┼────────────────────────┤ │ 5. Parse the response object’s │ │ The 125 numbered │ │ web_search_tool_result blocks │ My fallback code │ refdefs you see in AI │ │ and write a ### Citations │ (claudeService.ts:addCitations) │ Copilots.md:135–391 │ │ section │ │ │ └─────────────────────────────────┴──────────────────────────────────┴────────────────────────┘

So Claude provided the raw URL data; my code authored the section. The trailing-period format [N]: Title. is my code’s signature; if Claude had attached per-claim citations directly, you’d also see (a) inline [N] markers within the prose and (b) > cited_text blockquotes on each refdef. Neither is present, confirming step 4 didn’t happen.

About the blog you linked. It’s the launch announcement for the Citations API as a document-grounding feature — quote: “user-provided source documents (PDF documents and plain text files).” It predates web-search-citation integration entirely. The blog’s primary use case is “give Claude a PDF, get answers with cited passages from that PDF.” Web search citations are a later addition with different mechanics — and with web_search_20260209 + dynamic filtering, per-claim attachment to text blocks isn’t surviving the post-processing round-trip. That’s the real gap.