Every Step Writes Its Output To File
A run's destination folder is decided before the first step, and every step writes its result there as it finishes. Nothing lives only in memory.
- Path
- reminders/Every-Step-Writes-Its-Output-To-File.md
- Authors
- Michael Staton
- Augmented with
- Claude Code on Claude Opus 5 (1M context)
- Tags
- Orchestrator · Artifacts · Versioning · Resumability · Run-Discipline · Reminders
Every Step Writes Its Output To File
The rule
- Every run has a destination outputs folder, resolved before the first step
does any work.
output/<Company>-v0.0.x/for a legacy run,io/<firm>/deals/<Deal>/outputs/<Deal>-v0.0.x/for a firm-scoped one. The version manager picks it; nothing else invents a path. - Every step writes its own output to that folder the moment it completes. Not at the end of the phase, not when the graph finishes. When the step is done, the file exists.
- Nothing of value lives only in memory or only in
state. State is how steps talk to each other. It is not where results are kept. - Never write run output to a scratch, temp, or session directory. Not for a test, not for a trial, not “just to look at it first.” If it is worth producing it is worth producing where it belongs.
Why
Long runs get killed. By the operator, by a timeout, by an API failure in step nine, by a laptop lid. That is normal and must be survivable.
A run that writes only at the end converts every interruption into total loss. Worse, it converts a recoverable interruption into total loss: the expensive work was done, the answers existed in memory, and they evaporated because nobody had written them down.
The concrete failure this came from. analyze_dataroom on the ProfileHealth
dataroom:
line 81 extraction_results = _run_extractors(...) # 8 extractors, ~30 min
# competitive, cap table,
# financial, traction, team…
line 140 save_dataroom_analysis_artifacts(...) # the first and only write
The run was stopped during the legal pass at line 81. Scanning had completed. Classification of 158 documents had completed — a 194-second model pass. Five extractors had completed and found real data, including a cap table with fifteen shareholders and seven SAFEs. None of it reached disk. The output directory contained one empty folder.
Everything above had to be paid for twice.
What this looks like in practice
- An extractor finishes → its
.jsonand.mdare written before the next extractor starts.0-dataroom-inventoryshould exist while1-competitiveis still running. - A section is drafted →
2-sections/03-market-context.mdis on disk before the writer moves to section four. (The section-by-section writer already does this; it is the model to copy, not the exception.) - A step that produces nothing still records that it ran and found nothing. “No cap table data extracted” is a result, and a rerun should not have to rediscover it.
- Killing a run at any moment leaves a directory that says how far it got.
Resumability is the point, not a bonus
Because each step’s output is a file, a rerun can look at the folder and skip
what is already there. That is what --resume means, and it only works if the
files exist. A run that keeps its results in memory cannot be resumed even in
principle — there is nothing to resume from.
This is also what the version directory is for. <Deal>-v0.0.1 is not just a
label on a finished memo; it is the workspace the run fills in as it goes, and
the reason a killed run can be restarted rather than rerun.
When a step legitimately holds state
Passing a value between two adjacent steps in memory is fine — that is what
MemoState is for. The rule is about results, not about plumbing. The test:
if the process died right now, would anything of value be lost? If yes, it
should already have been written.
See also
context-v/reminders/Round-Closing-Timeline-Nuances.md—anomalies.json, which is written incidentally during a run and follows the same disciplinesrc/versioning.py— the version manager that resolves the destination foldersrc/artifacts.py— the artifact writers each step should be calling