@dsh-external/dsh-tool-failure-journal

Readme

<div align="center">

# Failure Journal

**The same error, five times, is one row — written to disk before it scrolls away.**

A DeepSeek Harness host plugin that appends every tool call that exits abnormally
to a durable JSONL journal, folds near-identical failures into signatures with
counts, and exposes the whole thing to the agent as one tool: `failure_journal`.

[![License: MIT](https://img.shields.io/badge/license-MIT-3DA639.svg)](LICENSE)
[![DeepSeek Harness plugin](https://img.shields.io/badge/DeepSeek%20Harness-tool%20plugin-4D6BFE.svg)](#install)
[![version](https://img.shields.io/github/package-json/v/catsenior507/dsh-tool-failure-journal?color=4D6BFE)](package.json)
[![node](https://img.shields.io/badge/node-%3E%3D20-3DA639.svg)](package.json)
[![stars](https://img.shields.io/github/stars/catsenior507/dsh-tool-failure-journal?color=4D6BFE)](https://github.com/catsenior507/dsh-tool-failure-journal/stargazers)

[English](README.md) · [简体中文](README.zh.md)

</div>

---

## The problem

A coding agent does not only fail. It fails the *same way* over and over, and the
evidence leaves the context window before anyone can see the pattern: the tool
result is folded, compacted, or simply buried under the next four attempts.

The harness keeps a session log — but the session log **is** the model's context.
It is rewritten, folded, and pruned. It cannot be the place a failure history
lives.

So this plugin keeps a second record, outside the context window, in a format that
outlives the process.

## What it captures

One listener on `tools/result`, which is the harness's own final notification for
a call — emitted after pre-policy, guards, the body, post-policy, and output
validation. That single hook covers:

- a tool that threw
- a tool whose return value violated its declared output schema
- an unknown tool name
- a call denied before dispatch
- a call cancelled by the caller

**One hook instead of four**, and no polling.

Each record carries its own context, because a record has to stay useful after the
session it came from is gone:

```json
{"v":1,"at":"2026-09-11T00:31:07.412Z","sessionId":"session-c9a2…","callId":"call_00_…",
 "turn":12,"step":3,"tool":"pwsh","aborted":false,"errorCode":"COMMAND_NOT_FOUND",
 "message":"'lake' is not recognized as an error…","messageHead":"…","argsChars":41,
 "args":"{\"command\":\"lake build\"}","content":"…","signature":"9f2c1ab73e04",
 "tag":"failure","recurrence":3,"recurring":true,"firstSeenAt":"2026-09-11T00:28:51.003Z",
 "remediation":["The binary is not on PATH for this shell; …"]}
```

## What makes it more than a log

**Signatures.** Every failure is hashed to a stable id built from the tool, the
error code, and a *normalized* message — paths, timestamps, UUIDs, and long
numbers replaced by placeholders. Forty near-identical failures become one row
with a count, which is the unit a reader can act on.

**`regression: true`.** A tool that already *succeeded* earlier in the session and
fails afterwards is marked. That is the strongest signal in the journal: something
that worked has stopped working, and it points at whatever changed in between.

**`aborted` is not `failure`.** A withdrawn call is tagged separately and never
counted as a defect. Counting cancellations as failures would poison the
recurrence numbers that make the journal worth reading.

**Shipped remediation.** For the failure classes whose fix is unambiguous —
`EDIT_NO_MATCH`, `EDIT_NOT_UNIQUE`, `COMMAND_NOT_FOUND`, `TIMEOUT`,
`PERMISSION_DENIED`, `SYNTAX`, `BAD_ARGS` — the record carries the cause and the
next action that is actually *different* from the one that just failed.

## The tool

| Action | What it answers |
| --- | --- |
| `stats` | What keeps breaking, folded by signature. **Start here.** |
| `list` | What failed most recently, newest first |
| `show` | Every occurrence of one signature, by prefix |
| `sessions` | Which journal files exist on disk, with sizes |
| `clear` | Archive the live page and start fresh |
| `selftest` | Prove the observer is attached and the directory is writable |

`stats` is the one that changes behaviour. "The same error five times" printed as
one row with `count=5, recurring=true` is a different instruction to a reader than
five separate stack traces.

<a id="install"></a>
## Install

The plugin is installed as a package into a dsh **profile**, the same way every
other dsh plugin is. `dsh plugin` forwards to `pnpm` inside the profile directory,
so any spec pnpm accepts works.

```bash
# from GitHub (the published form)
dsh plugin --profile web add github:catsenior507/dsh-tool-failure-journal

# a local checkout, while developing
dsh plugin --profile web add /absolute/path/to/dsh-tool-failure-journal
```

`web` is the shipped GUI profile; substitute `headless`, `sdk`, `acp`, or your own
profile name. On Windows, use forward slashes in a path.

Then **restart the host** so the profile recomposes, and confirm with:

```
failure_journal action=selftest
```

`selftest` reports whether the observer is attached and writes a probe file into
the journal directory to prove writability.

### What install does *not* do

- **No build step.** The published JavaScript *is* the source — there is no
  `dist`, no bundler, and no `prepare` script, so nothing runs on install.
- **No dependencies.** `dependencies` and `peerDependencies` are both empty; the
  plugin needs only the harness it is loaded into. Cordis is provided by the host
  at runtime.
- **No native code, no compiler, no network at run time.**

Node.js 20 or newer, because the harness itself requires it.

## Where it writes

`<DSH_HOME>/failure-journal/sessions/<sessionId>.jsonl`, one file per session,
rotated by size (`maxBytes`, default 4 MiB, `maxRotated` generations kept).

Rotated generations are **kept, not deleted**: a session that loops on one error
fifty times is the case worth reading afterwards, and truncating the file at the
moment it becomes interesting would defeat the point.

## Configure

`dsh plugin add` already inserted the plugin row. To change the defaults, edit
that row's `config` in the profile's `cordis.patch.yml`:

```yaml
- insert:
    - id: tool-failure-journal
      name: '@dsh-external/dsh-tool-failure-journal'
      config:
        maxBytes: 4194304
        clusterThreshold: 3
        excludeTools: ['todo_write']
        recordSuccesses: false
        exposeTool: true
```

`recordSuccesses: true` also journals successful calls — useful for "what did the
command actually do". Note that success *tracking* for the `regression` signal
runs regardless; the flag only controls what reaches the disk.

## Design constraints worth stating

- **The observer runs on every tool call in the process**, so it must never throw
  and must never block. Every derived field reads through a total accessor, and
  every write is a synchronous one-line append — microseconds, and it survives the
  process dying mid-turn. An async queue would lose exactly the records this
  plugin exists to keep.
- **A store failure is counted, not hidden.** `selftest` reports records written,
  dropped, and rotated, so a journal that stopped working says so.
- **A tool call is never failed by this plugin.** Writing happens in a listener
  the harness contains; the journal cannot break the tool it observes.

## Development

```bash
npm test        # 31 tests, including a real cordis mount and a real event
```

`npm test` needs `@deepseek-ai/cordis` resolvable, which comes with a dsh
installation. Running from a checkout rather than an installed package, point
`node_modules` at the profile's copy — on Windows a directory junction works.

The mount test registers a fake `tools` service on a real cordis `Context`, emits
the genuine `tools/result` event, and reads the journal file back through the
tool — because "`inject` was declared wrong so the observer never attached" is
exactly the failure a hand-rolled fake context cannot catch.

| File | Role |
| --- | --- |
| `lib/index.js` | Cordis plugin: resolve config, attach, register |
| `lib/observer.js` | The `tools/result` and `session/event` listeners |
| `lib/store.js` | JSONL storage, rotation, archiving |
| `lib/signature.js` | Failure identity, normalization, remediation catalog |
| `lib/tool.js` | The `failure_journal` tool |

## License

MIT