dsh-ledger-compact

Readme

# dsh-ledger-compact

English · [中文](README.zh.md)

**dsh-plugin** for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness). Large tool results are shaped **before** they reach the model. An optional local fold can replace DSH’s LLM `/compact`.

[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![dsh-plugin](https://img.shields.io/badge/dsh-plugin-111111)](https://github.com/topics/dsh-plugin)
[![test](https://github.com/telagod/dsh-ledger-compact/actions/workflows/test.yml/badge.svg)](https://github.com/telagod/dsh-ledger-compact/actions/workflows/test.yml)

---

## Why

Long agent turns dump huge `tool_result` blocks into the next request. DSH already has an excellent LLM compact (`@deepseek-ai/dsh-compaction-basic`), but it costs an extra model call and runs *after* pressure builds.

This plugin does two cheaper things first, and optionally a third:

1. **Ingress shaping** — freeze oversized *unsent* tool results into a head/tail excerpt on the idle pre-step. Only the step that just finished is touched: its first request is the one being prepared, so a rewrite never invalidates a provider-cached prefix.
2. **Mechanical fold** — `/fast-compact` and the input-bar bolt collapse older history into a short ledger card. No model call.
3. **Replace default compact** *(off)* — reuse the official engine’s `summarize` hook so `/compact`, auto-compaction, and overflow recovery also skip the LLM.

Keep `dsh-compaction-basic` mounted. This plugin does not replace the engine.

## Install

Requires **dsh ≥ 0.1.2-rc.1**; verified on **0.1.5-rc.1**. Both API renames in that range are handled: `session.events` → `session.eventAt(seq)` and the positional replace keys `start/end` → `startSeq/endSeq`.

```bash
dsh plugin --profile web add github:telagod/dsh-ledger-compact
```

Restart the web profile after install. The bundle patch inserts plugin id `dsh-ledger-compact`.

From a local checkout:

```bash
git clone https://github.com/telagod/dsh-ledger-compact.git
dsh plugin --profile web add ./dsh-ledger-compact
```

## What you get

| Layer | When | Model call | What changes |
| --- | --- | --- | --- |
| Ingress shaping | Idle `agent/pre-step` | No | Current unsent `tool_result` only |
| Bolt / `/fast-compact` | You click or type it | No | Older history → mechanical fold card |
| Replace default compact | `/compact`, auto, overflow | No | Same official transaction; local summarizer |
| Official `/compact` | Default DSH path | Yes | LLM `<compacted-summary>` checkpoint |

The input-bar **bolt** is a context-pressure meter. By default it needs a second click before folding (can be turned off). After a fold, the chat shows an expandable row in the same style as the official compact line; the open view is a slim left-axis timeline, not a model essay.

## Settings

Open **Settings → 快速压缩**. Changes write through to the Host document immediately. The page is tabbed:

- **入境** — the four shaping settings below.
- **折页** — fold confirmation, replace-default, and a live preview of the fold card.
- **诊断** — plugin version, the session-event accessor this DSH exposes, ingress counters and the last failure, and how many compaction engines are hooked (plus any engine that would silently fall back to a model call).

| Setting | Default | Meaning |
| --- | --- | --- |
| Ingress shaping (`enabled`) | on | Excerpt large unsent `tool_result` blocks before the next request |
| Allow dense PNG (`snapImages`) | off | Attach a bitmap only when the main model lists `image` in `inputModalities` **and** the image is cheaper than the original |
| Confirm before fold (`confirmFold`) | on | Bolt requires a second click |
| Replace default compact (`replaceDefault`) | **off** | `/compact`, automatic pressure compaction, and overflow recovery use the mechanical summarizer |
| Ingress threshold (`minSnapTokens`) | `3000` | Approximate tokens before an excerpt (or PNG) is considered. Range 200–200000 |
| PNG savings (`savingsRatio`) | `0.85` | Excerpt + estimated image tokens must be ≤ this fraction of the original |

Turn **Replace default compact** off at any time to restore DSH’s LLM checkpoint. `/fast-compact` stays mechanical either way.

## Commands

```
/fast-compact
/fast-compact status
```

There is no vision sidecar. `/fast-compact vision …` was removed.

`/compact` remains the official DSH command. With replace-default **off**, it still calls the model. With it **on**, the same command runs the mechanical fold through the existing engine.

`/fast-compact status` also reports health: ingress shaped/snapped/saved counters, the last ingress error when one is repeating, and any compaction engine that lacks a `summarize` hook (with replace-default on, such an engine would silently call the model).

## Ingress excerpt

Default is text only — no PNG, no LLM. Only results from the **immediately preceding step** of the current turn are candidates; everything already sent stays byte-identical, which is what keeps the provider prefix cache intact. Already-shaped placeholders are not rewritten. `skill` and `context` tool results are skipped. If `tokenMeter` is missing, ingress is skipped so the plugin never `replace`s without writing `compaction/prune`.

```
[Snapcompact: N tokens → excerpt]
first 16 lines
… (K lines elided; see image if attached. To inspect or edit exact bytes, re-read with offset/limit) …
last 8 lines
```

A PNG is attached only when all of these hold:

- “Allow dense PNG” is on
- the routed model’s `inputModalities` **explicitly includes `image`** (names like `gpt-4o` are never guessed)
- excerpt + estimated image tokens ≤ original × savings ratio (default 15% cheaper)
- the text actually drawn into the image must be **more expensive as text** than the image itself (default: image ≤ text × 0.85). Three thousand lines of three characters each lose money as a PNG, so only the excerpt ships

### Dense-image layout

The canvas is billed by area (512px tile bands — a full band and a blank one cost the same), so the layout has exactly one job: **fill the canvas that has already been paid for**.

- **The canvas cap is tile-aligned**: openai-family frames are `1024×2048`, not `1024×1540`. Same 1445 tokens, 93 text rows instead of 70 (+33%) — the old height crossed into a paid band for four pixels.
- **Short lines pack into columns.** The planner tries 1–24 columns and keeps the highest occupancy; a candidate is dropped as soon as more than 2% of its cells would wrap, so one-line-per-cell holds whenever it can, and only content wider than the whole canvas wraps inside a cell. `seq 1 3000` becomes a 20-column numeric grid: 1860 source lines for the price of 93.
- **Line ruler**: from 50 source lines up, the left gutter prints the source line number every 5 grid rows next to a faint rule. That makes the image addressable — a model that spots a bad row can `offset/limit` its way back to exact bytes instead of guessing.
- **Layered ink**: body `16`, column rules `205`, ruler `150`, paper `245`. Hierarchy costs no tokens; it only costs pixels, and the pixels are already paid for.
- The excerpt notice names the layout: `[Snapcompact: 3480 tokens → 1024x624 PNG ~434 tokens · 20 cols · line ruler]`, so the model never has to guess whether it is reading one column or twenty.

### Cost and fidelity (deepseek-flash)

A dense image is billed as **input tokens**, at the published DeepSeek rates (api-docs.deepseek.com, read 2026-09-10):

| | peak | off-peak |
| --- | --- | --- |
| input, cache hit /1M | $0.006 | $0.003 |
| input, cache miss /1M | $0.30 | $0.15 |
| output /1M | $1.20 | $0.60 |

Two things below are measured, not assumed:

- **Draw inside the request pixel budget.** The request pipeline resizes any frame above `640000` px before dispatch, and the provider both bills and reads the resized image. Measured on identical content: drawing `1024x2046` (sent as `566x1131`) answered 1–2 of 10 questions; drawing `1024x624` (no resize) answered 8. DeepSeek routes therefore use the `8on16-budget 1024x624` geometry (39 rows).
- **The image-token curve is measured too**: `≈ clamp(70 + 5.7e-4·px, 213, 1043)` on the resized dimensions. The published "one image caps at 384 tokens" calculator does not match live `usage`; this fit is within ~3% (est 434 vs measured 446, the gap being the prompt text).
- A full frame costs about **$0.00013**; the same content as raw text costs $0.001–$0.014 — **10–100x more**.
- Fidelity has a ceiling: even unresized, roughly 20% of **exact values** (coordinates, IPs, status codes) are misread. Exact bytes therefore stay in the text excerpt and the re-readable source; the image carries bulk and structure. That is exactly why the `re-read with offset/limit` contract exists.

`bench/report.md` holds the raw run (fixtures × layout variants × checkable questions, answers and `usage` cached); `bench/FINDINGS.md` holds the method, the noise floor and a verdict per axis. In short:

- **Draw inside the request pixel budget**: identical content answered 1–2 of 10 when the pipeline resized it, 8 of 10 drawn natively, for the same bill.
- **Packing** carries ~20x more lines per frame at equal accuracy.
- **A source line number on every row** lifted exact-value accuracy from 73% to 87% at no token or capacity cost — it aims straight at the dominant error class, reading the wrong line.
- **Colour was tried and rejected**: four palettes, paired 3-repeat run over 60 samples each — grayscale 46/60, colour 44/60, value accuracy identical. The encoder (`encodePngPalette`) and per-cell/per-digit ink stay in the library, off by default.
- **Prompt legend, roomier cells, single column as default**: measured, none earned its cost.
- **Ceiling**: about a quarter of exact-value questions are still misread in the best configuration. The image is an index, not a notary — exact bytes travel as text, in the fold card, or through `offset/limit` re-reads.

## Fold card

Mechanical, no model. Prior fold text is never nested, but it is not lost either: files, intents and errors recovered from an earlier `[Snapcompact]` card are carried forward into the new one (files only fill capacity the new span left over).

```
[Snapcompact] Fold ~N tok. Exact file bytes are not stored — re-read with offset/limit if a detail matters.
FILES
- [edit] path
- [read] path
INTENTS
- recent user goals
TOOLS
- read 4
COMMANDS
- …
ERRORS
- …
EXCERPT
[user] …
[tool] …
```

This is cheaper and more predictable than an LLM checkpoint. It also keeps less prose: paths, intents, tool counts, and a short excerpt — not a narrative of *why*.

## Working with DSH compact

Do **not** unmount `@deepseek-ai/dsh-compaction-basic`.

- Pressure, range selection, shrink checks, and durable `compaction/*` events stay on the official engine.
- This plugin only wraps `summarize()`.
- `/fast-compact` marks the current agent for one `compactNow()` so other sessions keep the LLM path.
- Replace-default is a live setting: every hooked engine uses the mechanical summarizer until you turn it off (or unload the plugin).

On web, compaction lives in the preset isolate. Host code reads it with `agentPresets.serviceFor(agent, "compaction")`, not `inject: ['compaction']`.

## Develop

```bash
node --test lib/ledger.test.js

# per-fixture layout economics: text vs image tokens, lines carried, canvas occupancy.
# Pure arithmetic, no model calls.
node bench/layout-bench.mjs

# fidelity + real billing: fixtures x layout variants x checkable questions, cached in bench/.cache
node bench/fidelity-bench.mjs

# one test binds a real dsh-session; point it at the installed package to run it
DSH_SESSION_MODULE="$DSH/node_modules/@deepseek-ai/dsh-session/lib/index.js" \
DSH_SCHEMASTER_MODULE="$DSH/node_modules/@deepseek-ai/schemastery/lib/index.mjs" \
  node --test lib/ledger.test.js
```

| File | Role |
| --- | --- |
| `lib/ctx.js` | Optional service lookup + version-tolerant session event access |
| `lib/schema-envelope.js` | Canonical Schemastery `{uid, refs}` settings envelope |
| `lib/ingress.js` | Previous-step `tool_result` shaping (cache-safe by construction) |
| `lib/vision.js` | Dense PNG gated by `inputModalities` + settings |
| `lib/excerpt.js` / `lib/fold.js` / `lib/snapfont.js` | Excerpt, fold card, bitmap and grid raster |
| `lib/layout.js` | Dense-image layout: column packing, in-cell wrap, line ruler, occupancy |
| `lib/hook.js` | Mechanical `summarize` hook |
| `lib/resolve.js` | Find isolated compaction engines |
| `lib/index.js` | Command, settings, pre-step |
| `lib/client.js` | Bolt, fold row, settings page |

## License

MIT. Bitmap fonts under `lib/fonts/` are documented in [`lib/fonts/README.md`](lib/fonts/README.md).