dsh-hyper-tools

Readme

# dsh-hyper-tools

Keeps Charm Hyper model requests under the gateway's strict 10 MiB request-body cap.

Charm Hyper's HTTP gateway (Google Frontend in front of a Go backend) rejects any request body above 10 MiB with:

```json
{ "error": { "message": "invalid request body", "type": "invalid_request_error", "code": null } }
```

Long conversations accumulate images, and every image is inlined as a base64 `image_url` part. Once the serialized body crosses the cap, the whole request fails before inference — even though the images are ordinary attachments. This plugin registers one `llm/stream` middleware for the configured Charm Hyper provider routes:

1. **Before dispatch** it projects durable history so the accumulated base64 image payload fits `maxRequestImageBytes` (default 6 MiB). The **oldest** images become the harness-standard deterministic placeholders (`[image omitted to fit request image limits; …]`) and the **newest images stay**. A request that already fits is dispatched untouched through `next()` — no extra provider attempt.
2. **When the gateway still rejects the body** (the projection under-counted non-image overhead) it swallows that terminal `invalid request body` failure and re-dispatches once with the image budget multiplied by `recoveryBudgetFactor` (default `0.5`), which replaces more of the oldest images. If a tighter projection cannot remove anything further, the original failure is surfaced unchanged.

Accounting and placeholders come from the harness itself (`offloadRequestImagesWithPolicy`, `offloadedImageText`): base64 length of every image occurrence — nested tool-result images included — and the same deterministic placeholder text shipped adapters use.

## Verified behavior

Checked live against `https://hyper.charm.land/v1` (`deepseek-v4.1-flash`, six 1024×683 PNG screenshots):

| Request | Serialized body | Result |
| --- | ---: | --- |
| Accumulated images, unprojected | 16,794,575 bytes | `400 invalid request body` |
| Same history projected by this plugin | 5,599,267 bytes (4 oldest images replaced) | `200 OK` |

The gateway cap was measured at 10 MiB (10,485,760 bytes): a 10,400,000-byte body passes the gateway, a 10,490,000-byte body is rejected. `scripts/probe-hyper.py` reproduces the probe; `tests/live.test.ts` reproduces the table above.

## Install

```bash
dsh plugin --profile <profile> add dsh-hyper-tools
```

From a checkout or tarball:

```bash
dsh plugin --profile <profile> add ./dsh-hyper-tools
# or
npm pack && dsh plugin --profile <profile> add ./dsh-hyper-tools-0.1.0.tgz
```

From git (the built `lib/` is committed, so pnpm needs no build permission):

```bash
dsh plugin --profile <profile> add github:samuelrubiodev/dsh-hyper-tools#main
```

The bundle patch ([`cordis.patch.yml`](./cordis.patch.yml)) inserts one row (`hyper-tools`).

## Configuration

Every field is optional; the row `config` carries them.

| Field | Default | Meaning |
| --- | --- | --- |
| `providers` | `['charm-hyper', 'hyper']` | Provider routes to guard; matching is case-insensitive. |
| `maxRequestImageBytes` | `6291456` (6 MiB) | Aggregate base64 image payload one request may carry. |
| `maxImagesPerRequest` | absent | Optional bound on images per request. |
| `recovery` | `true` | Retry once when the gateway rejects the body. |
| `recoveryBudgetFactor` | `0.5` | Image-budget multiplier for the recovery retry. |
| `byteQuantum` | `1` | Whole-step removal quantum for byte overflow; `1` keeps the newest retainable images. |
| `countQuantum` | `1` | Whole-step removal quantum for count overflow. |

```yaml
- id: hyper-tools
  name: 'dsh-hyper-tools'
  config:
    providers: ['charm-hyper']
    maxRequestImageBytes: 6291456
```

## Notes and limits

- The budget counts **image payload only** — the platform's documented unit for request-image bounds. Text, tool schemas, and JSON structure ride outside it; the defaults leave several MiB of headroom under the 10 MiB cap, and the recovery retry covers a misestimate.
- The projection estimates each occurrence from durable attachment metadata (`attachment.bytes`); any provider-side re-encode only makes the actual payload smaller, so the estimate errs toward replacing slightly early.
- Only the routes listed in `providers` are touched; every other provider request passes through untouched.
- The plugin is adapter-agnostic: it only keys on `GenerateOptions.provider`, so it guards Charm Hyper whether it is served by `@deepseek-ai/dsh-llm-pi-ai`, a custom gateway adapter, or anything else.

## Development

```bash
npm ci
npm run typecheck
npm test                 # unit + integration against the real cordis + @deepseek-ai/dsh-llm runtime
npm run build
HYPER_API_KEY=... npx vitest run tests/live.test.ts   # live check against Charm Hyper
```

The compiled `lib/` is committed so `github:` installs need no build step; `npm pack` / `npm publish` refresh it through `prepack`.

Compatibility: `@deepseek-ai/dsh-llm ^0.1.5-rc.1`, `@deepseek-ai/cordis ^4.0.1`.

## License

[MIT](./LICENSE)