dsh-typesafe-ask

Readme

# dsh-typesafe-ask

English | [中文](README.zh.md)

Give your DeepSeek Harness agent a **decision oracle**: `typesafe_ask` calls
[TypeSafe](https://typesafe.ai) **Jev**, a System One model that evaluates a state against
typed questions and returns structured answers — a choice from a list, a score on a rubric,
or a probability that a statement is true — with calibrated probabilities attached.

Use it when the agent needs a judgment rather than a sentence: route a request to the right
handler, classify a ticket, score how urgent or how frustrated something is, rank candidates,
or gate an action behind a confidence threshold.

## What you get

| Surface | What it does |
| --- | --- |
| Tool `typesafe_ask` | Sends `state` + a map of typed questions to `POST /v1/systemone` and returns the answers in the conversation |
| Settings section | **Settings → TypeSafe / Jev** — paste your API key once; it is stored through the harness credential store, never in a profile config file |

## Install

From the plugin market: **Settings → Plugin Market → search `typesafe-ask` → Install**.
Or by hand:

```sh
dsh plugin --profile <your-profile> add dsh-typesafe-ask
```

Then restart or refresh the harness, open **Settings → TypeSafe / Jev**, and paste a key from
[console.typesafe.ai/keys](https://console.typesafe.ai/keys).

## Use

Once the key is set, just ask the agent in natural language ("which team should own this
ticket?") or let it call the tool itself. The tool takes:

| Parameter | Type | Required | Meaning |
| --- | --- | --- | --- |
| `state` | string \| object \| array | yes | What to evaluate: the text, record, or application state |
| `questions` | object | yes | Map of question id → typed question (see below) |
| `model` | string | no | Defaults to `jev-latest` |

Three question types:

```jsonc
{
  "department": {                     // choice — pick one option
    "type": "choice",
    "instructions": "Which team should handle this",
    "criteria": {
      "billing": "Payment or subscription issues",
      "technical": "Bugs or integration problems",
      "sales": "Pricing or account questions"
    }
  },
  "frustration": {                    // score — position on an ordered rubric
    "type": "score",
    "instructions": "How frustrated the customer appears",
    "criteria": ["Calm, just stating facts", "Frustrated but civil", "Very angry, strong language"]
  },
  "is_urgent": {                      // noul — probability the statement is true
    "type": "noul",
    "instructions": "The message conveys urgency or time-sensitivity"
  }
}
```

Answers come back keyed by your question ids:

```
department: choice=technical (confidence=0.78) [technical=0.85, billing=0.15, sales=0]
frustration: score=1=Frustrated but civil (confidence=1)
is_urgent: noul=0.99
```

## Designing good questions

- **Ask independent questions together.** Every question in one call is evaluated in
  parallel against the same state and cannot see the other answers. Send a second call only
  when an earlier answer is needed to build new state or new options.
- **One narrow, coherent judgment per question.** Split independent dimensions; do not
  destroy the relationship being judged. Question ids are for your code and are never sent
  to the model, so the full meaning must live in `instructions`.
- **`noul` has no separate confidence.** If several labels can hold at once, ask one `noul`
  per label instead of forcing a `choice`.
- **`score` levels must stand on their own** as concrete situations, not "low/medium/high".
  The returned `score` is a probability-weighted position and may be fractional.
- **Low confidence means the options overlap**, not that the model is broken. For `choice`,
  go back and make `criteria` mutually exclusive rather than lowering a threshold.
- **Thresholds belong to your data.** Calibrate them on real cases; `noul ≈ 0.5` means the
  yes/no split is even, not "medium intensity".

## Configuration

Defaults live in the plugin (`apiKeyRef` → `TYPESAFE_API_KEY`, `model` → `jev-latest`,
endpoint and timeout as documented above), so the plugin row itself is a bare insert — which
is what lets the market hot-mount it without a restart. To change a default, target the row
from a later patch layer, i.e. your profile's own `cordis.patch.yml`:

```yaml
- id: dsh-typesafe-ask
  config:
    apiKeyRef: MY_TYPESAFE_KEY                           # credential reference name
    model: jev-latest
    baseUrl: https://api.typesafe.ai/v1/systemone
    timeoutMs: 60000
```

## Boundaries

- Jev does **not** generate text, write code, stream, or call tools. It cannot replace the
  chat model behind your agent — TypeSafe says so explicitly in
  [Jev with coding agents](https://docs.typesafe.ai/introduction/coding-agents.md).
- The key is a server-side secret: it stays in the credential store and is only read on the
  host when a tool call is made.
- Requires network access to `api.typesafe.ai`.

## Development

The plugin is plain ESM with no build step — what is in `src/` is what ships.

```sh
# make the dsh packages resolvable for the offline tests (symlink, not installed)
ln -s "$DSH_NODE_MODULES" node_modules   # e.g. the dsh dependency's node_modules

npm test          # host + client + live client-module checks
```

- `test/host.test.mjs` — mounts the plugin on a stub context, registers the tool through the
  **real** `defineTool`, runs a live `/v1/systemone` call, and covers the config override,
  the missing-key path, invalid questions, and a deployment with no credential store.
- `test/client.test.mjs` — loads `src/client.js` through the same
  `window.__ModuleLoader__.load({ id, factory })` contract the dsh web shell uses, asserts the
  bundle requires only baseline modules, mounts it on a stub context, checks the
  `settings.section` registration and the `remote.credentials` call shapes, and renders the
  page with real React.
- `test/smoke.mjs` — the API client alone, against a real key
  (`$TYPESAFE_API_KEY`, or a path as the first argument).

Test the whole loop against a local dsh install:

```sh
dsh plugin --profile <profile> add /absolute/path/to/dsh-typesafe-ask
# restart the harness: a new bundle row is composed at boot
```

## License

MIT