dsh-llm-sampling

Readme

# dsh-llm-sampling

English | [中文](README.zh.md)

Installable DeepSeek Harness bundle that enforces sampling policy for exact provider/model routes through the `agent/request` waterfall. The Harness core owns the provider-neutral request fields and durable request header; adapters own wire translation. This plugin owns deployment policy only.

The plugin is dormant until `llm-sampling.providers` names a route. A configured model's `default` profile replaces all sampling values on every request. When `reasoningEffort` is explicitly `off`, `off` overlays that complete profile. Unconfigured routes pass through unchanged.

## Compatibility

The plugin requires a DeepSeek Harness build whose `LlmCallConfig` and `GenerateOptions` include `topP`, `topK`, `minP`, `presencePenalty`, and `repetitionPenalty`. Until that core change reaches an npm release, install the plugin only with a matching Harness source checkout.

Adapters must map the configured fields. `@deepseek-ai/dsh-llm-pi-ai` supports the extended fields for OpenAI Chat Completions and rejects them for other protocols.

## Install

Pin the reviewed commit when installing from GitHub:

```sh
dsh plugin --profile web add github:kuma-loong/dsh-llm-sampling#<commit>
```

Git installs run this package's `prepare` script. pnpm 10 and later require an explicit build allowance in the profile's `pnpm-workspace.yaml`:

```yaml
allowBuilds:
  dsh-llm-sampling@https://codeload.github.com/kuma-loong/dsh-llm-sampling/tar.gz/<commit>: true
```

Copy the exact key printed by pnpm, then re-run the `dsh plugin add` command. Grant this permission only after reviewing the pinned source because `prepare` executes on the host during installation.

## Configure

Add an `llm-sampling` section to `$DSH_HOME/settings.yaml`:

```yaml
llm-sampling:
  providers:
    sparse-vllm:
      models:
        Qwen3.8-27B:
          default:
            temperature: 1
            topP: 0.95
            topK: 20
            minP: 0
            presencePenalty: 0
            repetitionPenalty: 1
          off:
            temperature: 0.7
            topP: 0.8
            presencePenalty: 1.5
```

Supported fields are `temperature`, `topP`, `topK`, `minP`, `presencePenalty`, and `repetitionPenalty`. Profiles are policy, not caller defaults: configured values win over earlier `agent/request` proposals. A later request policy may deliberately replace them through the normal waterfall order.

## Model Experience

### Exact-model sampling policy

#### What the model sees

No prompt text or tool schema is added. The model receives the configured sampling values in its provider request, and the effective values are recorded in the session's `request/header` before dispatch.

#### Token effect

The plugin adds no tokens. Sampling changes generation distribution and may change output length.

#### KV Cache effect

No prompt prefix changes. Providers may include sampling controls in request-cache identity, so a policy or reasoning-mode change can affect provider-side reuse even with identical input tokens.

## Known Limitations and Deferred Work

- The `off` profile is selected only for an explicit `reasoningEffort: off`; an omitted effort preserves the `default` profile because provider-owned implicit reasoning state is not guessed.
- Extended fields require adapter support; the plugin cannot determine wire compatibility before dispatch.