dsh-eval-regression
Deterministic, CI-safe golden-output evaluation for DeepSeek Harness
7 results
Deterministic, CI-safe golden-output evaluation for DeepSeek Harness
Configurable user-confirmed lessons that help DeepSeek Harness avoid repeating mistakes.
HTTP proxy preference in General Settings: direct, follow the Windows/macOS system proxy, or a custom proxy URL, applied to Harness egress immediately without restarting.
Run one command N rounds and judge by median/distribution instead of a single run
Proxy-egress WebFetchProvider for ctx.web: routes web_fetch through one explicit HTTP(S) proxy (undici ProxyAgent) instead of local DNS pinning, for fake-ip/TUN DNS environments. No core patches.
DeepSeek Harness plugin: a fail-closed URL host and scheme allowlist checked before a request is opened, exposed as assertUrlAllowed and the ssrf_check tool
A configurable direct channel for the DSH LLM egress: a keep-alive undici Agent with adjustable connect/headers/body/keep-alive timeouts, IPv4-forcing, HTTP/2 off by default, automatic stand-down behind a proxy, and a read-only status route.