@deepseek-ai/dsh-tool-test-runner

Readme

# @deepseek-ai/dsh-tool-test-runner

`test_runner` 工具插件，为 DSH 模型提供跨测试框架的自动发现、执行、失败分析、覆盖率统计与测试生成建议能力。

## 插件简介

该插件向 DSH Agent 注册一个名为 `test_runner` 的工具，使模型能够：

- 自动识别项目中的测试框架（jest / vitest / pytest / mocha / playwright）
- 在指定路径执行测试并解析结构化结果
- 对失败测试做根因分类与定位建议
- 读取并汇总覆盖率报告
- 基于源代码导出符号，建议缺失的测试用例

所有操作均返回结构化 JSON，便于模型在工具链中串联使用。

## 安装与注册

通过 `cordis.patch.yml` 注册插件：

```yaml
# cordis.patch.yml
# dsh-tool-test-runner bundle patch: registers the test_runner tool plugin.
- insert:
    - id: tool-test-runner
      name: '@deepseek-ai/dsh-tool-test-runner'
      config:
        testRunnerTimeoutMs: 120000
        testRunnerMaxOutputBytes: 1048576
```

`cordis.patch.yml` 会被 DSH 构建系统自动发现并注入。包入口为 `lib/index.js`，类型定义入口为 `lib/types/index.d.ts`。

## 配置参数

| 参数 | 类型 | 默认值 | 说明 |
| --- | --- | --- | --- |
| `testRunnerTimeoutMs` | `number` | `120000` | 测试命令超时时间（毫秒） |
| `testRunnerMaxOutputBytes` | `number` | `1048576` | 单次 stdout / stderr 最大捕获字节数 |

## 5 种 Action 详解

### 1. `discover`

自动扫描当前工作区，检测测试框架、配置文件与测试文件。

**调用示例**

```json
{
  "action": "discover",
  "target": "tests/unit"
}
```

**返回示例**

```json
{
  "action": "discover",
  "framework": "vitest",
  "configFile": "/workspace/vitest.config.ts",
  "command": "npx vitest run tests/unit",
  "testFiles": [
    "/workspace/tests/unit/combat.test.ts",
    "/workspace/tests/unit/player.test.ts"
  ],
  "coverage": {
    "reportDir": "coverage",
    "reportPattern": "coverage/coverage-summary.json"
  },
  "warnings": []
}
```

**行为说明**

- 按优先级扫描以下文件来识别框架：

  | 框架 | 识别文件 |
  | --- | --- |
  | jest | `jest.config.js`, `jest.config.ts`, `jest.config.cjs`, `package.json` |
  | vitest | `vitest.config.js`, `vitest.config.ts`, `vite.config.js`, `vite.config.ts`, `package.json` |
  | pytest | `pytest.ini`, `pyproject.toml`, `tox.ini`, `setup.cfg` |
  | mocha | `.mocharc.js`, `.mocharc.cjs`, `.mocharc.json`, `mocha.config.js`, `package.json` |
  | playwright | `playwright.config.js`, `playwright.config.ts` |

- 递归收集 `.test` / `.spec`（JS/TS）与 `test_*.py`（Python）文件。
- 若未检测到配置，仍返回 `unknown` 框架并给出警告。

---

### 2. `run`

执行测试并返回结构化运行结果。

**调用示例**

```json
{
  "action": "run",
  "target": "tests/unit/combat",
  "timeout_ms": 120000,
  "max_output_bytes": 1048576
}
```

**返回示例**

```json
{
  "action": "run",
  "framework": "vitest",
  "command": "npx vitest run tests/unit/combat",
  "durationMs": 3214,
  "stdout": "[vitest]...\nTests 3 | Passed 2 | Failed 1\n",
  "stderr": "",
  "summary": {
    "passed": 2,
    "failed": 1,
    "skipped": 0,
    "total": 3,
    "exitCode": 1,
    "timedOut": false
  },
  "testResults": [
    {
      "file": "/workspace/tests/unit/combat.test.ts",
      "name": "Combat rounds apply damage",
      "status": "passed",
      "durationMs": 120,
      "error": null
    },
    {
      "file": "/workspace/tests/unit/combat.test.ts",
      "name": "Player death when health <= 0",
      "status": "failed",
      "durationMs": 80,
      "error": "Error: expect(health).toBe(0)\\n  Expected: 0\\n  Received: -1"
    }
  ],
  "failures": [
    {
      "file": "/workspace/tests/unit/combat.test.ts",
      "name": "Player death when health <= 0",
      "message": "Error: expect(health).toBe(0)",
      "stack": "Error: expect(health).toBe(0)\\n  at ...",
      "diff": "Expected: 0\\nReceived: -1",
      "codeContext": "function testPlayerDeath() { ... }"
    }
  ],
  "suggestions": [
    "Run analyze-failure for each failing test to inspect root cause."
  ]
}
```

**行为说明**

- 构建与框架匹配的命令并执行；如未指定 `target`，则对当前目录执行。
- 输出按 `max_output_bytes` 截断，超时则返回 `timedOut: true`。
- 自动解析 jest / vitest / pytest / mocha / playwright 的输出摘要。

---

### 3. `analyze-failure`

对指定失败测试文件做根因分类（基于启发式）。

**调用示例**

```json
{
  "action": "analyze-failure",
  "file": "/workspace/tests/unit/combat.test.ts"
}
```

**返回示例**

```json
{
  "action": "analyze-failure",
  "framework": "vitest",
  "failures": [
    {
      "testName": "combat.test.ts",
      "file": "/workspace/tests/unit/combat.test.ts",
      "category": "logic_error",
      "confidence": "medium",
      "explanation": "Heuristic classification: logic_error. Review the failing assertion for exact cause.",
      "stack": "Error: expect(health).toBe(0)\\n  at Context.<anonymous> (combat.test.ts:45)",
      "assertionDiff": "Expected: 0\\nReceived: -1",
      "codeContext": "test('Player death when health <= 0', () => {\\n  const player = createPlayer({ health: 1 });\\n  applyDamage(player, 5);\\n  expect(player.health).toBe(0);\\n});",
      "suggestions": [
        "Re-run the single failing test in isolation to confirm reproducibility.",
        "Check recent changes to the code under test."
      ]
    }
  ],
  "suggestions": [
    "Re-run the single failing test in isolation to confirm reproducibility.",
    "Check recent changes to the code under test."
  ]
}
```

**行为说明**

- 必须提供 `file` 参数，指向失败测试文件。
- 通过关键词匹配将失败归类为：

  - `logic_error`：断言失败、类型错误、未定义/空值
  - `environment`：网络、超时、文件缺失、权限错误
  - `data`：JSON/语法/解析错误
  - `concurrency`：锁、死锁、并发修改
  - `unknown`：无法识别

- 默认返回 `medium` 置信度；未读文件时返回 `low`。

---

### 4. `coverage`

汇总项目覆盖率报告。

**调用示例**

```json
{
  "action": "coverage",
  "target": "src"
}
```

**返回示例**

```json
{
  "action": "coverage",
  "framework": "vitest",
  "reportPath": "coverage/coverage-summary.json",
  "totals": {
    "lines": { "pct": 72, "covered": 288, "total": 400 },
    "functions": { "pct": 68, "covered": 102, "total": 150 },
    "branches": { "pct": 65, "covered": 104, "total": 160 },
    "statements": { "pct": 74, "covered": 296, "total": 400 }
  },
  "files": [
    {
      "path": "src/combat.ts",
      "lines": 120,
      "linesCovered": 108,
      "functions": 18,
      "functionsCovered": 16,
      "branches": 24,
      "branchesCovered": 20,
      "statements": 118,
      "statementsCovered": 105
    }
  ],
  "uncoveredFiles": [
    "src/combat.ts",
    "src/network.ts"
  ],
  "highRiskFiles": [
    "src/network.ts",
    "src/combat.ts",
    "src/auth.ts"
  ],
  "suggestions": [
    "Overall line coverage is 72%. Target 80%+ by adding tests for uncovered files.",
    "2 file(s) have uncovered lines.",
    "High-risk files (low coverage + branching): src/network.ts, src/combat.ts, src/auth.ts."
  ]
}
```

**行为说明**

- 优先查找框架默认覆盖率报告文件：

  | 框架 | 报告目录 | 报告文件 |
  | --- | --- | --- |
  | jest | `coverage/` | `coverage/coverage-summary.json` |
  | vitest | `coverage/` | `coverage/coverage-summary.json` |
  | pytest | `htmlcov/` | `coverage.xml` |
  | mocha | `coverage/` | `coverage/coverage-summary.json` |
  | playwright | `playwright-report/` | `index.html` |

- 若未找到报告，返回空数据并提示先运行带 coverage 的测试。
- `highRiskFiles` 按“低行覆盖率 + 低分支覆盖率”加权排序。

---

### 5. `suggest`

扫描源文件并建议缺失的测试用例。

**调用示例**

```json
{
  "action": "suggest",
  "target": "src"
}
```

**返回示例**

```json
{
  "action": "suggest",
  "framework": "vitest",
  "sourceFile": "/workspace/src/combat.ts",
  "suggestions": [
    {
      "symbol": "applyDamage",
      "scenario": "Unit coverage for applyDamage",
      "cases": [
        "normal input",
        "boundary input",
        "invalid input"
      ],
      "targetPath": "tests/unit/combat/combat_applydamage_test.js"
    },
    {
      "symbol": "heal",
      "scenario": "Unit coverage for heal",
      "cases": [
        "normal input",
        "boundary input",
        "invalid input"
      ],
      "targetPath": "tests/unit/combat/combat_heal_test.js"
    }
  ]
}
```

**行为说明**

- 扫描常见源码目录，最多分析前 20 个源文件。
- 仅提取 `export` 的函数、类、常量作为测试符号。
- 对每个符号生成 3 个标准场景：normal input / boundary input / invalid input。

## 支持框架列表

- **jest**
- **vitest**
- **pytest**
- **mocha**
- **playwright**

## 错误处理说明

| 场景 | 行为 |
| --- | --- |
| 未知 action | 抛出 `Unknown test_runner action: ...` |
| 沙箱拒绝文件访问 | 返回 `FS_SANDBOX_DENIED` 并附带提升提示 |
| 未找到配置文件 | `discover` 返回 `framework: "unknown"` + `warnings` |
| 超时 | `run` 返回 `timedOut: true`，exitCode 为 -1 |
| 无覆盖率报告 | `coverage` 返回空数据并建议先运行带 coverage 的测试 |
| `analyze-failure` 未提供 `file` | 返回警告，要求提供具体文件路径 |

工具在执行时会受 DSH 沙箱策略约束；若遇到 `FS_SANDBOX_DENIED`，可通过 `sandbox_permissions` 与 `justification` 参数申请一次性提升。

## 类型定义说明

完整类型定义位于 `lib/types/index.d.ts`，主要接口包括：

- **`TestRunnerConfig`**：插件配置
- **`TestRunnerDiscoverResult`**：discover 返回
- **`TestRunnerRunResult`**：run 返回
- **`TestRunnerAnalyzeFailureResult`**：analyze-failure 返回
- **`TestRunnerCoverageResult`**：coverage 返回
- **`TestRunnerSuggestResult`**：suggest 返回
- **`TestRunnerResult`**：以上结果的联合类型
- **`TestRunnerParameters`**：工具入参结构

示例参数常量：

```ts
import { TEST_RUNNER_EXAMPLE_PARAMS } from '@deepseek-ai/dsh-tool-test-runner';
// { action: 'discover' }
```