@dsh-external/dsh-kb-rag

Readme

# @dsh-external/dsh-kb-rag

**Local-first RAG knowledge tools for DSH** — thin tool wrapper around the
[kb-rag](https://github.com/YuMu247/kb-rag) Python pipeline (Ollama bge-m3 +
ChromaDB + SiYuan). Four tools, all on-premises, zero API cost, your documents
never leave the machine.

| Tool | What it does | Backing script |
|---|---|---|
| `kb_query` | Semantic search over your local vector DB (top-k chunks with source + similarity) | `query.py --json` |
| `kb_ingest` | Ingest a docs directory into ChromaDB (incremental, manifest hash-idempotent; Shiyu notes auto-stripped of front-matter + machine layer) | `ingest.py` |
| `kb_crawl` | Batch-crawl URLs to clean Markdown (Scrapling + markdownify) | `crawl.py` |
| `kb_related` | Related-document recommendations: retrieval-style, graph-style knowledge walk, or from a known doc | `related.py` |

The plugin is a pure forwarder: it locates Python, spawns the kb-rag scripts and
parses their output. All knowledge-base logic lives in the Python project, which
you can also use standalone.

## Install

```bash
# requires a dsh build with the `dsh plugin` subcommand; <profile> = tui/headless/web/...
dsh plugin --profile <profile> add github:YuMu247/dsh-kb-rag
dsh --profile <profile>        # restart: kb_query / kb_ingest / kb_crawl / kb_related injected
```

Peer dependencies (`@deepseek-ai/dsh-tools`, `@deepseek-ai/cordis`) are provided
by the dsh profile.

## Prerequisites (one-time, on the host)

```bash
# 1. the Python pipeline
git clone https://github.com/YuMu247/kb-rag
pip install -r kb-rag/requirements.txt

# 2. Ollama + embedding model
ollama pull bge-m3        # ~1.2GB

# 3. tell the plugin where kb-rag lives (either one)
#    a) plugin config:  kbRagDir: "/path/to/kb-rag"
#    b) env var:        KB_RAG_DIR=/path/to/kb-rag
#    c) default:        ./kb-rag under the session working directory

# 4. ingest something, then ask
python kb-rag/ingest.py --docs /path/to/docs
```

Optional (Windows daily sync): `examples/sync_daily.cmd` in the kb-rag repo
starts Ollama, syncs Shiyu video notes, ingests and imports into SiYuan.

## Tool arguments

- **kb_query** — `query` (required), `k` (default 5, max 20). Returns `{ok, count, chunks:[{source, chunk, similarity, text}]}`.
- **kb_ingest** — `docs` (default `<kb-rag>/docs`), `force`, `reset`, `heading`, `chunkSize` (default 512), `overlap` (default 64). Returns `{ok, summary}`.
- **kb_crawl** — `urls` (array) or `urlsFile` (path), `out` (default `<kb-rag>/out`), `delay` (default 1.5s, min 0.2). Returns `{ok, summary, outDir}`.
- **kb_related** — `seed` or `fromDoc` (exactly one), `k` (default 5), `graph` (>0 enables graph-style walk). Returns `{ok, text}`.

## Plugin config

All keys optional, set in the profile plugin config:

| Key | Default | Meaning |
|---|---|---|
| `pythonCmd` | `python` (win) / `python3` (other) | Python executable |
| `kbRagDir` | `KB_RAG_DIR` env, then `<cwd>/kb-rag` | kb-rag checkout directory |

## Development

```bash
npm install                 # typescript + @types/node + dsh-tools/cordis types (compile only)
npm run typecheck           # tsc -b
```

Runtime has zero npm dependencies; `src/` is the source of truth and `lib/`
the compiled copy (both shipped).

## 中文说明

DSH 的本地优先知识库插件:`kb_query`(语义检索)/ `kb_ingest`(增量入库)/
`kb_crawl`(网页批量抓取)/ `kb_related`(关联推荐),底层调用
[kb-rag](https://github.com/YuMu247/kb-rag) Python 管线(Ollama bge-m3 +
ChromaDB),全程本地、零 API 成本。安装:`dsh plugin --profile <profile> add
github:YuMu247/dsh-kb-rag`。前置:克隆 kb-rag、`pip install -r requirements.txt`、
`ollama pull bge-m3`,并用配置 `kbRagDir` 或环境变量 `KB_RAG_DIR` 指明 kb-rag 路径。

## License

[MIT](LICENSE)