dsh-workspace-memory

Readme

# dsh-workspace-memory

**English** · [中文说明](#中文说明)

Workspace-scoped persistent memory for the DSH (cordis) architecture. Under each session workspace the plugin maintains a `project_memory/` directory and injects a bounded node index into the model context, so agents navigate by covering proposition instead of scanning an untyped directory.

```mermaid
flowchart LR
  subgraph W ["write path — session end"]
    EV["session events"] --> EX["extract + dedup"] --> CL["LLM classify"] --> RD["review doc"] --> PV["browser panel<br/>preview diff → commit"]
  end
  PV --> FN
  subgraph S ["project_memory/ per workspace"]
    FN["node P funnel<br/>core.md · regular.md · log"]
    VS["vec shards<br/>(cosine)"]
    GR["graph.json<br/>nodeEdges + entryLinks"]
    FN --> VS
    FN --> GR
  end
  subgraph R ["read path — next session"]
    Q["query"] --> HR["hybrid retrieval<br/>cosine + lexical → RRF"] --> CTX["bounded node index<br/>→ model context"]
  end
  VS --> HR
  GR --> HR
```

## Memory model (v3)

Atomic propositions extracted from conversations are classified in two semantic layers:

1. **Layer 1 — semantic classification**: propositions are grouped under covering propositions ("big propositions", persisted as node `p`).
2. **Layer 2 — vector network**: each node P and every funnel entry is embedded; relevance between layers is vector-space distance. Fine-grained entries may link to multiple higher-layer nodes (many-to-many `entryLinks`), and nodes connect to their k nearest neighbours (`nodeEdges`).

Below the node network sits the **three-layer funnel** (per node): `core.md` (core facts), `regular.md` (regular memory), `log/<sessionId>.md` (process logs, raw, not vectorized). Together this forms the hierarchical vector network. Retrieval is hybrid: vector cosine + lexical scoring merged by RRF, so an unconfigured workspace degrades to word-face recall without losing graph structure.

The storage/computation layer has no "dimension" concept; the legacy dimension path survives only as `legacyLabel` — a display label the GUI uses for grouped navigation.

## Storage layout

```
project_memory/
  .registry.json                     # workspace registry
  _backup/v2/<id>-<ts>/              # pre-migration snapshots (keep 2 per id)
  _system/                           # migration locks
  <subprojectId>/
    manifest.json                    # schema:3 — nodes[], graphVersion, migratedFrom?
    doctrine.md
    nodes/<nodeId>/funnel/{core.md, regular.md, log/<sid>.md}
    vec/index.json + vec/<nodeId>.json   # vector shards (JSON, brute-force cosine)
    graph.json                       # nodeEdges + entryLinks (many-to-many)
```

Workspaces created by earlier versions (schema-less manifests with a `dimensions` tree) migrate automatically on the first `ensureWorkspace` after load: deterministic node ids, entry ids preserved through the same sha1 scheme, `dimTreeVersion` carried over as `graphVersion`, whole-directory backup before any write, and restartable recovery via `_system/migrate-<id>.lock`.

## Embedding configuration

An OpenAI-compatible endpoint is configured through `api.json` at the plugin root (a `<workspace>/project_memory/api.json` override is also honored):

```json
{ "embedding": { "baseUrl": "https://…/v1", "apiKey": "sk-…", "model": "text-embedding-3-small", "dimensions": 1536 } }
```

Missing or invalid config → lexical-only mode (`needsReindex` flags the subprojects for later `memoryReindex`).

## Install

The bundle is intentionally installed out of tree:

```powershell
dsh plugin --profile web add <path-to-this-plugin>
```

Server changes require restarting the dsh web process. The browser half is served as `lib/client.js`.

Client surfaces:

- **MemoryButton** (`conversation.input.memory`) — opens the memory preview/confirm panel for the current session.
- **MemoryPreviewPanel** (`conversation.input.overlay`) — reads the bound subproject's doc, previews an update diff and commits it (`previewUpdate` / `commitUpdate`); binds/unbinds the session.
- **MemoryTreeSection** (`sidebar.workspaces.memoryTree`) — renders workspace memory groups inline with the normal sidebar tree.

## Remote API

The `workspaceMemory` Remote namespace exposes 18 methods: `listSubprojects` / `listSubprojectsByPath`, `bindSession`, `classify` / `commitClassify`, `previewUpdate` / `commitUpdate`, `renameSubproject` / `renameSubprojectByPath`, `deleteByPath` (+ `getDeleteConfirmationTokenByPath`), `reconcileAccountedSessions`, `listSessionIdsByPath`, `governance`, `search`, `seed`, `memoryReindex` / `memoryStatus`.

## Development

```powershell
npm install      # pinned devDependencies — @deepseek-ai/* packages from public npm, esbuild, typescript
npm run build    # server/ TS -> lib/; client esbuild bundle -> lib/client.js
npm test         # 7 suites via node:test
```

Source layout: `server/` holds the TypeScript source (compiled to `lib/`), `client/index.js` is the browser entry (bundled by `scripts/build-client.mjs`), `scripts/` holds build tooling. `lib/` is build output and is not tracked in git.

## Requirements & scope

- Building needs only public npm — the `@deepseek-ai/*` host packages are published there. Running the plugin requires a DSH host: the `dsh` CLI / web process that loads cordis plugins.
- An embedding endpoint is optional; without one the workspace runs in lexical-only mode while keeping the graph structure.

## License

MIT

## Other

> **Scope:** This project is designed as **Session Memory**, primarily for preserving and restoring project progress, task state, key decisions, previous attempts, and contextual continuity across Agent sessions. It is **not intended to be a large-scale Knowledge Base or a general-purpose long-term knowledge storage system**.
>
> The design targets tasks that are typically completed within a few to a dozen sessions, as well as text-production workflows. The primary goal is to help an Agent quickly recover **where the task left off, what has been tried, why certain decisions were made, and what should happen next** in a new session.

- There may still be some undiscovered bugs in this project. Thank you for your understanding.

---

## 中文说明

[English](#dsh-workspace-memory) · **中文说明**

面向 DSH(cordis)架构的工作区级持久记忆插件。插件在每个会话工作区下维护一个 `project_memory/` 目录,并向模型上下文注入有界节点索引,使 agent 按"覆盖命题"导航记忆,而不是扫描无类型目录。

```mermaid
flowchart LR
  subgraph W ["写入路径 — 会话结束"]
    EV["会话事件"] --> EX["提取 + 去重"] --> CL["LLM 分类"] --> RD["审阅文档"] --> PV["浏览器面板<br/>预览 diff → 确认提交"]
  end
  PV --> FN
  subgraph S ["project_memory/ 每工作区"]
    FN["节点 P 漏斗<br/>core.md · regular.md · log"]
    VS["向量分片<br/>(余弦)"]
    GR["graph.json<br/>nodeEdges + entryLinks"]
    FN --> VS
    FN --> GR
  end
  subgraph R ["读取路径 — 下次会话"]
    Q["查询"] --> HR["混合检索<br/>余弦 + 词法 → RRF"] --> CTX["有界节点索引<br/>→ 模型上下文"]
  end
  VS --> HR
  GR --> HR
```

### 记忆模型(v3)

从对话中提取的原子命题按两个语义层分类:

1. **第一层 — 语义分类**:命题被归入覆盖命题(大命题,持久化为节点 `p`)。
2. **第二层 — 向量网络**:每个节点 P 与每个漏斗条目都被向量化,层间相关性以向量空间距离度量。细粒度条目可挂接多个上层节点(多对多 `entryLinks`),节点之间连接各自的 k 近邻(`nodeEdges`)。

节点网络之下是**三层漏斗**(每节点):`core.md`(基础事实)、`regular.md`(常规记忆)、`log/<sessionId>.md`(过程日志,原始内容,不做向量化)。二者共同构成层级向量网络。检索是混合式的:向量余弦 + 词法打分经 RRF 融合,因此未配置 embedding 的工作区会降级为词面召回,而不丢失图结构。

存储/计算层没有"维度"概念;旧的维度路径仅以 `legacyLabel` 的形式保留——GUI 用于分组导航的显示标签。

### 存储布局

```
project_memory/
  .registry.json                     # 工作区注册表
  _backup/v2/<id>-<ts>/              # 迁移前快照(每个 id 保留 2 份)
  _system/                           # 迁移锁
  <subprojectId>/
    manifest.json                    # schema:3 — nodes[]、graphVersion、migratedFrom?
    doctrine.md
    nodes/<nodeId>/funnel/{core.md, regular.md, log/<sid>.md}
    vec/index.json + vec/<nodeId>.json   # 向量分片(JSON,暴力余弦)
    graph.json                       # nodeEdges + entryLinks(多对多)
```

由早期版本创建的工作区(无 schema、带 `dimensions` 树的 manifest)会在加载后的首次 `ensureWorkspace` 时自动迁移:确定性节点 id、条目 id 经同一 sha1 方案保留、`dimTreeVersion` 顺延为 `graphVersion`、任何写入前先整体目录备份、经 `_system/migrate-<id>.lock` 支持断点恢复。

### 嵌入配置

通过插件根目录的 `api.json` 配置 OpenAI 兼容端点(也支持 `<workspace>/project_memory/api.json` 覆盖):

```json
{ "embedding": { "baseUrl": "https://…/v1", "apiKey": "sk-…", "model": "text-embedding-3-small", "dimensions": 1536 } }
```

缺失或无效配置 → 纯词法模式(`needsReindex` 标记相应子项目,等待后续 `memoryReindex`)。

### 安装

本 bundle 有意安装在树外(out of tree):

```powershell
dsh plugin --profile web add <path-to-this-plugin>
```

服务端改动需重启 dsh web 进程。浏览器半边以 `lib/client.js` 提供。

客户端界面:

- **MemoryButton**(`conversation.input.memory`)——打开当前会话的记忆预览/确认面板。
- **MemoryPreviewPanel**(`conversation.input.overlay`)——读取已绑定子项目的文档,预览更新 diff 并提交(`previewUpdate` / `commitUpdate`);绑定/解绑会话。
- **MemoryTreeSection**(`sidebar.workspaces.memoryTree`)——把工作区记忆分组内联渲染进侧栏树。

### Remote API

`workspaceMemory` Remote 命名空间暴露 18 个方法:`listSubprojects` / `listSubprojectsByPath`、`bindSession`、`classify` / `commitClassify`、`previewUpdate` / `commitUpdate`、`renameSubproject` / `renameSubprojectByPath`、`deleteByPath`(附 `getDeleteConfirmationTokenByPath`)、`reconcileAccountedSessions`、`listSessionIdsByPath`、`governance`、`search`、`seed`、`memoryReindex` / `memoryStatus`。

### 开发

```powershell
npm install      # 安装锁定的 devDependencies —— @deepseek-ai/* 来自公共 npm,以及 esbuild、typescript
npm run build    # server/ TS 编译 -> lib/;client 经 esbuild 打包 -> lib/client.js
npm test         # 7 个测试套件,由 node:test 运行
```

源码布局:`server/` 为 TypeScript 源码(编译到 `lib/`),`client/index.js` 为浏览器入口(由 `scripts/build-client.mjs` 打包),`scripts/` 为构建工具。`lib/` 是构建产物,不入 git。

### 环境与范围

- 构建只需公共 npm——`@deepseek-ai/*` 宿主包发布在那里。运行插件需要 DSH 宿主:加载 cordis 插件的 `dsh` CLI / web 进程。
- embedding 端点是可选的;未配置时工作区以纯词法模式运行,同时保留图结构。

### 许可证

MIT
### 其他事项

> **定位说明：** 本项目定位为 **Session Memory（会话记忆）**，主要用于跨多个 Agent session 保存和恢复项目进度、任务状态、关键决策、尝试结果及上下文连续性。它并非面向超大规模数据集的知识库（Knowledge Base）或通用长期知识存储系统。
>
> 设计目标主要是支持通常在数个到十几个 session 内完成的任务，以及文字生产等场景；重点是帮助 Agent 在新的 session 中快速恢复“之前做到哪里、做过什么、为什么这样做以及下一步是什么”。

- 该项目可能包含未发现的小bug，请见谅