dsh-memory-graph

Readme

<div align="center">

# dsh-memory-graph

**Local-first long-term memory and a temporal knowledge graph for DeepSeek Harness.**

[![CI](https://github.com/zmh2000829/dsh-memory-graph/actions/workflows/ci.yml/badge.svg)](https://github.com/zmh2000829/dsh-memory-graph/actions/workflows/ci.yml)
[![Version](https://img.shields.io/badge/version-0.7.0-2563eb)](package.json)
[![Node.js](https://img.shields.io/badge/Node.js-%5E22.19%20%7C%7C%20%3E%3D24-339933?logo=nodedotjs&logoColor=white)](package.json)
[![License](https://img.shields.io/badge/license-MIT-0f172a)](LICENSE)

[简体中文](README.zh-CN.md) · [Installation](#installation) · [Configuration](#configuration) · [Security](#data-and-security)

</div>

`dsh-memory-graph` gives a DSH agent durable, auditable memory without changing DeepSeek Harness itself. It is installed as a regular Cordis plugin, stores data in a private local SQLite database, recalls relevant facts before the first model step, and renders the resulting entity graph inside DSH Web.

## Why dsh-memory-graph

- **Non-invasive by design.** No Harness source patch and no agent-loop fork. Unloading the plugin removes its tools, listeners, route, and UI contributions.
- **Memory with history.** Stable facts are superseded instead of overwritten, retain provenance, and can be reverted to the previous version.
- **A maintainable graph.** Canonical names, non-destructive alias conflict reporting, duplicate suggestions, entity merge/rename/delete, orphan discovery, reference-aware garbage collection, and normalized predicates keep the graph usable after months of conversations.
- **Chinese retrieval that works.** FTS5 trigram indexing plus a short-query fallback handles continuous CJK text without relying on whitespace tokenization.
- **Hybrid relevance.** Recall combines lexical match, graph proximity, importance, time decay, and explicit access reinforcement. Automatic recall is read-only and does not artificially keep old memories alive.
- **Optional semantic recall.** An OpenAI-compatible embedding endpoint can add cosine similarity to the local hybrid ranker. It is off by default and falls back to lexical/graph recall when no key or endpoint is available.
- **Bounded hot paths.** Write-side embedding runs in a deduplicating background queue. Query-side embedding uses a short timeout, TTL cache, and failure cooldown instead of exposing the 20-second indexing timeout to tools or first-step injection.
- **A real per-turn control.** DSH Web exposes `Default`, `On this turn`, and `Off this turn` beside the composer. The choice is latched for the next turn only and then resets.
- **Recoverable source archives.** Completed turns can retain bounded local source text, optionally including bounded tool results. Failed extraction can be retried without replaying the conversation.
- **Durable failure handling.** Retryable SQLite writes enter a private file outbox and replay at startup; malformed or permanent failures move to a dead-letter directory.
- **Inspectable, not opaque.** Every automatic memory can retain session, turn, event sequence, model route, extraction request, and extraction response.
- **Local-first and fail-open.** The database stays on the host. Background summarization failures are logged but never replace or block the original agent response.

## Architecture

```mermaid
flowchart LR
  T["Conversation turn"] --> C["Per-turn policy"]
  C --> X["Local turn archive"]
  X --> S["Background summarizer"]
  S --> M["Temporal memory store"]
  M <--> G["Canonical entity graph"]
  M --> R["Hybrid recall"]
  E["Optional embedding endpoint"] -.-> R
  G --> R
  R --> A["First agent step"]
  M --> V["DSH Web graph"]
  G --> V
  B["JSONL backup"] <--> M
```

The plugin uses DSH lifecycle extension points only. Model-visible recalled context is written to the session log, so a recorded session remains reconstructable.

## Requirements

- DeepSeek Harness with a configured profile such as `web`
- Node.js `^22.19.0` or `>=24.0.0`
- Git and npm

Confirm that the CLI and profile are available:

```bash
dsh --version
dsh plugin --profile web list
```

## Installation

The project is currently distributed directly from GitHub. Clone it, validate it, then link the local checkout into a DSH profile:

```bash
git clone https://github.com/zmh2000829/dsh-memory-graph.git
cd dsh-memory-graph
npm ci
npm run check
dsh plugin --profile web add "$PWD"
dsh web
```

Open the DSH Web interface and expand **Memory** in the sidebar. The default patch enables automatic recall, automatic turn summarization, and visualization.

The dashboard leads with what DSH remembers and how it can affect later answers. Before a response, relevant durable preferences, facts, constraints, decisions, and lessons can be recalled into context. After a successful turn, reusable information can be extracted without treating every message as memory. This usually works quietly and only for a relevant future question.

The **Knowledge graph** is a collapsed advanced view over the **profile-wide database shared by all conversations**, not a diagram of the open conversation. Nodes are extracted entities, edges are relationships, and `×N` means N memories support the same relationship. It is useful for inspecting associations, duplicate entities, and incorrect relationships; the memory list is the primary view for understanding what may influence an answer. Use a separate profile or database `path` when isolation is required.

New summaries default to `summaryLanguage: auto`, which follows the dominant user language and emits Simplified Chinese for Chinese conversations while preserving established proper names. Existing English records are not silently translated or rewritten during upgrade.

Before submitting a prompt, the native-style **Memory · On**, **Memory · Off**, or **Memory · Mixed** control shows the effective profile defaults directly. Its menu can use that profile default or override the next turn only. **On once** forces recall, archive, and summarization even when their global defaults are off; **Off once** suppresses those automatic actions. The override resets after the turn, and manual `memory_*` tools remain available. Delete buttons permanently remove a selected memory and its provenance-owned relations after confirmation.

Verify the installed link at any time:

```bash
dsh plugin --profile web list dsh-memory-graph
```

### Upgrade

The profile points to the local checkout, so an upgrade does not require reinstalling the plugin entry:

```bash
cd /path/to/dsh-memory-graph
git pull --ff-only
npm ci
npm run check
```

Restart the running DSH process after the check completes. Database schema migrations run transactionally when the plugin starts.

### Uninstall

```bash
dsh plugin --profile web remove dsh-memory-graph
```

Uninstalling removes the profile link but deliberately preserves the database and JSONL backups. Delete those files separately only when permanent data removal is intended.

## Configuration

[`cordis.patch.yml`](cordis.patch.yml) contains a production-ready local profile. Change the plugin entry in your profile and restart DSH for updates to take effect.

```yaml
- id: memory-graph
  name: dsh-memory-graph
  config:
    enabled: true
    path: !!js dshHomePath('memory-graph.sqlite')
    backupDirectory: !!js dshHomePath('memory-graph-backups')
    outboxDirectory: !!js dshHomePath('memory-graph-outbox')

    autoRecall: true
    autoRecallLimit: 4
    autoRecallMinScore: 0.18
    maxContextTokens: 1200
    recallPreviewTokens: 220

    predicateAliases:
      created_by: developed_by

    autoSummarize: true
    archiveTurns: true
    archiveMaxInputTokens: 30000
    archiveRetentionDays: 30
    archiveMaxTurnsPerSession: 200
    captureToolResults: false
    summarizeEveryTurns: 1
    summaryConcurrency: 2
    summaryMaxAttempts: 2
    summaryRetryBaseMs: 1000
    summaryRetryFailedOnStart: 3
    summaryLanguage: auto
    summaryMaxInputTokens: 6000
    summaryMaxOutputTokens: 3200

    semanticEnabled: false
    semanticEndpoint: https://api.openai.com/v1/embeddings
    semanticModel: text-embedding-3-small
    semanticApiKeyEnv: OPENAI_API_KEY
    semanticQueryTimeoutMs: 3500
    semanticQueryCacheMs: 300000
    semanticFailureCooldownMs: 60000

    visualizationEnabled: true
    visualizationAutoOpen: false
    visualizationRefreshMs: 5000
```

The major switches are independent:

| Option | Default patch | Purpose |
| --- | ---: | --- |
| `enabled` | `true` | Master switch for the entire plugin |
| `autoRecall` | `true` | Recall relevant memory before the first step of each turn |
| `autoSummarize` | `true` | Extract structured memory after successful turns |
| `archiveTurns` | `true` | Retain a bounded local source transcript for enabled completed turns |
| `archiveRetentionDays` | `30` | Delete turn archives older than the retention window |
| `archiveMaxTurnsPerSession` | `200` | Keep only the newest bounded archive count per session |
| `captureToolResults` | `false` | Include bounded tool result text in archives and extraction input |
| `summaryLanguage` | `auto` | Follow the user's dominant language, or force `zh-CN`/`en` |
| `semanticEnabled` | `false` | Add optional embedding similarity; lexical/graph fallback remains available |
| `visualizationEnabled` | `true` | Register the dashboard, turn selector, tool views, and protected action route |
| `visualizationAutoOpen` | `false` | Open the Memory dashboard automatically when DSH Web starts |

`summaryProvider` and `summaryModel` may be configured together to route extraction to a lower-cost or local model. When omitted, summarization uses the current session route. `summaryReasoningEffort` is optional and must be supported by that exact model; leave it unset unless the model advertises the chosen effort. Summary calls are serialized per session, bounded globally by `summaryConcurrency`, and retried only up to `summaryMaxAttempts`; startup also retries the oldest failed archives up to `summaryRetryFailedOnStart`. `predicateAliases` maps deployment-specific predicate spellings to one normalized vocabulary. Ranking weights, graph depth, limits, decay, timeouts, and summary thresholds are all configurable in [`cordis.patch.yml`](cordis.patch.yml).

`maxContextTokens` and `summaryMaxInputTokens` use a conservative CJK-aware estimate (approximately 1.5 tokens per CJK character and one token per four other characters). Character limits remain hard size ceilings. Preference and temporal signals, lexical overlap, normalized-content deduplication, graph proximity, and optional semantic scores feed the same bounded ranker.

Semantic recall requires an OpenAI-compatible embeddings endpoint. Set `semanticEnabled: true`, configure the endpoint/model, export the key named by `semanticApiKeyEnv`, restart DSH, and run `memory_semantic_reindex` once for existing memories. Use an empty `semanticApiKeyEnv` only for a trusted keyless local endpoint. Write indexing runs in the background; query fallback is bounded by `semanticQueryTimeoutMs`, `semanticQueryCacheMs`, and `semanticFailureCooldownMs`. At least one non-semantic ranking weight must remain positive so endpoint failure always produces finite scores.

For non-loopback Web deployments, add the exact host or `host:port` values to `visualizationTrustedHosts`. Remote access is denied by default.

## Tools

| Tool | Purpose |
| --- | --- |
| `memory_remember` | Atomically write a memory, canonical entities, aliases, and directed relations |
| `memory_recall` | Run hybrid ranked retrieval with optional graph context |
| `memory_expand` | Expand one layered recall preview to full stored text |
| `memory_graph` | Explore a bounded neighborhood or the global graph overview |
| `memory_forget` | Delete one memory and relations produced by it |
| `memory_archive_expand` | Read the bounded retained tail for one turn and report clipping explicitly |
| `memory_archive_retry` | Retry extraction from a failed turn archive |
| `memory_semantic_reindex` | Embed existing active memories when semantic recall is configured |
| `memory_entity_merge` | Merge duplicate nodes and redirect relations |
| `memory_entity_find_duplicates` | Preview conservative duplicate candidates without changing the graph |
| `memory_entity_rename` | Change an entity's canonical display name |
| `memory_entity_delete` | Remove an entity with explicit reference handling |
| `memory_entity_orphans` | List graph nodes with no active memory references |
| `memory_revert` | Restore the previous superseded version of a stable fact |
| `memory_backup` | Export or restore an exact JSONL database snapshot |

Entity and relation counts returned by write tools represent **net-new** records. Supplemental aliases never cause an implicit destructive merge: a conflicting alias remains with its existing owner and is reported to the caller. Manual rename records a preferred display spelling separately from occurrence counts. Predicates normalize to lowercase `snake_case`, with optional configured aliases.

## Visualization

DSH Web receives an interactive graph with search, drag, zoom, memory details, permanent deletion, effective configuration status, and periodic refresh. Repeated source-provenanced facts are aggregated into one visible relation with a mention count. The sidebar reads the store directly through `/memory-graph/snapshot`; opening the graph never requires a model call. ETag revalidation avoids unchanged state updates, and polling pauses while the page is hidden. Explicit mutations use a same-origin, token-protected POST endpoint.

Visualization failure is isolated from tool rendering. If the dashboard route is unavailable, historical `memory_graph` tool results still replay normally.

## Backup and recovery

Use `memory_backup` with `operation: "export"` before migrations, experiments, or manual cleanup. A restore is accepted only when the target database is empty, preventing an import from silently overwriting existing memory.

Backup paths are confined to the real `backupDirectory`, import rejects file symlinks that resolve outside it, and file names cannot contain directory components. The JSONL importer applies database defaults for columns absent from an older compatible backup. The format captures memories, embeddings, turn archives, entities, aliases, relations, provenance, and summary history.

## Data and security

- The default database is `$DSH_HOME/memory-graph.sqlite` and is created with mode `0600`.
- The store rejects an incompatible schema and databases owned by another application.
- Dashboard reads return `Cache-Control: no-store`; writes require POST, JSON, a process-random token, and validated Host, Origin, and Fetch Metadata.
- Recalled memory is explicitly framed as reference data rather than instructions.
- Automatic summarization sends the selected conversation excerpt to the configured model route. Disable `autoSummarize` or use a local route when that data must not leave the host.
- `archiveTurns` stores a bounded turn tail locally and applies both age and per-session count retention. `memory_archive_expand.truncated` says explicitly whether the archive is complete. `captureToolResults` is off by default because tool output may contain sensitive data.
- Semantic recall sends memory/query text to `semanticEndpoint` only when explicitly enabled. Use a local endpoint when those texts must not leave the host.

## Current scope

- The plugin consumes an embedding service but does not ship one. New memories are indexed automatically; existing memories require `memory_semantic_reindex`.
- Storage uses synchronous `node:sqlite` `DatabaseSync`. It is appropriate for a personal local memory store, not a multi-host database or a high-concurrency write service.
- A single DSH process should own a writable database. SQLite WAL and a bounded busy timeout protect normal transactions, but this plugin does not provide distributed writer coordination.
- `node:sqlite` may emit an experimental API warning on supported Node.js releases.
- JSONL restore intentionally requires an empty database.

## Development

```bash
npm ci
npm run typecheck
npm test
npm run build
npm pack --dry-run
```

`npm run check` runs type checking, all unit tests, and the production build. See [CONTRIBUTING.md](CONTRIBUTING.md) before proposing a change.

Release history is recorded in [CHANGELOG.md](CHANGELOG.md).
The reviewed improvement decisions and remaining roadmap are maintained in [IMPROVEMENTS.md](IMPROVEMENTS.md).
The source-level comparison with OpenViking's DSH adapter is in [COMPARISON.md](COMPARISON.md).

## License

[MIT](LICENSE) © dsh-memory-graph contributors.