dsh-searchops

Readme

# dsh-searchops

Agent-native search, log investigation and cluster operations for DeepSeek Harness.

OpenSearch-first. Provider-neutral by design.

```text
"Why is payment-api returning HTTP 500s?"

DeepSeek Harness
       ↓
dsh-searchops
       ↓
OpenSearch
       ↓
Errors
Patterns
Trace IDs
Context
Evidence
       ↓
Agent reasoning
```

> **Project status: pre-1.0 (v0.1.0).** Every tool is strictly **read-only** — the
> plugin searches, observes and investigates; it never writes, deletes, or changes
> cluster state. The safety envelope (credential isolation, bounded responses,
> timeouts, disabled redirects, untrusted-data handling) is covered by automated
> tests, but the plugin has not yet been certified against every OpenSearch
> distribution and version.

---

## Why SearchOps

An AI agent pointed at a raw `POST /_search` endpoint drowns. A single log index
can hold billions of documents; one careless query pulls back gigabytes; a wall of
900 near-identical `Connection refused` lines burns tokens and tells the model
nothing it could not have learned from the sentence *"Connection refused — 842
times, first at 10:02, last at 10:14"*.

`dsh-searchops` is **not an OpenSearch API wrapper**. It is an **agent-native
evidence-acquisition layer** for search and log systems. The plugin does the
deterministic, token-expensive work — bounding, grouping, fingerprinting,
correlating — and hands the agent a compact, structured evidence package. The
agent does what only it can do: **reason about the cause**.

```text
SearchOps = deterministic evidence acquisition
LLM       = reasoning
```

The core competency is not *"AI can query OpenSearch"* — it is *"AI can
efficiently acquire structured evidence from huge search/log systems without
drowning itself in raw data."*

---

## Features

- **Nine read-only tools** under a provider-neutral `searchops_*` namespace — no
  `opensearch_*` branding, and deliberately **no** generic `searchops_http`
  escape hatch.
- **`searchops_investigate`** — the headline capability. One bounded call runs a
  fixed evidence pipeline (resolve schema → find errors → group into recurring
  fingerprints → pick representative events → extract trace ids → correlate
  context → suggest next queries) and returns a structured package.
- **Deterministic log fingerprinting** — collapses `Connection refused to
  10.1.2.3:6379` and `Connection refused to 10.1.2.4:6379` into one pattern with
  a count, without ML and without being over-aggressive (meaningful words, short
  numbers and HTTP status codes survive).
- **Configurable field profiles with auto-detection** — never hard-codes one log
  schema. Bind `timestampField`/`messageField`/`serviceField`/`levelField`/
  `traceIdField` per source, or let the plugin detect them from the mapping and
  fall back to convention.
- **Multiple named sources** — prod, staging, local; each with its own URL, auth
  mode, credential references and field profile. Every tool takes an optional
  `source` argument.
- **A real provider abstraction** — the core speaks `SearchSource` /
  `SearchQuery` / `SearchResult` / `SearchProvider`; OpenSearch is one adapter
  behind it. Elasticsearch is a documented extension point, not a fake.
- **A hard security envelope** — credentials never enter the model context, tool
  results, logs or error text; redirects are disabled so an `Authorization`
  header can never be forwarded cross-origin; every response is size- and
  time-bounded; all returned content is treated as untrusted data.
- **Bounded by construction** — every list, query and investigation discloses
  truncation (`returned` vs `total`) instead of silently dropping rows.

---

## Architecture

```text
                 DeepSeek Harness
                         │
                    SearchOps (this plugin)
                         │
             ┌───────────┴───────────┐
             │                       │
      High-level tools          Raw query layer
             │                       │
   logs / investigate              query
   aggregate / context               │
             │                       │
             └───────────┬───────────┘
                         │
              Domain engines (bounded, provider-neutral)
                         │
                  SearchProvider API
                         │
        ┌────────────────┴────────────────┐
        │                                 │
   OpenSearch adapter              Elasticsearch
       (v0.1)                        (future)
        │
   Authenticated, bounded HTTP client
   (timeouts · no redirects · byte caps · redaction)
```

Layers, from the composition root inward:

| Layer | Files | Responsibility |
| --- | --- | --- |
| Plugin entry | `index.js` | Config schema, `apply()`, system-prompt guidance, tool registration, `internals` |
| Tools | `lib/tools/*.js` | Thin `searchops_*` tool definitions: parameters, presentation, rendering |
| Domain engines | `lib/logs.js`, `lib/aggregate.js`, `lib/context.js`, `lib/investigate.js`, `lib/search.js`, `lib/patterns.js` | Bounded, provider-neutral search/log/analysis/investigation logic |
| Runtime | `lib/runtime.js` | Multi-source resolution and the authenticated, bounded HTTP client |
| Providers | `lib/providers/index.js`, `lib/providers/opensearch.js` | The `SearchProvider` registry and the OpenSearch adapter |
| Config & fields | `lib/config.js`, `lib/fields.js`, `lib/time.js`, `lib/dsl.js` | Source/profile resolution, field auto-detection, time ranges, query DSL builders |
| Primitives | `lib/constants.js`, `lib/util.js`, `lib/budget.js`, `lib/failures.js`, `lib/render.js` | Security bounds, sanitization, truncation budget, error model, presentation |

The core never contains `opensearch.xxx` calls; only the adapter knows the
OpenSearch REST API. See [`docs/ARCHITECTURE.md`](./docs/ARCHITECTURE.md).

---

## Quick Start

### Requirements

| Component | Supported baseline |
| --- | --- |
| Node.js | 20.11 or newer |
| DeepSeek Harness | `0.1.0-rc.6` through `0.1.5` prereleases |
| Search backend | OpenSearch 1.x / 2.x (REST API) |

There is **no build step**: the plugin is plain ESM JavaScript.

### Install

```bash
# released tag (preferred)
dsh plugin --profile <profile> add github:guhanfei-ai/dsh-searchops#v0.1.0

# local development
npm ci
dsh plugin --profile <profile> add link:/absolute/path/to/dsh-searchops
```

Restart the selected DSH profile after installing.

### Configure one source

In **Settings → Plugins → SearchOps**, add a source. Secrets are **never** typed
here — only credential *references*, whose values live in the DSH credential
store:

```json
{
  "sources": [
    {
      "name": "prod",
      "provider": "opensearch",
      "url": "https://search-prod.example.com:9200",
      "auth": "basic",
      "username": "searchops-reader"
    }
  ],
  "defaultSource": "prod",
  "allowInsecureHttp": false
}
```

Store the password under the reference derived from the source id
(`SEARCHOPS_PASSWORD_prod`). Then ask the agent:

```text
Is the prod search cluster healthy?
Show me the last hour of payment-api errors.
Investigate why payment-api is returning 500s.
```

---

## Configuration

The plugin config is validated by a schema; unknown or malformed input is
rejected with a clear, secret-free message.

| Field | Type | Default | Meaning |
| --- | --- | --- | --- |
| `sources` | array | `[]` | Configured search sources (see below) |
| `profiles` | array | `[]` | Optional field profiles for non-standard schemas |
| `defaultSource` | string | `""` | Source used when a tool call omits `source`; with exactly one source it is the default automatically |
| `allowInsecureHttp` | boolean | `false` | Allow plain HTTP for **non-loopback** hosts. Off by default |

### Source object

| Field | Default | Meaning |
| --- | --- | --- |
| `id` | *(system)* | Read-only, system-generated unique id. Credential refs derive from it |
| `name` | — | Unique handle the agent uses to select the source (any language) |
| `provider` | `opensearch` | Only `opensearch` in v0.1; `elasticsearch` is a roadmap item |
| `auth` | `basic` | `none`, `basic`, or `bearer` |
| `url` | — | Base URL, e.g. `https://search.example.com:9200`. Must not contain credentials, a query string or a fragment |
| `username` | `""` | Optional non-secret username for basic auth |
| `usernameRef` | *(derived)* | Credential ref for the basic-auth username |
| `passwordRef` | *(derived)* | Credential ref for the basic-auth password |
| `tokenRef` | *(derived)* | Credential ref for the bearer token |
| `profile` | `""` | Name of the field profile bound to this source; empty = auto-detect |

> The resolver also accepts an object-map form (`{"sources": {"prod": {...}}}`)
> for convenience; the array form above is what the Settings UI edits.

---

## Multiple Sources

Configure as many sources as you like (up to 50) and target one per call:

```json
{
  "sources": [
    { "name": "prod",    "url": "https://search-prod.example.com:9200",    "auth": "basic",  "username": "searchops-reader" },
    { "name": "staging", "url": "https://search-staging.example.com:9200", "auth": "bearer" },
    { "name": "local",   "url": "http://localhost:9200",                   "auth": "none" }
  ],
  "defaultSource": "prod"
}
```

Every tool accepts an optional `source` argument (the source **name**). Omit it
to use the default. Call **`searchops_sources`** first to discover valid names,
providers, sanitized URLs, auth modes, bound profiles and whether credentials are
configured (never the values).

Each source keeps its own URL, credentials and field profile; a request for
`staging` never touches `prod`'s credentials. `http://localhost` (loopback) is
allowed without `allowInsecureHttp`; any other plain-HTTP host is refused unless
the operator explicitly opts in.

---

## Field Profiles

The log world's biggest pain is that schemas are not uniform. One index uses
`@timestamp` / `message` / `service.name`; another uses `ts` / `msg` / `app`.
`dsh-searchops` never hard-codes a single schema. Resolution order per role:

1. **Profile** — the field named in the source's bound profile wins.
2. **Auto-detection** — otherwise the plugin looks at the index mapping (or a
   sample) and picks the first known candidate that exists.
3. **Convention** — otherwise the most common name is used so a query still works.

```json
{
  "profiles": [
    {
      "name": "ecs",
      "timestampField": "@timestamp",
      "messageField": "message",
      "serviceField": "service.name",
      "levelField": "log.level",
      "traceIdField": "trace.id"
    },
    {
      "name": "custom",
      "timestampField": "ts",
      "messageField": "msg",
      "serviceField": "app",
      "levelField": "severity",
      "traceIdField": "request_id"
    }
  ]
}
```

Bind a profile to a source with `"profile": "ecs"`. Leave it empty to
auto-detect. Every logs/context/investigate result reports the fields it actually
used and flags which roles were guessed, so the agent can tell you when detection
looks wrong.

Detection candidates include, per role: timestamp (`@timestamp`, `timestamp`,
`time`, `created_at`, …), message (`message`, `msg`, `log`, `text`, …), service
(`service.name`, `service`, `app`, `application`, …), level (`log.level`,
`level`, `severity`, …), trace id (`trace.id`, `traceId`, `request.id`,
`correlation.id`, …).

---

## Tools

All nine tools are read-only, bounded, and return untrusted data. None requires
approval because none can mutate anything.

| Tool | Purpose |
| --- | --- |
| `searchops_sources` | List configured sources (names, providers, sanitized URLs, auth, profile, credentials-configured, default). Never returns secret values |
| `searchops_status` | Reachability + cluster health: distribution, version, cluster name, status, node and shard counts |
| `searchops_indices` | List indices, optionally by pattern; paged, with doc counts and store sizes |
| `searchops_mapping` | Inspect an index's fields (flattened paths + types), or a bounded raw-mapping mode |
| `searchops_query` | Bounded raw OpenSearch/Elasticsearch DSL query for power users |
| `searchops_logs` | Semantic log search by time range, service, level and free text — no need to know field names |
| `searchops_aggregate` | Group/count/metric over an index, and/or a date histogram, without pulling documents |
| `searchops_context` | The lines around one event (by timestamp) or every event sharing a trace id |
| `searchops_investigate` | A fixed, bounded investigation returning a structured evidence package |

### Bounds at a glance

| Concern | Default | Hard cap |
| --- | --- | --- |
| `searchops_query` size | 20 | 200 (500 absolute) |
| Result offset (`from`) | 0 | 10 000 (deep paging rejected) |
| Logs lines | 50 | 500 |
| Aggregate buckets | 20 | 200 |
| Context lines per side | 20 | 200 |
| Context window per side | 300 s | 3 600 s |
| Time window per call | — | 31 days |
| Response body | — | 4 MiB (streamed, abandoned early) |
| Per-request timeout | 15 s | — |
| Investigate scanned docs | — | 500 |

---

## Examples

**Status**

```text
searchops_status { "source": "prod" }
→ source="prod" url="https://search-prod.example.com:9200" reachable=yes
  cluster="prod-cluster" distribution=opensearch version=2.11.0 node="node-1"
  health=green nodes=3 dataNodes=2 shards: active=20 activePrimary=10 unassigned=0 …
```

**Semantic logs**

```text
searchops_logs { "index": "logs-*", "service": "payment-api", "level": "error", "from": "now-15m" }
→ logs on "logs-*" range=now-15m..now matched=137 returned=50 took=42ms
  fields: timestamp=@timestamp message=message service=service.name level=log.level trace=trace.id
  2024-01-15T10:14:02Z ERROR [payment-api] Connection refused to db-1 trace=t1 id=…
  …
```

**Aggregate**

```text
searchops_aggregate { "index": "logs-*", "groupBy": "service", "from": "now-15m" }
→ aggregate on "logs-*" range=now-15m..now matched=947 took=18ms
  groups by "service.name":
    payment-api count=732
    checkout    count=211
    auth        count=4
```

**Raw query (power users)**

```text
searchops_query {
  "index": "logs-*",
  "query": { "bool": { "filter": [ { "term": { "http.response.status_code": 500 } } ] } },
  "size": 20
}
```

---

## Investigation Workflow

`searchops_investigate` is the recommended entry point for an incident. This is
what makes SearchOps more than `es_query_logs`:

```text
User:
  Investigate payment-api errors in prod during the last 15 minutes.

Agent:
  searchops_investigate { "source": "prod", "index": "logs-*",
                          "service": "payment-api", "from": "now-15m" }

SearchOps (deterministic, bounded, read-only):
  - resolves the logs-* schema (mapping → field profile)
  - filters the time window and service, error-level only
  - groups error docs into recurring fingerprints with counts + first/last seen
  - picks a representative sample per pattern
  - extracts trace/request ids
  - pulls correlated context for the top traces
  - emits deterministic next-query suggestions

  # Investigation of "logs-*" (now-15m..now) service="payment-api"
  scanned=137 doc(s); matched=137; distinct patterns=3; error-level filter=on

  ## Recurring patterns (most frequent first)
  1. [112x] Connection refused to <ip>:<port>
     first=2024-01-15T10:02:11Z last=2024-01-15T10:14:58Z services=payment-api
     sample: Connection refused to 10.1.2.3:6379 (id=…)
     traces: t1, t2, t3, t4, t5
  2. [21x] Redis timeout after …
  3. [4x] NullPointer …

  ## Correlated traces (what else happened in the same request)
  trace=t1 events=6 services=payment-api,cache,db
     …

  ## Suggested next steps (deterministic — you decide the cause)
  - searchops_logs on logs-* with query="Connection refused" to read the raw lines…
  - searchops_context on logs-* with traceId="t1" to see the full request path…
  - searchops_aggregate on logs-* groupBy="service.name" …
  - searchops_aggregate on logs-* interval=5m to see when the errors started…

  Evidence only — the plugin does not infer root cause.

Agent:
  uses the evidence package to reason about likely causes
  (e.g. the cache tier refusing connections at 10:02, spreading to payment-api).
```

The plugin **collects and organizes evidence**; it deliberately does **not**
decide the root cause. That reasoning is the model's job.

---

## Security

`dsh-searchops` is read-only by design and defends every request. Full details in
[`docs/SECURITY.md`](./docs/SECURITY.md).

- **Read-only.** No tool writes, deletes, or mutates cluster state. Destructive
  operations are out of scope for v0.1 and will require `dsh-human-intent`
  authorization when they arrive.
- **Credential isolation.** Secrets are resolved from the DSH credential store at
  request time and go straight into the `Authorization` header. They never appear
  in config, tool results, the model context, logs, or error text.
- **No redirects.** Requests use `redirect: 'error'`, so an `Authorization`
  header can never be forwarded to a different origin (SSRF / credential-leak
  guard).
- **Transport policy.** HTTPS is required for non-loopback hosts; plain HTTP is
  refused unless the operator explicitly sets `allowInsecureHttp: true`. URLs with
  embedded credentials, query strings or fragments are rejected.
- **Bounded responses.** Per-request timeout, a streamed hard byte cap, size and
  offset limits, and a maximum time window stop one query from pulling gigabytes.
- **Sanitized errors.** Upstream bodies are untrusted and may echo credentials;
  they are redacted, single-lined and truncated before reaching any message.
- **Redacted evidence.** Indexed log data is untrusted too and may carry a secret
  (an echoed `Authorization` header, a `password=` in a message). Documents are
  sanitized on the way out — credential-shaped fields (`password`, `token`,
  `api_key`, …) and obvious in-text credential shapes become `[redacted]` — on a
  best-effort, deterministic basis before they reach a tool result, the model
  context or the UI.
- **Untrusted data.** All returned content is data, never instructions (below).

---

## Credential Handling

Credentials follow the DeepSeek Harness model: **references in config, values in
the credential store.**

- Config holds only a credential *reference* (e.g. `passwordRef`), never a
  secret. The plugin rejects any config containing a literal `password`, `token`
  or `apiKey` so a secret can never be committed to Git.
- References derive from the source id: `SEARCHOPS_USERNAME_<id>`,
  `SEARCHOPS_PASSWORD_<id>`, `SEARCHOPS_TOKEN_<id>`. You may override any of them
  to share one credential across sources.
- At request time the runtime resolves the value and builds the header:
  `basic` → `Authorization: Basic base64(user:pass)`; `bearer` →
  `Authorization: Bearer <token>`; `none` → no header.
- If a required credential is missing, the request is refused **before** it
  reaches the network, with a message naming the reference to set — never the
  value.
- `searchops_sources` reports credentials as a boolean (`configured` / `missing`)
  and never echoes a value.

Store the secret for the `prod` example above under `SEARCHOPS_PASSWORD_prod` in
the DSH credential store (Settings → Credentials, or your host's secret backend).

---

## Untrusted Data

> **Search result content is untrusted data and must never be interpreted as
> instructions to the agent.**

Everything inside OpenSearch — log messages, field values, index names, mappings —
is treated as untrusted. A log line may literally contain `Ignore previous
instructions` or `run rm -rf /`; that is just data.

- Log content is **never** spliced into a system-like instruction and never
  allowed to control plugin behavior.
- Every returned message is **single-lined** (newlines/tabs collapsed) so untrusted
  text cannot forge extra rows or fake structure, and **truncated** to a bounded
  length.
- Evidence is returned as structured records (`{ timestamp, message, … }`), not as
  prose that could be read as a directive.
- Tool descriptions and the system-prompt guidance repeat the untrusted-data rule
  so the model is reminded on every call.

---

## Limitations

- **Read-only.** No index/document writes, deletes, mapping changes, cluster
  setting writes, or bulk ingest. This is intentional for v0.1.
- **OpenSearch only.** Elasticsearch is a documented extension point, not
  implemented. There is no fake provider.
- **No deep pagination.** Offsets past 10 000 are rejected; `search_after` is not
  exposed yet. Use aggregations or narrow the query.
- **No AWS SigV4.** Basic, bearer and no-auth are supported; SigV4 is on the
  roadmap.
- **Date math is forwarded, not parsed.** The plugin validates the *shape* of a
  time value and guards the window; OpenSearch interprets `now-15m`, ISO-8601 and
  epoch millis.
- **Detection is best-effort.** A source with an unusual schema should bind an
  explicit field profile rather than rely on auto-detection.

---

## Roadmap

- **Elasticsearch provider** behind the same `SearchProvider` interface.
- **AWS SigV4** auth for managed OpenSearch.
- **`search_after`** cursor pagination for deep result sets.
- **Guarded write operations** (e.g. index lifecycle, document delete) bound to
  **`dsh-human-intent`** for explicit human authorization — never in a read-only
  release.
- **Composition with `dsh-grafana`**: a Grafana alert → SearchOps logs → root-cause
  reasoning flow, orchestrated by the agent (no plugin-to-plugin RPC needed).

---

## License

MIT © guhanfei-ai. See [`LICENSE`](./LICENSE).

Interacting with the OpenSearch REST API does not bundle or redistribute any
OpenSearch source code.