Sovereign, agent-driven LLM evaluation harness forked from deepseek-harness (dsh) — Cordis plugin architecture, EntheAI backends, and multi-model benchmarking
  • TypeScript 97%
  • CSS 1.6%
  • Python 0.7%
  • JavaScript 0.5%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
2026-11-17 23:56:32 -05:00
.agents Merge remote-tracking branch 'origin/master' into agent/opencode-local-astra-20261101 2026-11-01 10:54:26 +01:00
.claude docs: accuracy sweep, architecture restructure, two ADRs, review skill 2026-06-13 22:05:34 +08:00
.github Merge pull request #2252 from deepseek-harness/worktree/fix-1463-rich-content-bridge 2026-08-17 18:38:01 +08:00
apps Merge commit '7b7c14c2ed' into agent/depsb-yaml-compat-20261116 2026-11-16 01:04:18 +01:00
assets feat(branding): add quantum singularity hero banner and OG preview card to README and landing page 2026-08-18 09:57:42 +02:00
docs Merge remote-tracking branch 'origin/master' into agent/opencode-local-astra-20261101 2026-11-01 10:54:26 +01:00
examples Merge remote-tracking branch 'origin/master' into agent/opencode-local-astra-20261101 2026-11-01 10:54:26 +01:00
native feat(context): host hardware runtime context via neon sysinfo addon 2026-08-18 22:31:36 +02:00
packages fix(acp-snapshot): declare compatible Node type dependency 2026-11-16 01:07:33 +01:00
patches chore(subprocess-local): bump node-pty beta 2026-08-13 17:58:13 +08:00
personas feat: add GAIA quant engine, voice tool, and all-engines overlay 2026-11-01 18:05:05 +09:00
python build(deps): bump pydantic from 2.14.0 to 2.14.1 in /python/sdk 2026-11-10 20:05:54 +00:00
scripts Merge remote-tracking branch 'origin/master' into agent/opencode-local-astra-20261101 2026-11-01 10:54:26 +01:00
vendor chore(vendor): sync cordis 4.0.5, cosmokit 1.8.6, schemastery 3.18.5 from 8b-is fork 2026-10-22 12:01:58 +09:00
website fix(docs): infer VitePress server types from its plugin context 2026-11-16 01:21:12 +01:00
.editorconfig Declare LF and final-newline conventions in .editorconfig 2026-07-15 16:09:12 +08:00
.gitattributes feat(i18n): compose pairing records during merges 2026-08-08 21:11:59 +08:00
.gitignore fix(harness): clear pre-existing e2e gate drift 2026-11-01 12:56:54 +09:00
.gitlab-ci.yml fix: address Python release review feedback 2026-08-11 20:09:33 +08:00
.jscpd.json chore(lint): bring .tsx files into the eslint, lefthook, and jscpd lanes 2026-07-27 21:42:08 +08:00
.oxlintrc.json refactor: make lint workflows Oxlint-only 2026-08-09 14:55:15 +08:00
.oxlintrc.staged.json refactor: make lint workflows Oxlint-only 2026-08-09 14:55:15 +08:00
.rgignore chore: exclude archived Agent Notes from rg 2026-08-10 14:49:30 +08:00
AGENTS.md feat(context): host hardware runtime context via neon sysinfo addon 2026-08-18 22:31:36 +02:00
BENCHMARK.md feat(harness): benchmark gate in core + ULTRA-LSP in dsh-base 2026-08-19 15:35:16 +02:00
CLAUDE.md Initialize repo with README, AGENTS.md, and CLAUDE.md symlink 2026-06-10 22:58:56 +08:00
CONTRIBUTING.i18n.yaml docs(i18n): polish contribution greeting 2026-08-12 14:40:32 +08:00
CONTRIBUTING.md docs(i18n): refine contribution guide wording 2026-08-12 14:24:44 +08:00
CONTRIBUTING.zh.md docs(i18n): polish contribution greeting 2026-08-12 14:40:32 +08:00
KICKOFF-ULTRACODE.md feat(ui): client-side UI/UX layer, landing page dashboard, docs & design system 2026-08-17 17:56:57 +02:00
knip.json feat: launch Deepsiper with OpenCode ChatGPT credentials 2026-11-01 08:02:55 +01:00
lefthook.yml Revert "fix(client): declare browser-only externals as devDependencies" 2026-08-15 02:19:09 +08:00
LICENSE Adopt MIT for DSH packages 2026-08-13 13:07:24 +08:00
package.json fix: keep custom YAML schema consumers on compatible major 2026-11-16 00:36:21 +01:00
pnpm-lock.yaml build(deps): bump @hono/node-server from 2.1.5 to 2.1.8 2026-11-16 00:28:16 +00:00
pnpm-workspace.yaml feat(session): local Postgres durable session-persistence backend 2026-08-18 22:31:36 +02:00
pytest.ini Merge remote-tracking branch 'origin/master' into codex/trim-ai-prose 2026-07-14 00:40:36 +08:00
README.i18n.yaml fix(harness): clear pre-existing e2e gate drift 2026-11-01 12:56:54 +09:00
README.md fix(harness): clear pre-existing e2e gate drift 2026-11-01 12:56:54 +09:00
README.zh.md fix(harness): clear pre-existing e2e gate drift 2026-11-01 12:56:54 +09:00
THIRD_PARTY_NOTICES.md feat(session): local Postgres durable session-persistence backend 2026-08-18 22:31:36 +02:00
tsconfig.base.client.json refactor: replace overloaded surface terminology 2026-08-11 15:23:05 +08:00
tsconfig.base.json feat: secure hosted settings and manage persistent host MCP servers 2026-11-01 10:51:11 +01:00
tsconfig.client.json fix(session-query): rename session log export package 2026-08-13 05:02:00 +08:00
tsconfig.host.json Merge remote-tracking branch 'origin/master' into agent/opencode-local-astra-20261101 2026-11-01 10:54:26 +01:00
tsconfig.json refactor(tsconfig): single root solution graph over host/client aggregates 2026-07-23 03:59:05 +08:00
tsdown.config.ts fix(build): drop deps.neverBundle — conflicts with package-level external option 2026-08-17 18:10:54 +02:00
vitest.config.ts ci: skip 2026-08-13 18:29:09 +08:00
vitest.e2e.config.ts fix(typert): close remote gateway review gaps 2026-08-07 21:47:15 +08:00
vitest.shared.ts fix(typert): close remote gateway review gaps 2026-08-07 21:47:15 +08:00
vitest.snapshot.config.ts chore: remove remaining SDK toolchain residue 2026-08-11 16:37:29 +08:00
vitest.web-stress.config.ts test(web): add opt-in reasoning chunk stress lane 2026-08-04 14:37:31 +08:00
vitest.web.config.ts fix: vitest 2026-08-13 01:35:47 +08:00
vitest.web.perf.config.ts test: add opt-in web performance scenario 2026-08-04 14:23:34 +08:00

the N+1 — enthea v108's examiner

The unified kernel (kernel8 · ayeOS · cordis · mem8) lives under enthea v108; THIS harness is its N+1: the sovereign, agent-driven evaluation lane that watches the lane. Cordis plugin architecture + EntheAI backends + multi-model benchmarking — the examiner side of the same kernel. The corpus above all; the evaluations row the verdicts.


Deepsiper Enthea

Version License: MIT Org Upstream

Deepsiper Enthea — Sovereign LLM Evaluation Harness

English | 中文

Deepsiper Enthea (deepsiper-enthea) is a sovereign, agent-driven LLM evaluation harness forked from deepseek-ai/deepseek-harness (dsh 0.1.0-rc.7). It provides end-to-end multi-model orchestration, self-hosted and sovereign backend integration, Cordis-powered extensible plugin pipelines, and JSON-RPC automation for modern LLM evaluation workflows.


Fork Enhancements

  • Sovereign Backend: Native integration with EntheAI and self-hosted inference nodes with zero external data leakage.
  • hf-mac / Osaurus seam: The llm-osarous provider drives hf-mac's local Osaurus MLX server on 127.0.0.1:1337, keeping weights and inference in the native macOS app while the harness owns the agent loop (wiring).
  • ULTRA-LSP in core: Language-server code intelligence (goToDefinition, findReferences, goToImplementation, hover) shipped in the dsh-base core bundle with a TypeScript/JavaScript stdio provider, so the lsp tool is available in every profile.
  • Evaluation Plugins: Pluggable evaluation suites including tool-eval, eval-entheai, and custom benchmarking metrics.
  • OpenCode Bridge & JSON-RPC SDK: Drive evaluation runs and automated agent loops programmatically from OpenCode or custom orchestrators.
  • Multi-Model Orchestration: Unified provider interfaces for DeepSeek, Gemini, local models, and any OpenAI-compatible endpoint.
  • Granular Sandboxing & Telemetry: Native Landlock isolation, deterministic replay, and structured session persistence.

Architecture & Stack

Everything in Deepsiper Enthea is a composable Cordis plugin.

                  ┌─────────────────────────────────────┐
                  │    OpenCode / JSON-RPC / CLI / Web   │
                  └──────────────────┬──────────────────┘
                                     │
                  ┌──────────────────▼──────────────────┐
                  │      Cordis Kernel (Context & DI)   │
                  └────┬──────────────┬───────────────┬─┘
                       │              │               │
        ┌──────────────▼─────┐ ┌──────▼──────┐ ┌──────▼──────────────┐
        │  Sovereign Backends│ │ Eval Plugins│ │ Sandboxed Tool Seams│
        │  (EntheAI / Local) │ │ (tool-eval) │ │ (Landlock / Bash)   │
        └────────────────────┘ └─────────────┘ └─────────────────────┘
  • Runtime & Language: Node.js >=22.19.0 or >=24.0.0, TypeScript 6 (Strict ESM)
  • Bundler & Build: tsdown / rolldown + tsc project references
  • Testing & Quality: Vitest 4, Oxlint, JSCPD clone detection
  • Plugin Kernel: Vendored Cordis framework with spatiotemporal composability

Quick Start

Prerequisites

  • Node.js ^22.19.0 || >=24.0.0
  • pnpm >= 11.0.0

Installation & Build

# Clone repository
git clone https://github.com/8b-is/deepsiper-enthea.git
cd deepsiper-enthea

# Install dependencies and build harness
pnpm install
pnpm build

Empirical AST & Logic Benchmark Suite

Deepsiper Enthea includes an automated multi-case AST evaluation harness (examples/eval-entheai/driver/benchmark_sweep.py) verifying mathematical precision, algorithm syntax, and zero reward-hacking across isolated execution environments:

Benchmark Task Category Trials Pass Rate Mean Latency
FizzBuzz Logic & 18 Edge Cases Logic Verification 2 / 2 100.0% ~16.8 s
O(\log N) Matrix Power Fibonacci Mathematical Exponentiation 2 / 2 100.0% ~16.1 s
Kademlia 256-bit XOR Metric Distance Distributed DHT Routing 2 / 2 100.0% ~12.8 s
BitLinear \{-1, 0, +1\} Quantizer Ternary Weight Mapping 2 / 2 100.0% ~13.9 s
AST Invariant & Pure Syntax Validator Syntax Tree Verification 2 / 2 100.0% ~13.6 s

Run the Benchmark Sweep

# Keyless fast gate (no dsh runtime): grading logic + canonical validator
pnpm bench:entheai:verify

# Full multi-case sweep against the built dsh runtime (inline stub backend)
pnpm bench:entheai

Documentation


Community & Ecosystem


🌐 The Sovereign Constellation


License

MIT © 8b-is & DeepSeek AI contributors. Third-party dependency notices are listed in THIRD_PARTY_NOTICES.md.

Genesis Seal: 7c242080f5f821e5eaf563fe2208d60632c451687baf65f4fe8e4a0d226e3ecf · WE. {-1, 0, +1}. <3