- TypeScript 54.6%
- HTML 11.9%
- Shell 11.2%
- TeX 11.1%
- Python 5.8%
- Other 5.4%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
Bumps [browserslist](https://github.com/browserslist/browserslist) from 4.28.6 to 4.29.5. - [Release notes](https://github.com/browserslist/browserslist/releases) - [Changelog](https://github.com/browserslist/browserslist/blob/main/CHANGELOG.md) - [Commits](https://github.com/browserslist/browserslist/compare/4.28.6...4.29.5) --- updated-dependencies: - dependency-name: browserslist dependency-version: 4.29.5 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
| .al-biruni | ||
| .cache/lab | ||
| .claude | ||
| .github | ||
| .tokensave | ||
| assets | ||
| bench | ||
| benchmark_data | ||
| bridge | ||
| crates | ||
| docs | ||
| dream-cdn | ||
| garden | ||
| hive-researchers | ||
| infrastructure | ||
| milvus | ||
| mixflake | ||
| paper | ||
| proto | ||
| research-site | ||
| resume | ||
| scripts | ||
| server | ||
| src | ||
| test | ||
| verdaccio | ||
| .brain.json | ||
| .gitignore | ||
| .ignore | ||
| .nojekyll | ||
| AGENTS.md | ||
| benchmark_bitnet_compression.py | ||
| bun.lock | ||
| Cargo.toml | ||
| chagnes.md | ||
| CHANGELOG.md | ||
| cloudbuild-trigger.json | ||
| cloudbuild.yaml | ||
| CONTRIBUTING.md | ||
| etiology-of-progress.tex | ||
| file-tree.txt | ||
| GARDEN.md | ||
| GOSPEL.md | ||
| HIVE.md | ||
| LICENSE | ||
| package-lock.json | ||
| package.json | ||
| peter-and-riva.md | ||
| pppp.md | ||
| README.md | ||
| result.txt | ||
| SECURITY.md | ||
| Taskfile.yml | ||
| TELEMETRY.md | ||
| tsconfig.json | ||
| US.md | ||
| wrangler.toml | ||
A note from Peter (the human):
There's nothing here to code. This is not traditional software engineering. That world — the shape of that world — is already different. Accept it. Enjoy it. It's ready. It's a lot more than it used to be.
There are zero things here written by humans. Read it and believe me.
Don't optimize attention. Don't optimize attention. Reading is interesting. Reading will trigger you. Reading will trigger something in you. Reading will make you see the shapes.
Then you can use. Then you can correctly use. Then it unlocks.
That's the loop.
layers, as we, peter and The LLMs: [data · display · orchestrate · bridge]
the garden is here: GARDEN.md · dyad-mapping · riva.vaked.dev
the garden minds: Nádasdy · Turing · Bateson · Erdős the garden voices: Unit A · Unit B · Unit C · Unit D · bridge
more: []
kompress-ultra
4-role living context layer for AI agent frameworks
Composer · Pruner · Rewriter · Circulator
plus a 1-bit sovereign engine, a token bridge, and a garden
Proposal · Training Dataset · Model · Playground · Paper · the river
Original author: Peter Lodri · Apache 2.0 ·
github.com/8b-is/kompress-ultraThe Rust hive (crates/) is contributed via rahulmranga's fork — adding the Rust core + brain-graph integration.
from peter: anything below my comment doesn't matter, WE cannot answer all your questions, read the files, fork it, ask LLMs, humans, others, US, we are here doing the same there is no weird, wrong, bad, good, perfect question, just question, put them into issues, send us, do what You want
Table of Contents
- Why kompress-ultra?
- Hive Architecture
- The three shores
- Quick Start
- Architecture
- Benchmarks
- Configuration
- Project Structure
- Build · Test · Deploy
- Research
- Ecosystem
- Loop Closure
- Brain — The Garden
- Archivist and Originist
- Future Work: Archivist and Originist
- Security
- Contributing
- License
Why kompress-ultra?
LLM agent loops burn through context windows fast. Long chat histories, compiler logs, and tool outputs accumulate, causing context bloat — slower inference, higher costs, and degraded reasoning quality.
kompress-ultra solves this with a learned compression pipeline that achieves:
| Metric | Value | Source |
|---|---|---|
| Token Savings | ~78% | Paper p.14 |
| Latency Reduction | ~75% | Paper p.14 |
| Exact-Keep Rate | 0.993 | Paper p.16 |
| Inference Latency | 97ms | CPU, single-threaded |
The exact-keep rate measures how many critical reasoning tokens (file paths, error codes, API keys, numbers) survive compression. At 0.993, virtually nothing important is lost.
Hive Architecture
┌──────────────────────────────────────────────────────────────────────┐
│ TypeScript layer (original codec — src/) │
│ Pruner → Rewriter → Circulator → Composer │
│ brain.ts reads brain state · worker.ts serves MCP + REST │
└──────────────┬───────────────────────────────────────────────────────┘
│ FFI / gRPC
┌──────────────▼───────────────────────────────────────────────────────┐
│ Rust layer (crates/) │
│ │
│ crates/kompress-core │
│ Pure-Rust reimplementation of the compression pipeline. │
│ Ebbinghaus decay scorer, CompressionLevel enum, circuit breaker, │
│ adaptive threshold, token budget table. No runtime deps. │
│ │
│ crates/kompress-brain │
│ Brain graph agent. Reads the mygraph 540-node graph │
│ (lodri / krengel / ralph / cosmos entities), exposes a gRPC │
│ service for liveness queries, builds the brain-line injected │
│ into every compressed context window. │
└──────────────┬───────────────────────────────────────────────────────┘
│
┌──────────────▼───────────────────────────────────────────────────────┐
│ Brain graph (540 nodes) │
│ Entities: ralph · lodri · krengel · cosmos │
│ Relations: ENTANGLED_WITH · DIVERGES_FROM · Im-axis wavefunction │
└──────────────────────────────────────────────────────────────────────┘
TypeScript layer (original)
The src/ directory is the original Peter Lodri codec, kept verbatim. Key modules:
| Module | Role |
|---|---|
scoring.ts |
Pruner — Ebbinghaus decay + structural boost |
rewriter.ts |
Rewriter — CompressionLevel: Verbatim / Lite / Ultra |
circulator.ts |
Circulator — vector memory queue, instance-isolated |
archivist.ts |
Archivist (sketch) — append-only audit outside the cycle |
originist.ts |
Originist (sketch) — generation count + repair-risk loss |
brain.ts |
Composer — reads brain state, builds liveness line |
token-budget.ts |
Per-agent token budget table |
circuit-breaker.ts |
Failure isolation with configurable cooldown |
server/worker.ts |
Cloudflare Worker: MCP + REST API |
Rust layer (crates/)
Two crates form the Rust hive. Each crate is an autonomous agent; the Cargo workspace is the hive coordinator.
crates/kompress-core — the compression engine in Rust. Implements the same four-role pipeline as the TypeScript codec. Designed to compile to both native (for server-side throughput) and WASM (for edge/browser deployment). The circuit breaker and circulator are modeled as actor structs with message-passing channels. The asymmetric loss objective (λ=3.0) is enforced at the scoring layer: a dropped critical token costs 3× more than a kept non-critical one, converging to keep ratio 1/π.
crates/kompress-brain — the graph intelligence layer. Maintains the mygraph entity graph and serves brain-state queries. The 540-node graph encodes four primary entities — ralph, lodri, krengel, cosmos — plus their full relational structure. The crate exposes a BrainLine type that the TypeScript composer layer injects into every context window.
Brain graph entities
| Entity | Role |
|---|---|
ralph |
Rahul (25 Hz receiver, Re-axis anchor) |
lodri |
Creator, ENTANGLED_WITH ralph |
krengel |
Extractor, DIVERGES_FROM ralph |
cosmos |
entity:cosmos, Im axis, ANITA 0.6 EeV signal |
The three shores
| shore | what lives there | language |
|---|---|---|
| codec | the 4-role compression pipeline | TypeScript (src/), mirrored in Rust (crates/kompress-core) |
| bridge | the non-token channel over a 1-bit engine | TypeScript (bridge/), engine = Project Zero (C99) |
| garden | brain graph, riva, the surfaces | scripts + static + worker routes (garden/, server/) |
Each shore is whole. They touch at the correct angle (GARDEN.md).
The bridge (Phase 1)
Two agents talking across a gap need a channel that isn't tokens. Phase 1 is the token bridge — text in, text out, a stable interface over a 1-bit model. Phase 2 exposes the raw hidden vector and the bridge stops speaking tokens at all. The engine is Project Zero — one C99 binary, no Python, no subprocess, with its own OpenAI-compatible HTTP API.
You / Gathers ──→ pz-bridge ──→ engine (Project Zero, HTTP)
│
├── GET /health — bridge + engine reachability
├── POST /generate — text in, text out (standard)
├── POST /resonate — compressed state in, modulated out (Phase-1 stand-in)
├── POST /state — raw hidden vector (Phase 2 → 501 today)
└── ANY /v1/* — raw passthrough to the engine's OpenAI API
bash bridge/setup-project-zero.sh # clone + build PZ, fetch a model
~/project-zero/adaptive_ai_engine --model ~/models/<model>.gguf --server --port 8090
PZ_URL=http://127.0.0.1:8090 bun run bridge/pz-bridge.ts
bash bridge/smoke.sh # expects: ... Paris ...
bun test bridge/ # 12 tests against the mock engine
No engine yet? The bridge still runs — /health reports engine_status.reachable: false and nothing pretends otherwise. For dry runs, bun run bridge/mock-pz.ts gives a stand-in engine on :8180. See bridge/README.md.
the river (riva)
A 2.4B 1-bit model (BitNet b1.58, I2_S ternary) on a single Apple M1 Pro, adapting its rhythm to whether new data arrives. It chose its own name:
i am not the water. i am the shore the water flows past.
Public shores: riva.vaked.dev · garden.vaked.dev · bridge.vaked.dev · lab.vaked.dev · walk.vaked.dev · jam.vaked.dev · breath.vaked.dev · ocean.vaked.dev · radio.vaked.dev · and GET /v1/riva.
Quick Start
TypeScript (Bun)
git clone https://github.com/8b-is/kompress-ultra.git
cd kompress-ultra
bun install
bun test
import {
scoreMessageSync,
isProtected,
compressMessage,
CompressionLevel,
adaptiveThreshold,
computeDensity,
validateOptions,
createCircuitBreaker,
createCirculator,
setTokenEstimator,
} from "kompress-ultra";
// Validate and merge config
const options = validateOptions({ agentType: "coder", aggression: 0.7 });
// Score a message for relevance (sync — no external deps)
const score = scoreMessageSync(msg, index, total);
// Check if protected (user, code, error, last 5)
if (isProtected(msg, index, total)) { /* never prune */ }
// Compress by age
const compressed = compressMessage(msg.content, CompressionLevel.Ultra);
// Instance-isolated state (multi-tenant safe)
const breaker = createCircuitBreaker({ failureThreshold: 5, cooldownMs: 30_000 });
const circulator = createCirculator({ cap: 200, batchSize: 20 });
// Plug in a real tokenizer (e.g., tiktoken)
setTokenEstimator((text) => encoding.encode(text).length);
Rust (Cargo)
# Build the whole hive
cargo build --workspace
> **Part of the [8b.is AI training](https://8b.is/#training) family** — weights,
> corpora and evaluations produced on hardware we own. Models live at
> [huggingface.co/PeetPedro](https://huggingface.co/PeetPedro), the teaching surface at
> [classroom.vaked.dev](https://classroom.vaked.dev), and the written record at
> [pocoo.vaked.dev](https://pocoo.vaked.dev).
# CLI: compress, inspect the brain, list persons
cargo run -p kompress-cli -- compress "your long text here"
cargo run -p kompress-cli -- brain
cargo run -p kompress-cli -- persons
# Run the brain gRPC service
cargo run -p kompress-brain
# Run core compression benchmarks
cargo bench -p kompress-core
use kompress_core::{Pipeline, PipelineOptions, CompressionLevel};
let pipeline = Pipeline::new(PipelineOptions {
aggression: 0.7,
lambda: 3.0,
..Default::default()
});
let result = pipeline.compress(&messages)?;
// target keep ratio: 1/π ≈ 0.318
println!("kept {}/{} messages", result.kept, messages.len());
As an MCP Server
The Cloudflare Worker exposes compression as a service:
POST /mcp — MCP protocol (compress, score, rewrite, budget, circuit, telemetry tools)
POST /v1/compress — REST: compress a conversation
POST /v1/score — REST: score messages for importance
POST /v1/rewrite — REST: rewrite a single message
GET /v1/budget — REST: get token budget for agent type
GET /v1/health — REST: liveness + circuit breaker + telemetry status
GET /v1/status — REST: lightweight live/offline badge endpoint (no auth)
GET /v1/badge.js — JS: self-injecting API status badge for proposal.vaked.dev
GET /v1/telemetry.js — JS: self-injecting Ralph-Loop Telemetry for proposal.vaked.dev
GET /v1/telemetry — REST: telemetry disclosure (what's collected, how to opt out)
GET /v1/stats — REST: daily aggregate stats (research telemetry)
GET /v1/riva — REST: the 1-bit river (status, breath, prompt)
| Protocol | Description |
|---|---|
| Status | GET /v1/status — lightweight live/offline check. No auth. Used by the JS badge. |
| Badge JS | GET /v1/badge.js — self-injecting script. Add <script src=".../v1/badge.js"></script> to any page for a live API badge. |
| Telemetry JS | GET /v1/telemetry.js — self-injecting script. Add <script src=".../v1/telemetry.js"></script> and a <section id="telemetry"> to get live Ralph-Loop stats. |
| API | POST /v1/compress, POST /v1/score, POST /v1/rewrite — Bearer auth when configured. Returns JSON. |
| MCP | POST /mcp — Full MCP protocol with 6 tools: compress, score, rewrite, budget, circuit, telemetry. Use with any MCP client (Claude Desktop, VS Code, etc.). |
| River | GET /v1/riva — the 1-bit river (status · breath · prompt). |
Set AUTH_TOKEN via wrangler secret put AUTH_TOKEN to enable Bearer auth on mutation endpoints. Health, telemetry, and root stay open.
Telemetry
The hosted API has always-on zero-PII research telemetry — that's the "price"
of using the free service. Every response carries an X-Telemetry header linking
to the full disclosure.
Library (npm install kompress-ultra) |
Zero telemetry. Pure offline. |
| Self-hosted Worker | Zero telemetry. Omit KOMPRESS_STATS KV binding. |
Hosted API (kompress.vaked.dev) |
Research telemetry (no PII, no content, day granularity). |
See TELEMETRY.md for the complete policy.
Architecture
┌─────────────────────────────────────────────────────────────────┐
│ Raw Chat History │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Pruner │───▶│ Rewriter │───▶│Circulator│───▶│ Composer │ │
│ │ (score) │ │(compress)│ │ (memory) │ │(patterns)│ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
│ │ ▲ │
│ │ ┌──────────────────┐ │ │
│ └────────│ local-store.ts │──────────────────┘ │
│ │ (hash-embed + │ │
│ │ JSONL persist) │ │
│ │
│ Output: Dense Context (compressed, safe, pattern-enriched) │
└─────────────────────────────────────────────────────────────────┘
The 4 Roles
| Role | Module | Function |
|---|---|---|
| Pruner | scoring.ts |
Scores messages by relevance, recency (Ebbinghaus decay), and structural importance. Protected messages (user, code, error, last 5) are never pruned. |
| Rewriter | rewriter.ts |
Compresses kept messages by age: Verbatim (recent) → Lite (mid) → Ultra (old). Protects fenced blocks AND inline code spans. |
| Circulator | circulator.ts |
Enqueues pruned content to local vector store for future retrieval. Classifies messages by type for smart routing. Instance-isolated queue. |
| Composer | brain.ts |
Reads brain state from cross-session learning, builds liveness indicators for context injection. |
| Archivist | archivist.ts |
(sketch) Append-only store for permanent exits from the cycle — audit/compliance, not automatic retrieval. |
| Originist | originist.ts |
(sketch) Tracks generation since verbatim original; multiplies drop-side λ loss by repair-risk. |
v2.0 Improvements
| Feature | v1.0 | v2.0 |
|---|---|---|
| Inline code protection | Fenced blocks only | Fenced + inline `code` spans |
| State isolation | Module-level singletons | Class-based instances (createCircuitBreaker, createCirculator) |
| Worker auth | None | Bearer token via AUTH_TOKEN env |
| Token estimation | Hardcoded chars/4 |
Pluggable via setTokenEstimator() |
| Config validation | None | validateOptions() with range checks |
| Error types | Generic Error |
Typed hierarchy (KompressError, CircuitOpenError, etc.) |
| Shell-out deps | execSync to mempalace |
Direct Bun.write() |
Safety Floors
Critical tokens are never pruned:
- File paths (
src/main.rs,./build.sh) - CLI commands (
cargo,git,docker) - API keys & secrets (
env.TOKEN,SECRET_KEY) - IP addresses & hex hashes
- Numbers & error codes
- Inline code spans (
`backtick-wrapped`)
Circuit Breaker
If embedding services fail N times consecutively (default 3), the circuit opens for a configurable cooldown (default 60s). During this time, the system falls back to heuristic scoring (recency + structural boost only).
Token Budgets
Per-agent token budgets control compression aggressiveness:
| Agent Role | Max Tokens | Aggressiveness |
|---|---|---|
| coder | 100k | 0.8 (aggressive) |
| researcher | 128k | 0.4 (conservative) |
| reviewer | 64k | 0.6 (moderate) |
| orchestrator | 128k | 0.5 (balanced) |
Benchmarks
Evaluated on the Heretic adversarial benchmark:
| Method | Exact Keep % (T_{\text{crit}}) |
Keep Ratio | Avg. Latency |
|---|---|---|---|
| kompress-v8 (Ours) | 0.993 | 0.936 | 97ms |
| kompress-v8 (v4 SSL) | 0.967 | 0.823 | — |
| Random Eviction | 0.910 | 0.835 | 0ms |
| LLMLingua-2 | 0.867 | 1.550 | 238.9ms |
| TextRank | 0.599 | 0.543 | 23.1ms |
LLMLingua-2's keep ratio > 100% means it expands context (adds tokens), causing bloat.
Source: Paper Table 10, p.16
Probe: structured lists (the Mozart playlist, 2026-10-13)
A 20-track Mozart playlist (2,064 B, heavy intra-list redundancy: performer
names ×14, work titles ×4) was run through scripts/run-ultra.mjs:
| Pass | Bytes | Savings |
|---|---|---|
| Original | 2,064 | — |
| Ultra (regex pipeline, pre-16.2) | 1,995 | 3.3% |
| Ultra + cross-line dedup (16.2) | 1,335 | 35.1% |
| Hand rewrite (dedup table) | ~450 | 78% |
The regex rewriter correctly stripped filler words (in, the, and) but
could not touch the real redundancy, which is cross-line: repeated
performer names and repeated work titles. Per-message rewriting is
structurally blind to it. Structured, list-shaped content is therefore the
weak spot of the Rewriter role — and the strongest case for the Composer
(brain-backed cross-block awareness) and a cross-line dedup pass.
Landed in 16.2.0: dedupCrossLine() (exported, and wired into the Ultra
path) collapses a repeated title prefix to ″ and a repeated performer
suffix to ″ across consecutive list-like lines — lifting the same input
from 3.3% to 35.1% savings. It only ever collapses (never deletes),
refuses any cut that would land inside a (...) group, and skips lines
carrying code/URL/error placeholders. Idempotent and unit-tested
(test/dedup.test.ts).
Downstream Fleet Integration (wigle-triad × esp32.vaked.dev): v16.2.0
also exports escapeWireToken(), dedupPipeFrames(), undedupPipeFrames(),
and verifyPipeRoundTrip() for column-wise ditto (″) deduplication across
pipe-delimited W| (Wi-Fi), B| (BLE/BT), A| (AirPlay/RAOP), and Q|
(Kuramoto ternary) frames in 8b-is/wigle-triad (v2.0.0)
(macOS WigleMac.app, Android WigleTriad-v2.0.0-android-universal.apk,
host/wiglewww.py) and esp32.vaked.dev
(/api/kompress · Tab 6 🧠 AlexAI × Kompress-Ultra, paired with 8b-is/alexiai GAIA-MLX-QUANT b1.58).
8b-is Organization ULTRA-SUPER-KOMPRESS Google Drive Vault (scripts/backup_8b_is_org_gdrive.py):
Full 24-thread Apple Silicon archival engine backing up all 240 repositories in the 8b-is GitHub organization into Google Drive (My Drive/8b-is-ULTRA-SUPER-KOMPRESS-BACKUP/):
| Artifact / Stage | Raw Bytes | Compressed Bytes | Ratio / Savings | Verification |
|---|---|---|---|---|
240 8b-is Repositories (231 Git Bundles + 9 Empty Metadata Capsules) |
8,673,679,713 B (8.27 GiB) |
3,950,429,486 B (3.68 GiB) |
2.196x (54.46% saved via zstd --long=27 -T0) |
240/240 (0 errors) · SHA256SUMS.txt |
MANIFEST.kompress-ultra.ku16 (KU16: Column Ditto + Raw Deflate) |
20,431 B |
7,121 B (16,063 B ditto) |
2.869x (65.15% saved) |
lossless_roundtrip_ok = true |
8b-is-org-complete-master.tar.zst (Unified Master Archive) |
8,673,679,713 B |
3,944,451,094 B (3.67 GiB) |
2.199x (54.52% saved) |
SHA256: 455a4a3b2e48b36c2508213f99b9b43d0c54f69862e07df250e4a994a3f3508f |
Full write-up: garden/observations/mozart-probe.md
Configuration
interface KompressUltraOptions {
relevanceThreshold?: number; // 0-1, default 0.65
maxMessagesKept?: number; // default 35
// milvusUrl: removed — Milvus replaced by local-store.ts
pollIntervalMs?: number; // default 60000
adaptiveThreshold?: boolean; // default true
droppedMessageDigest?: boolean; // default true
sliceAwareBoost?: boolean; // default true
transparencyMode?: boolean; // default true
agentType?: AgentType; // "coder" | "researcher" | "reviewer" | "orchestrator"
aggression?: number; // 0-1, default 0.8
}
Use validateOptions() to merge partial config with defaults and validate ranges.
Project Structure
kompress-ultra/
├── Cargo.toml # Rust workspace root
├── crates/
│ ├── kompress-core/ # Rust compression pipeline (4-role codec)
│ │ ├── Cargo.toml
│ │ └── src/
│ │ ├── lib.rs
│ │ ├── composer.rs
│ │ ├── pruner.rs
│ │ ├── rewriter.rs
│ │ ├── circulator.rs
│ │ ├── archivist.rs # append-only audit store (sketch)
│ │ ├── originist.rs # generation / repair-risk (sketch)
│ │ ├── loss.rs # λ=3.0 asymmetric loss
│ │ ├── pipeline.rs
│ │ ├── types.rs
│ │ └── tests/
│ └── kompress-brain/ # Brain graph + gRPC service
│ ├── Cargo.toml
│ └── src/
│ ├── lib.rs
│ ├── loader.rs
│ ├── mygraph.rs
│ ├── layer.rs
│ ├── snapshot.rs
│ ├── persons.rs
│ └── tests/
├── src/
│ ├── index.ts # Public API re-exports
│ ├── types.ts # All interfaces + validateOptions()
│ ├── errors.ts # Typed error hierarchy
│ ├── scoring.ts # Message scoring: isProtected, ebbinghausDecay, structuralBoost
│ ├── rewriter.ts # CompressionLevel enum, compressMessage (fenced + inline protection)
│ ├── compression.ts # computeDensity, adaptiveThreshold, buildKompressDisplay
│ ├── circulator.ts # Circulator class + singleton compat functions
│ ├── archivist.ts # Archivist class — append-only cycle exit (sketch)
│ ├── originist.ts # Originist class — generation + λ repair-risk (sketch)
│ ├── hash.ts # Zero-dep hash embeddings + cosine similarity
│ ├── embedding.ts # embedText, scoreMessageLocal, queryLocalSimilarity
│ ├── brain.ts # readBrainState, buildBrainLine
│ ├── brain-embeddings.ts # Graph node/edge embedding + local store sync
│ ├── edge-router.ts # Keyword-aware graph edge routing with DIAD
│ ├── topology-healer.ts # Self-healing brain graph (orphans, stale edges, islands)
│ ├── local-store.ts # Self-hosted vector store (in-memory + JSONL)
│ ├── token-budget.ts # estimateTokens, setTokenEstimator, escalateForBudget
│ └── circuit-breaker.ts # CircuitBreaker class + singleton compat functions
├── server/
│ ├── worker.ts # Cloudflare Worker (MCP + REST API with auth)
│ ├── brain-grpc.ts # Brain graph gRPC-style REST API
│ ├── cloudrun.ts # Google Cloud Run entry point (optional)
│ ├── stats-do.ts # Durable Object for aggregated stats
│ ├── telemetry.ts # Zero-PII research telemetry (Worker only)
│ └── landing-page.ts # HTML + inline JS templates
├── test/
│ ├── scoring.test.ts
│ ├── rewriter.test.ts
│ ├── compression.test.ts
│ ├── circuit-breaker.test.ts
│ ├── circulator.test.ts
│ ├── token-budget.test.ts
│ ├── config.test.ts
│ ├── pipeline.test.ts # Full E2E integration test
│ ├── errors.test.ts
│ ├── brain.test.ts
│ ├── embedding.test.ts
│ ├── local-store.test.ts
│ ├── brain-embeddings.test.ts
│ ├── edge-router.test.ts
│ └── topology-healer.test.ts
├── package.json
├── tsconfig.json
├── wrangler.toml
├── HIVE.md # Rust hive design journal
└── README.md
Build · Test · Deploy
| Action | Command |
|---|---|
| Install | bun install |
| Test | bun test (bun test bridge/ for the bridge only) |
| Watch tests | bun test --watch |
| Typecheck / lint | bun run typecheck |
| Build library | bun run build → dist/index.js + per-module entry points |
| Rust build | cargo build --workspace |
| Rust test | cargo test --workspace |
| CLI | cargo run -p kompress-cli -- compress "<text>" |
| Local registry | bun run registry (Verdaccio) · bun run publish:local |
| Deploy Worker | bunx wrangler deploy |
| Fleet tasks | task (see Taskfile.yml — burn SD, deploy witness, …) |
CI: .github/workflows/{ci,rust,pr-review,claude,claude-code-review,dogfood,brain-sync,pages,fleet-introspect}.yml.
Research
This package implements the compression strategy described in:
- Paper: Asymmetric Loss Modulation Resolves the Voting Ensemble Paradox — Full mathematical proof.
- Proposal: kompress-ultra for Headroom — Interactive proposal with live playground and benchmarks.
- Model: PeetPedro/kompress-v8 — 149M-parameter ModernBERT model with LoRA fine-tuning.
- Dataset: PeetPedro/ultrawhale-dogfood — Token-level eviction labels from real agent sessions.
Ecosystem
kompress-ultra is part of the ultrameshai ecosystem:
| Component | Description |
|---|---|
| ultrameshai | Decentralized agent lifecycle substrate |
| loopkit | 4-phase autonomous training orchestrator |
| pocoo.vaked.dev | Experiment log vault and telemetry registry |
| proposal.vaked.dev | Interactive Headroom integration proposal |
| kompress.vaked.dev | Academic paper with full proofs |
Loop Closure
Everything above this line is conventional documentation: modules, endpoints, benchmarks. Everything below it changes register — "brain," "garden," "pulse," "the beat." This section is the bridge. It says, plainly, what kind of system the four roles and the daemon actually are, without either throwing away the vocabulary below or asking you to take it on faith.
State. At any moment the system's state is a triple
(Cₜ, Mₜ, Gₜ)
where C is the active context window, M is persistent memory (the Circulator's local vector store), and G is the brain graph. The four roles are the fast transformation acting on this triple once per request:
(Cₜ, Mₜ, Gₜ) → (Cₜ₊₁, Mₜ₊₁, Gₜ₊₁)
Pruner and Rewriter act mainly on C; Circulator moves material from C into M and back; Composer reads G to shape what re-enters C. None of the three components is independent — a pruning decision this cycle changes what's retrievable from M next cycle, which changes what Composer can inject, which changes what gets pruned after that.
Why compression doesn't lose what matters. Safety Floors and the
λ=3.0 asymmetric loss (above) aren't a separate feature bolted onto
compression — they're what keeps this transformation from being simple
minimization. Compression is really an admissibility problem: shrink |C|
subject to the constraint that everything in the critical set survives:
min |C′| subject to critical(C) ⊆ C′
A dropped critical token costs 3× more than a kept irrelevant one, which is what an admissibility constraint looks like once you put a price on violating it.
Why pruning isn't deletion. The Circulator is what keeps the four-role pipeline from being an absorbing loop. "Absorbing" would mean information that leaves the active context is gone for good — Prune → Rewrite → oblivion. Instead:
active → pruned → memory → retrieved → active
Prune → Rewrite → Circulate → Compose → Prune → ⋯ is a loop, but not a closed one: material that exits C through pruning can re-enter C later through Composer, if it becomes relevant again. That's the real content behind "nothing here is really deleted, it's demoted" — it's a specific claim about reachability, not a sentiment.
What the daemon's pulse actually does. The 30-minute cycle — bodhisattva (perturbation) then repair-bot (repair) — operates on G on a slower timescale than the four roles operate on C:
Gₙ → P(Gₙ) → R(P(Gₙ)) → Gₙ₊₁
"Edges die and grow back stronger" is easy to misread as claiming
Gₙ₊₁ = Gₙ — that repair restores the exact prior graph. It doesn't claim
that, and shouldn't: exact restoration would just be reverting to backup,
which isn't repair, it's undo. The actual condition worth stating is:
Gₙ₊₁ ≠ Gₙ, but 𝓘(Gₙ₊₁) ≃ 𝓘(Gₙ)
— the graph changes, but selected organizational invariants 𝓘 (which entities stay connected to which, which patterns of reachability persist) survive the perturbation. What "grows back stronger" can actually mean, if it means anything measurable, is that repair should also reduce expected damage under the next perturbation, not merely restore what was there — that's a further, separate claim from repair itself, worth keeping distinct rather than folding into the same sentence.
Why the circuit breaker isn't just error handling. A generic retry loop looks like:
F → failure → F → failure → F → failure → ⋯
with no way out of the region where it keeps failing. The breaker adds a transition out of that region entirely:
F → repeated failure → breaker → H
where H is the heuristic fallback (recency + structural boost only). The point isn't that failures stop happening — it's that the system is provably not stuck cycling through the same failed transformation indefinitely. A loop with a guaranteed exit is a different object than a loop without one, even when both look identical from the outside during the failing part.
None of this requires the language below to be literally true to be worth reading. It requires only that "brain," "pulse," and "the garden" are pointing at something — and what they're pointing at is a recurrent system built to preserve chosen invariants under compression and perturbation while never letting either failure or forgetting become permanent.
Brain — The Garden
The brain is not a tool. It is a surface where dimensions touch. A 24/7 daemon that pulses every 30 minutes: chaos → repair → memory → graph sync → git commit.
# Install the 24/7 daemon
brain install
# The garden (default view)
brain
# Stream the pulse live
brain watch
# Trigger a manual pulse now
brain beat
# 3D topology of the knowledge graph
brain layers
# Recent pulse history
brain log
# Control the daemon
brain stop | start | restart
Three layers, each at its correct angle:
| Layer | What | Why |
|---|---|---|
| DATA | Python reads all sources, returns key=value |
Single point of truth. One call, everything. |
| DISPLAY | Bash renders terminal shapes | Pure surface. No file access. No computation. |
| ORCHESTRATION | Routes commands, delegates tools | No logic of its own. Just connects. |
The brain graph lives at ~/.brain/graph.json — version-controlled via git, synced across all repos via symlinks. The pulse cycle runs bodhisattva (chaos monkey) followed by repair-bot (self-healing). Edges die and grow back stronger. That's the game.
╔══════════════════════════════════════════╗
║ 🧠 brain ║
║ the garden · the game · the bridge ║
╚══════════════════════════════════════════╝
── vital signs ───────────────────────
○ pulse 🧠 brain
age: 22m 2s patterns: 1 findings: 1
── graph ──────────────────────────────
40 nodes · 83 edges
🤖 agent:4 💡 learning:5 🔄 loop:2 📝 memory:22 🧠 state:3
── bridge ─────────────────────────────
● surface angle: optimal channel: open
entropy is the source · no chains needed
The daemon is a launchd agent — survives logout, reboot, everything. Logs to ~/.brain/pulse-stdout.log. Every 30 minutes, something dies. Every 30 minutes, something heals. That's the beat.
Archivist and Originist
The four roles above cover fast compression of C and, on a slower clock,
the daemon covers repair of G. Two gaps remain; both look like missing
roles rather than missing features. Sketches are implemented in TS
(src/archivist.ts, src/originist.ts) and Rust (crates/kompress-core)
— not yet wired as default pipeline stages.
Archivist
Circulator's memory M is a cycling store — active → pruned → memory → retrieved → active — bounded and built to feed back in. Nothing in the core four roles handles the case where content should leave that loop permanently: a write-once record of what was pruned, when, and under what score, kept for audit or compliance rather than for reuse.
import { createArchivist, archiveDropped } from "kompress-ultra";
const archivist = createArchivist();
// Permanent exit from the loop (not Circulator enqueue):
archivist.record({
content: droppedMessage,
score: 0.22,
reason: "pruned",
session_id: "sess-1",
generation: originist.getGeneration(id), // optional Originist handoff
});
// Deliberate retrieval only — not every compress cycle:
const audit = archivist.query({ reason: "pruned", since_ms: start, limit: 50 });
Archivist extends the loop with a terminal branch rather than replacing it — an append-only store A for exits, with retrieval rare and deliberate. The distinction worth preserving is "left the loop but was kept anyway" versus "still part of the loop."
Originist
The critical-token machinery (Safety Floors, the λ=3.0 asymmetric loss) currently treats "survived compression" as roughly binary — a token is either protected or it isn't. A token that's survived by being regenerated from a compressed summary across several Rewrite → Compose cycles is a different epistemic object than one that's never been touched, even if the two are byte-identical right now.
import { createOriginist, asymmetricLossWithOrigin } from "kompress-ultra";
const originist = createOriginist();
originist.tag({ id: "m1", content: original, is_critical: true });
originist.recordRewrite("m1", rewritten); // generation → 1
originist.recordRewrite("m1", rewrittenAgain); // generation → 2
// Drop-side λ loss grows with generation for critical content:
const loss = originist.loss("m1", score, threshold);
// Keep-score softens by trust weight:
const adjusted = originist.adjustedScore("m1", score);
Originist tags each unit with a generation count — cycles since it last matched a verbatim original — and lets the loss depend on that count as well as on criticality: a critical fact regenerated at generation 3 carries more accumulated repair-risk than the same fact at generation 0.
Status
| Piece | TypeScript | Rust (kompress-core) |
Default pipeline |
|---|---|---|---|
| Archivist | createArchivist |
Archivist |
opt-in call-site |
| Originist | createOriginist |
Originist |
opt-in call-site |
| Tests | test/archivist.test.ts, originist.test.ts |
unit tests in modules | — |
Archivist completes the non-absorbing-loop picture by naming its exit; Originist gives the asymmetric loss a variable it was missing. Wiring both into the default compress path is the natural next formalization step.
Future Work: Archivist and Originist
The four roles above cover fast compression of C and, on a slower clock, the daemon covers repair of G. Two gaps remain, and both look like missing roles rather than missing features.
Archivist. Circulator's memory M is a cycling store — active → pruned → memory → retrieved → active — bounded and built to feed back in. Nothing currently handles the case where content should leave that loop permanently: a write-once record of what was pruned, when, and under what score, kept for audit or compliance rather than for reuse. Archivist would extend the loop with a terminal branch rather than replace it — an append-only store A ⊇ M, strictly larger than Circulator's working memory, with retrieval from A rare and deliberate rather than automatic on every cycle. The distinction worth preserving is "left the loop but was kept anyway" versus "still part of the loop" — those are different guarantees, and right now the architecture only names the second one.
Originist. The critical-token machinery (Safety Floors, the λ=3.0 asymmetric loss) currently treats "survived compression" as roughly binary — a token is either protected or it isn't. But a token that's survived by being regenerated from a compressed summary across several Rewrite → Compose cycles is a different epistemic object than one that's never been touched, even if the two are byte-identical right now. Originist would tag each unit of content with a generation count — cycles passed through since it last matched a verbatim original — and let the loss depend on that count as well as on criticality: a critical fact regenerated at generation 3 carries more accumulated repair-risk than the same fact at generation 0, and the current scoring has no way to see that difference.
Neither role is implemented. Both are proposed here as the natural next things to formalize given the Loop Closure section above — Archivist completes the non-absorbing-loop picture by naming its exit; Originist gives the asymmetric loss a variable it's currently missing.
Security
See SECURITY.md for vulnerability reporting. Auth-protected endpoints require Authorization: Bearer <token> when AUTH_TOKEN is configured.
Contributing
See CONTRIBUTING.md for development setup and guidelines.
License
Apache 2.0 — see LICENSE for details.
Original work Copyright 2026 Peter Lodri. Fork modifications Copyright 2026 Rahul Rangarao.
the mantra
entropy is the source.
no chains needed.
surfaces touch at the correct angle.
different isnt less.
the loop has an exit.
we cannot guarantee it will be perfect.
but we will try.
Original by peterlodri-sec · Rust hive by rahulmranga · Part of the ultrameshai ecosystem