- Nix 56.1%
- Python 20.1%
- Shell 13.6%
- Metal 10.2%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
Operator tooling for self-hosting Honcho: a full Hetzner + Tailscale + Docker runbook, NixOS modules, an MCP shim, a digest-pinned build recipe, and notes on running a local quantized model as the LLM provider. Contains no upstream Honcho source. Honcho itself is AGPL-3.0 by plastic-labs; the build recipe pins rev 7470866 and produces an AGPL-licensed image. This repository's own content is Apache-2.0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
| build | ||
| docs | ||
| mcp | ||
| nix | ||
| quantal | ||
| .gitignore | ||
| LICENSE | ||
| NOTICE | ||
| README.md | ||
| runbook.md | ||
honcho-selfhost
Deployment and operations material for running Honcho on hardware you control. What you get in this repo:
- a Compose runbook for a single VM, from provisioning to smoke test to backups
(
runbook.md) - NixOS modules for the same stack, declarative, with the honcho image pinned
by registry digest — the other two images (
pgvector/pgvector:pg15,redis:8.2) are pinned by tag, which is weaker (nix/) - a build recipe that reproduces the container image the modules deploy, pinned
to an upstream revision (
build/) - a single-file MCP server that puts the memory store behind five stdio tools,
so any MCP client can read and write it (
mcp/) - an agent integration guide, SDK and raw REST (
docs/agent-guide.md), and a live-dashboard recipe for the metrics surface (docs/perspective-dashboard.md) - notes on pointing the deriver at a local quantized model instead of a metered
API, plus the research artifacts that sit next to that work (
quantal/)
This is not Honcho. It contains no Honcho source code. It is the scaffolding around it.
License position
Read this before you build or distribute anything.
-
This repository is Apache-2.0 licensed (see
LICENSE). Its content is the operator's own deployment and operations work: a runbook, NixOS modules, an MCP shim, a build recipe, and provider documentation. -
Upstream Honcho is AGPL-3.0: https://github.com/plastic-labs/honcho. Confirmed against the GitHub API rather than assumed, and you can confirm it the same way:
curl -s https://api.github.com/repos/plastic-labs/honcho | jq -r '.license.spdx_id' # AGPL-3.0 -
No upstream source is vendored here.
build/build.shclones upstream at build time, pinned to revision7470866(full7470866d12845ed4b56bf3449d058e65df96b1c1, dated 2026-05-21, the v3.0.7 line). Nothing in this tree is derived from that source, which is what makes Apache-2.0 on this repository sound: a separate work that configures and deploys Honcho is not a derivative of Honcho. -
The image
build/build.shproduces is an AGPL-3.0 work, because it is built from AGPL-3.0 source. The Apache-2.0 license here does not and cannot relicense it. If you distribute that image (push it to a public registry, hand it to a client, ship it inside a product) you are distributing an AGPL-3.0 work and must comply with AGPL-3.0, including offering the corresponding source of the version you distributed. AGPL-3.0 section 13 (network use) applies too: if you modify Honcho and let users interact with your modified version over a network, you owe those users the corresponding source of your modified version. Running an unmodified build for yourself on your own tailnet does not trigger section 13. Note also that upstream's Dockerfile at the pinned revision does not copyLICENSEinto the image, so do not assume the license text travels with it. -
All credit for Honcho itself belongs to Plastic Labs. This project is not affiliated with, sponsored by, or endorsed by them.
NOTICE carries the formal attribution, including the other third-party
components this repository deploys, imports, or documents.
What this is, honestly
One operator's working setup, published as a reference. It is not a supported product. There is no release process, no test suite, no CI, and no promise that the next upstream release will not invalidate half of it. Read it, take what applies, adapt the rest.
Placeholders and literal examples are two different things here, and mistaking one for the other is how you end up deploying someone else's naming.
- Placeholdered, written in angle brackets: credentials, image digests,
registry handles, public IP addresses, tailnet MagicDNS names, and the
public hostnames built on them. Grepping for
<finds these. - Literal, and yours to rename: the example fleet's node names, which are
spelled out rather than bracketed.
honchois the VM (and the OS user the stack runs as),m3is the Apple Silicon inference box, and the.lanaliases built on them (honcho.lan,m3.lan) are Pi-hole records on a LAN that is not yours.docs/agent-guide.mddoes the same withhoncho-host. None of these are placeholders and no grep for<will catch them.
The "Before you start, what to substitute" tables at the top of runbook.md
list both kinds side by side and cover every placeholder used anywhere in this
repository, spelled as it appears. docs/agent-guide.md and
docs/perspective-dashboard.md each repeat the subset they use. Work through
the runbook's tables before you copy any command out of this repo.
No credential appears in this repository, and nothing here should become one: see Secrets below.
Two deployment shapes are documented, and they are not the same deployment:
runbook.md |
nix/ |
|
|---|---|---|
| Host | Hetzner Ampere arm64 VM, Ubuntu 24.04 | NixOS, linux/amd64 by default |
| Orchestration | Docker Compose, upstream compose file plus an overlay | virtualisation.oci-containers |
| Image | built from upstream source on the box | pre-built via build/, pinned by registry digest |
| Secrets | .env owned by the service user, mode 0600 |
sops-nix, materialized under /run/secrets |
They were written at different times and their LLM routing differs accordingly:
OpenAI plus OpenRouter plus MLX-LM in the runbook, a tailnet AI gateway in the
Nix modules, Ollama in quantal/, OpenAI in docs/agent-guide.md. Each
document names the routing it describes. Treat them as four worked examples,
not as one current configuration.
runbook.md cross-references companion files that are not in this
repository: scripts/honcho/*, scripts/build-kernel.sh,
scripts/hetzner.config, scripts/agentfield.config, rpi-homelab.md, and
optimized-kernel-build.md. What that costs you, section by section:
- The one-line quick deploy near the top curls
scripts/honcho/deploy.shfrom a fork of this repo. It will not work from this clone. - Sections 3, 4, 8 and 9 (provision, tailnet join,
tailscale serve, smoke test) need nothing outside this repository. The one exception is §4.3, the Pi-hole detour, which citesrpi-homelab.md; skip it and use MagicDNS names directly. - Sections 5 and 6 (the stack itself and LLM routing) give the Compose overlay
and the env deltas inline, so you can write those two files from the text.
The
scripts/honcho/copies they name are still absent. - Section 7 is the Apple Silicon inference setup. Its one-shot installer,
scripts/honcho/m3-inference-setup.sh, is not shipped; §7.2 lists what the script did so you can do most of it by hand, and §7.3 to §7.5 verify the result either way. One step is not covered anywhere in this repository and §7.2 says so plainly: how to makemlx_lm.serverbind a non-loopback address. That one comes from the MLX-LM documentation. - Section 3a (the agentfield-tuned kernel) is optional and depends entirely on
scripts/build-kernel.shand its config files, which are absent. Nothing else in the runbook needs it.
Architecture in brief
Four containers on one host:
tailnet (MagicDNS + HTTPS via `tailscale serve`)
|
| https://<HOST>.<your-tailnet>.ts.net -> 127.0.0.1:8000
v
api :8000 (loopback) HTTP API; runs DB migrations on start
deriver (no port) background: messages -> representations
postgres :5432 (loopback) pgvector; data under /var/lib/honcho (see below)
redis :6379 (loopback) cache; data on /var/lib/honcho/redis
|
+--> chat completions: any OpenAI-compatible endpoint (hosted or local)
+--> embeddings: follows that same endpoint unless overridden per
track; needs a real key wherever it lands
The parts that decide whether this works:
-
The tailnet is the auth boundary.
AUTH_USE_AUTH=falseis only defensible because nothing reaches the API except over Tailscale or loopback. Never publish 5432 or 6379. Do not enable Tailscale Funnel without turning auth on first. -
Two base-URL knobs, and embeddings sit under the coarse one.
LLM_OPENAI_BASE_URLsets the endpoint for the shared OpenAI-transport client, so it retargets every track at once;<TRACK>_MODEL_CONFIG__OVERRIDES__BASE_URLmoves one track off it (per reasoning level for dialectic, flat everywhere else). Both work. The trap is that a chat proxy with no embeddings endpoint still captures embeddings, because that track has no override until you write one.nix/honcho-env.nixshows the pairing and why one real API key stays unavoidable there,quantal/README.mdlists it as a failure mode people hit after assuming a keyless deployment, andrunbook.md§5.3/§6 spell out both mechanisms. -
State lives on bind mounts under
/var/lib/honcho, so a host rebuild does not wipe memory. The two deployment styles put Postgres in different subdirectories and neither is wrong: the Nix modules bind/var/lib/honcho/pgdata(nix/honcho.nix), the Compose overlay inrunbook.md§5.2 binds/var/lib/honcho/postgres. Redis is/var/lib/honcho/redisin both. Use the path belonging to the style you are deploying, and if you ever migrate between them, move the data rather than assuming the other path is already populated.nix/honcho-backup.nixadds a nightlypg_dump, local retention, and an optional off-box copy that lights up once its secret exists. -
The deriver is the expensive consumer. It re-processes conversation history continuously in the background, not per request, which is the whole argument for pointing it at a local model.
-
The image is pinned to source, not pulled from upstream. Observed on 2026-08-12: the published
ghcr.io/plastic-labs/honchorepository has no v3 tag, and its highest version tag isv2.0.3. It also carries floatinglatestandstagingtags, which are not version-pinned. That is an observation with a date on it, not a standing fact, so re-check it:skopeo list-tags docker://ghcr.io/plastic-labs/honcho \ | jq -r '.Tags[] | select(startswith("v"))'The MCP shim and the memory store here speak the
/v3API, so a v2 image does not degrade, it breaks them. Hencebuild/.
Quick start
runbook.md is the long form. The short version:
- Read the "Before you start, what to substitute" table at the top of
runbook.mdand decide what each placeholder maps to on your side. Nothing else reads cleanly until you have. - Provision and lock down the host: runbook section 3 (VM, cloud firewall, baseline hardening, data volume).
- Join it to your tailnet and set up DNS: section 4.
- Bring up the stack. Either section 5 (clone upstream, Compose, build on the
box), or the Nix path: run
build/build.shagainst your own registry perbuild/README.md, pin the resulting registry digest innix/honcho.nix, and adaptnix/honcho-env.nixto your provider. - Route the LLM tracks: section 6 for hosted providers,
quantal/README.mdfor a local Ollama provider. - Publish it on the tailnet: section 8,
tailscale servemapping:443tohttp://127.0.0.1:8000. - Smoke test end to end: section 9. Write a message, watch the deriver pick it up, ask the dialectic what it learned.
- Set up backups before you trust it with anything: section 11, or
nix/honcho-backup.nix. - Wire your agents:
mcp/README.mdfor the MCP shim,docs/agent-guide.mdfor the SDK and raw REST.
Once it is up:
curl -sf https://<HOST>.<your-tailnet>.ts.net/health
# {"status":"ok"}
One gotcha worth repeating from build/README.md, because it costs people an
afternoon: pin the digest that skopeo inspect reports from the registry, not
the one your local engine prints after a build. They are different values and
only the registry one resolves in a pull.
Repo layout
runbook.md Compose deployment on a single VM: provision,
firewall, tailnet, stack, LLM routing, serve,
smoke test, backups, upgrade/rollback, cheatsheet
build/
README.md Why the image is built from pinned source, the
AGPL consequences, the registry-digest gotcha,
reproducibility limits, upgrade procedure
build.sh Clone upstream at the pinned rev, assert the rev
and version, buildx, push, read + verify digest
nix/
honcho.nix oci-containers: api, deriver, pgvector, redis;
network unit, image pre-pull, loopback binds
honcho-env.nix Renders /etc/honcho/honcho.env declaratively,
secret-free by construction
honcho.env.example Human-readable reference for the same env
honcho-backup.nix Nightly pg_dump, local retention, optional
rclone copy off-box, systemd timer
mcp/
honcho-mcp.py Single-file MCP stdio server: recall, log_turn,
search, representation, info. PEP 723 deps.
README.md Tool semantics, env config, unix-socket
transport, client registration
docs/
agent-guide.md Core concepts, Python SDK patterns, multi-agent
workspaces, Claude Code integration, REST
cheatsheet, workspace layout advice
perspective-dashboard.md Worked recipe for the bridge service runbook.md
§12.5 leaves as a placeholder: pinned versions,
server.py + its three queries, frontend,
tailscale serve rule, systemd unit
quantal/
README.md Local quantized inference as honcho's provider:
why, wiring, verification order, failure modes
model-card.md Model card for quantal-ternary (documentation
only, no weights ship here)
attestal-proof.json Record of the training run: checks, evidence,
known failures, one annotated redaction. A
snapshot taken during the run, left as
written
kernels/
fused_ternary_matmul.metal Reference Metal kernel, source only
LICENSE Apache-2.0, this repository's own content
NOTICE Attribution: upstream Honcho and everything else
Local inference, and the quantal directory
quantal/ exists because the deriver is the wrong workload to send to a metered
third-party API. Its input is raw conversation content and its output is an
explicit model of the user, it runs continuously in the background, and its cost
scales with everything you have ever said to the system rather than with your
visible traffic. Self-hosting Honcho and then shipping deriver traffic off-box
gives back most of what self-hosting bought you. Pointing it at a local
OpenAI-compatible server makes it a fixed hardware cost and lets you verify at
the firewall that the box makes no outbound inference calls at all.
The trade is real and stated in that document: a small quantized model is meaningfully weaker than a frontier model, and the quality of the user representation depends on the model doing the deriving.
That directory holds two separate things, and conflating them is the easy mistake:
- The serving path.
qwen3:8b(GGUF, Q4_K_M) served by Ollama, reached over the tailnet. Ordinary and reproducible from a clone: install Ollama, pull the model, bind it off loopback, point Honcho at/v1. The bind address is the step that fails most often, so it is documented first, along with the 404, 401, cold-load, and dual-install failure modes. - The quantal research artifacts. A model card for a BitNet b1.58 ternary
quantization of
Qwen/Qwen2.5-0.5B, anattestal.proof.v1record of the training run, and one reference Metal kernel. This is not wired into Honcho and nothing here serves it.
Be clear about what is not here. No weights ship in this repository: not the 168 ternary matrices, not the safetensors checkpoint. The training run used rented GPU time and an export toolchain from a separate MLX fork that is not vendored or pinned here, so the proof record is provenance you can audit, not a script you can re-run. The Metal kernel compiles and that is the only claim this repo makes for it: there is no test vector, no host harness, and no golden output, and its own README points at a suspect bounds guard in the source that was preserved rather than silently fixed.
Secrets
Nothing in this repository is a credential, and nothing here should become one.
- Real secrets in this stack: the database password, the embedding provider API key, any chat provider key your routing needs, the Tailscale auth key, the cloud API token, and the rclone config for off-box backups.
- Keep them in a secret store (sops-nix, a systemd
LoadCredentialorpasswordFile, 1Password, Vault, your CI's secret manager) and inject them at runtime. In the Compose path that means an.envowned by the service user at mode 0600, sourced from that store. In the Nix path it means an extraenvironmentFilesentry pointing at/run/secrets. - In Nix specifically: anything you inline in a
.nixfile lands in the nix store, which is world-readable on the host. Anenvironmentattrset in anoci-containersdefinition is not a secret. Every module here says so at the top. - A committed database password is a committed database password, even when Postgres only listens on loopback.
Attribution
Honcho is built by Plastic Labs: https://github.com/plastic-labs/honcho, licensed AGPL-3.0. This repository only configures, deploys, and operates it. Bugs in Honcho belong upstream. Bugs in this material belong here.
See NOTICE for the full attribution list and LICENSE for this repository's
terms.