Self-hosting toolkit for Honcho: Hetzner+Tailscale runbook, NixOS modules, MCP shim, digest-pinned build recipe, local quantized LLM provider. Credits plastic-labs/honcho (AGPL-3.0).
  • Nix 56.1%
  • Python 20.1%
  • Shell 13.6%
  • Metal 10.2%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Lodri Péter 32fdd6b9cc feat: honcho self-host toolkit
Operator tooling for self-hosting Honcho: a full Hetzner + Tailscale +
Docker runbook, NixOS modules, an MCP shim, a digest-pinned build recipe,
and notes on running a local quantized model as the LLM provider.

Contains no upstream Honcho source. Honcho itself is AGPL-3.0 by
plastic-labs; the build recipe pins rev 7470866 and produces an
AGPL-licensed image. This repository's own content is Apache-2.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 10:23:28 +02:00
build feat: honcho self-host toolkit 2026-08-12 10:23:28 +02:00
docs feat: honcho self-host toolkit 2026-08-12 10:23:28 +02:00
mcp feat: honcho self-host toolkit 2026-08-12 10:23:28 +02:00
nix feat: honcho self-host toolkit 2026-08-12 10:23:28 +02:00
quantal feat: honcho self-host toolkit 2026-08-12 10:23:28 +02:00
.gitignore feat: honcho self-host toolkit 2026-08-12 10:23:28 +02:00
LICENSE feat: honcho self-host toolkit 2026-08-12 10:23:28 +02:00
NOTICE feat: honcho self-host toolkit 2026-08-12 10:23:28 +02:00
README.md feat: honcho self-host toolkit 2026-08-12 10:23:28 +02:00
runbook.md feat: honcho self-host toolkit 2026-08-12 10:23:28 +02:00

honcho-selfhost

Deployment and operations material for running Honcho on hardware you control. What you get in this repo:

  • a Compose runbook for a single VM, from provisioning to smoke test to backups (runbook.md)
  • NixOS modules for the same stack, declarative, with the honcho image pinned by registry digest — the other two images (pgvector/pgvector:pg15, redis:8.2) are pinned by tag, which is weaker (nix/)
  • a build recipe that reproduces the container image the modules deploy, pinned to an upstream revision (build/)
  • a single-file MCP server that puts the memory store behind five stdio tools, so any MCP client can read and write it (mcp/)
  • an agent integration guide, SDK and raw REST (docs/agent-guide.md), and a live-dashboard recipe for the metrics surface (docs/perspective-dashboard.md)
  • notes on pointing the deriver at a local quantized model instead of a metered API, plus the research artifacts that sit next to that work (quantal/)

This is not Honcho. It contains no Honcho source code. It is the scaffolding around it.

License position

Read this before you build or distribute anything.

  • This repository is Apache-2.0 licensed (see LICENSE). Its content is the operator's own deployment and operations work: a runbook, NixOS modules, an MCP shim, a build recipe, and provider documentation.

  • Upstream Honcho is AGPL-3.0: https://github.com/plastic-labs/honcho. Confirmed against the GitHub API rather than assumed, and you can confirm it the same way:

    curl -s https://api.github.com/repos/plastic-labs/honcho | jq -r '.license.spdx_id'
    # AGPL-3.0
    
  • No upstream source is vendored here. build/build.sh clones upstream at build time, pinned to revision 7470866 (full 7470866d12845ed4b56bf3449d058e65df96b1c1, dated 2026-05-21, the v3.0.7 line). Nothing in this tree is derived from that source, which is what makes Apache-2.0 on this repository sound: a separate work that configures and deploys Honcho is not a derivative of Honcho.

  • The image build/build.sh produces is an AGPL-3.0 work, because it is built from AGPL-3.0 source. The Apache-2.0 license here does not and cannot relicense it. If you distribute that image (push it to a public registry, hand it to a client, ship it inside a product) you are distributing an AGPL-3.0 work and must comply with AGPL-3.0, including offering the corresponding source of the version you distributed. AGPL-3.0 section 13 (network use) applies too: if you modify Honcho and let users interact with your modified version over a network, you owe those users the corresponding source of your modified version. Running an unmodified build for yourself on your own tailnet does not trigger section 13. Note also that upstream's Dockerfile at the pinned revision does not copy LICENSE into the image, so do not assume the license text travels with it.

  • All credit for Honcho itself belongs to Plastic Labs. This project is not affiliated with, sponsored by, or endorsed by them.

NOTICE carries the formal attribution, including the other third-party components this repository deploys, imports, or documents.

What this is, honestly

One operator's working setup, published as a reference. It is not a supported product. There is no release process, no test suite, no CI, and no promise that the next upstream release will not invalidate half of it. Read it, take what applies, adapt the rest.

Placeholders and literal examples are two different things here, and mistaking one for the other is how you end up deploying someone else's naming.

  • Placeholdered, written in angle brackets: credentials, image digests, registry handles, public IP addresses, tailnet MagicDNS names, and the public hostnames built on them. Grepping for < finds these.
  • Literal, and yours to rename: the example fleet's node names, which are spelled out rather than bracketed. honcho is the VM (and the OS user the stack runs as), m3 is the Apple Silicon inference box, and the .lan aliases built on them (honcho.lan, m3.lan) are Pi-hole records on a LAN that is not yours. docs/agent-guide.md does the same with honcho-host. None of these are placeholders and no grep for < will catch them.

The "Before you start, what to substitute" tables at the top of runbook.md list both kinds side by side and cover every placeholder used anywhere in this repository, spelled as it appears. docs/agent-guide.md and docs/perspective-dashboard.md each repeat the subset they use. Work through the runbook's tables before you copy any command out of this repo.

No credential appears in this repository, and nothing here should become one: see Secrets below.

Two deployment shapes are documented, and they are not the same deployment:

runbook.md nix/
Host Hetzner Ampere arm64 VM, Ubuntu 24.04 NixOS, linux/amd64 by default
Orchestration Docker Compose, upstream compose file plus an overlay virtualisation.oci-containers
Image built from upstream source on the box pre-built via build/, pinned by registry digest
Secrets .env owned by the service user, mode 0600 sops-nix, materialized under /run/secrets

They were written at different times and their LLM routing differs accordingly: OpenAI plus OpenRouter plus MLX-LM in the runbook, a tailnet AI gateway in the Nix modules, Ollama in quantal/, OpenAI in docs/agent-guide.md. Each document names the routing it describes. Treat them as four worked examples, not as one current configuration.

runbook.md cross-references companion files that are not in this repository: scripts/honcho/*, scripts/build-kernel.sh, scripts/hetzner.config, scripts/agentfield.config, rpi-homelab.md, and optimized-kernel-build.md. What that costs you, section by section:

  • The one-line quick deploy near the top curls scripts/honcho/deploy.sh from a fork of this repo. It will not work from this clone.
  • Sections 3, 4, 8 and 9 (provision, tailnet join, tailscale serve, smoke test) need nothing outside this repository. The one exception is §4.3, the Pi-hole detour, which cites rpi-homelab.md; skip it and use MagicDNS names directly.
  • Sections 5 and 6 (the stack itself and LLM routing) give the Compose overlay and the env deltas inline, so you can write those two files from the text. The scripts/honcho/ copies they name are still absent.
  • Section 7 is the Apple Silicon inference setup. Its one-shot installer, scripts/honcho/m3-inference-setup.sh, is not shipped; §7.2 lists what the script did so you can do most of it by hand, and §7.3 to §7.5 verify the result either way. One step is not covered anywhere in this repository and §7.2 says so plainly: how to make mlx_lm.server bind a non-loopback address. That one comes from the MLX-LM documentation.
  • Section 3a (the agentfield-tuned kernel) is optional and depends entirely on scripts/build-kernel.sh and its config files, which are absent. Nothing else in the runbook needs it.

Architecture in brief

Four containers on one host:

tailnet (MagicDNS + HTTPS via `tailscale serve`)
   |
   |   https://<HOST>.<your-tailnet>.ts.net  ->  127.0.0.1:8000
   v
 api        :8000  (loopback)   HTTP API; runs DB migrations on start
 deriver    (no port)           background: messages -> representations
 postgres   :5432  (loopback)   pgvector; data under /var/lib/honcho (see below)
 redis      :6379  (loopback)   cache;    data on /var/lib/honcho/redis
   |
   +--> chat completions:  any OpenAI-compatible endpoint (hosted or local)
   +--> embeddings:        follows that same endpoint unless overridden per
                           track; needs a real key wherever it lands

The parts that decide whether this works:

  • The tailnet is the auth boundary. AUTH_USE_AUTH=false is only defensible because nothing reaches the API except over Tailscale or loopback. Never publish 5432 or 6379. Do not enable Tailscale Funnel without turning auth on first.

  • Two base-URL knobs, and embeddings sit under the coarse one. LLM_OPENAI_BASE_URL sets the endpoint for the shared OpenAI-transport client, so it retargets every track at once; <TRACK>_MODEL_CONFIG__OVERRIDES__BASE_URL moves one track off it (per reasoning level for dialectic, flat everywhere else). Both work. The trap is that a chat proxy with no embeddings endpoint still captures embeddings, because that track has no override until you write one. nix/honcho-env.nix shows the pairing and why one real API key stays unavoidable there, quantal/README.md lists it as a failure mode people hit after assuming a keyless deployment, and runbook.md §5.3/§6 spell out both mechanisms.

  • State lives on bind mounts under /var/lib/honcho, so a host rebuild does not wipe memory. The two deployment styles put Postgres in different subdirectories and neither is wrong: the Nix modules bind /var/lib/honcho/pgdata (nix/honcho.nix), the Compose overlay in runbook.md §5.2 binds /var/lib/honcho/postgres. Redis is /var/lib/honcho/redis in both. Use the path belonging to the style you are deploying, and if you ever migrate between them, move the data rather than assuming the other path is already populated. nix/honcho-backup.nix adds a nightly pg_dump, local retention, and an optional off-box copy that lights up once its secret exists.

  • The deriver is the expensive consumer. It re-processes conversation history continuously in the background, not per request, which is the whole argument for pointing it at a local model.

  • The image is pinned to source, not pulled from upstream. Observed on 2026-08-12: the published ghcr.io/plastic-labs/honcho repository has no v3 tag, and its highest version tag is v2.0.3. It also carries floating latest and staging tags, which are not version-pinned. That is an observation with a date on it, not a standing fact, so re-check it:

    skopeo list-tags docker://ghcr.io/plastic-labs/honcho \
      | jq -r '.Tags[] | select(startswith("v"))'
    

    The MCP shim and the memory store here speak the /v3 API, so a v2 image does not degrade, it breaks them. Hence build/.

Quick start

runbook.md is the long form. The short version:

  1. Read the "Before you start, what to substitute" table at the top of runbook.md and decide what each placeholder maps to on your side. Nothing else reads cleanly until you have.
  2. Provision and lock down the host: runbook section 3 (VM, cloud firewall, baseline hardening, data volume).
  3. Join it to your tailnet and set up DNS: section 4.
  4. Bring up the stack. Either section 5 (clone upstream, Compose, build on the box), or the Nix path: run build/build.sh against your own registry per build/README.md, pin the resulting registry digest in nix/honcho.nix, and adapt nix/honcho-env.nix to your provider.
  5. Route the LLM tracks: section 6 for hosted providers, quantal/README.md for a local Ollama provider.
  6. Publish it on the tailnet: section 8, tailscale serve mapping :443 to http://127.0.0.1:8000.
  7. Smoke test end to end: section 9. Write a message, watch the deriver pick it up, ask the dialectic what it learned.
  8. Set up backups before you trust it with anything: section 11, or nix/honcho-backup.nix.
  9. Wire your agents: mcp/README.md for the MCP shim, docs/agent-guide.md for the SDK and raw REST.

Once it is up:

curl -sf https://<HOST>.<your-tailnet>.ts.net/health
# {"status":"ok"}

One gotcha worth repeating from build/README.md, because it costs people an afternoon: pin the digest that skopeo inspect reports from the registry, not the one your local engine prints after a build. They are different values and only the registry one resolves in a pull.

Repo layout

runbook.md                          Compose deployment on a single VM: provision,
                                    firewall, tailnet, stack, LLM routing, serve,
                                    smoke test, backups, upgrade/rollback, cheatsheet

build/
  README.md                         Why the image is built from pinned source, the
                                    AGPL consequences, the registry-digest gotcha,
                                    reproducibility limits, upgrade procedure
  build.sh                          Clone upstream at the pinned rev, assert the rev
                                    and version, buildx, push, read + verify digest

nix/
  honcho.nix                        oci-containers: api, deriver, pgvector, redis;
                                    network unit, image pre-pull, loopback binds
  honcho-env.nix                    Renders /etc/honcho/honcho.env declaratively,
                                    secret-free by construction
  honcho.env.example                Human-readable reference for the same env
  honcho-backup.nix                 Nightly pg_dump, local retention, optional
                                    rclone copy off-box, systemd timer

mcp/
  honcho-mcp.py                     Single-file MCP stdio server: recall, log_turn,
                                    search, representation, info. PEP 723 deps.
  README.md                         Tool semantics, env config, unix-socket
                                    transport, client registration

docs/
  agent-guide.md                    Core concepts, Python SDK patterns, multi-agent
                                    workspaces, Claude Code integration, REST
                                    cheatsheet, workspace layout advice
  perspective-dashboard.md          Worked recipe for the bridge service runbook.md
                                    §12.5 leaves as a placeholder: pinned versions,
                                    server.py + its three queries, frontend,
                                    tailscale serve rule, systemd unit

quantal/
  README.md                         Local quantized inference as honcho's provider:
                                    why, wiring, verification order, failure modes
  model-card.md                     Model card for quantal-ternary (documentation
                                    only, no weights ship here)
  attestal-proof.json               Record of the training run: checks, evidence,
                                    known failures, one annotated redaction. A
                                    snapshot taken during the run, left as
                                    written
  kernels/
    fused_ternary_matmul.metal      Reference Metal kernel, source only

LICENSE                             Apache-2.0, this repository's own content
NOTICE                              Attribution: upstream Honcho and everything else

Local inference, and the quantal directory

quantal/ exists because the deriver is the wrong workload to send to a metered third-party API. Its input is raw conversation content and its output is an explicit model of the user, it runs continuously in the background, and its cost scales with everything you have ever said to the system rather than with your visible traffic. Self-hosting Honcho and then shipping deriver traffic off-box gives back most of what self-hosting bought you. Pointing it at a local OpenAI-compatible server makes it a fixed hardware cost and lets you verify at the firewall that the box makes no outbound inference calls at all.

The trade is real and stated in that document: a small quantized model is meaningfully weaker than a frontier model, and the quality of the user representation depends on the model doing the deriving.

That directory holds two separate things, and conflating them is the easy mistake:

  1. The serving path. qwen3:8b (GGUF, Q4_K_M) served by Ollama, reached over the tailnet. Ordinary and reproducible from a clone: install Ollama, pull the model, bind it off loopback, point Honcho at /v1. The bind address is the step that fails most often, so it is documented first, along with the 404, 401, cold-load, and dual-install failure modes.
  2. The quantal research artifacts. A model card for a BitNet b1.58 ternary quantization of Qwen/Qwen2.5-0.5B, an attestal.proof.v1 record of the training run, and one reference Metal kernel. This is not wired into Honcho and nothing here serves it.

Be clear about what is not here. No weights ship in this repository: not the 168 ternary matrices, not the safetensors checkpoint. The training run used rented GPU time and an export toolchain from a separate MLX fork that is not vendored or pinned here, so the proof record is provenance you can audit, not a script you can re-run. The Metal kernel compiles and that is the only claim this repo makes for it: there is no test vector, no host harness, and no golden output, and its own README points at a suspect bounds guard in the source that was preserved rather than silently fixed.

Secrets

Nothing in this repository is a credential, and nothing here should become one.

  • Real secrets in this stack: the database password, the embedding provider API key, any chat provider key your routing needs, the Tailscale auth key, the cloud API token, and the rclone config for off-box backups.
  • Keep them in a secret store (sops-nix, a systemd LoadCredential or passwordFile, 1Password, Vault, your CI's secret manager) and inject them at runtime. In the Compose path that means an .env owned by the service user at mode 0600, sourced from that store. In the Nix path it means an extra environmentFiles entry pointing at /run/secrets.
  • In Nix specifically: anything you inline in a .nix file lands in the nix store, which is world-readable on the host. An environment attrset in an oci-containers definition is not a secret. Every module here says so at the top.
  • A committed database password is a committed database password, even when Postgres only listens on loopback.

Attribution

Honcho is built by Plastic Labs: https://github.com/plastic-labs/honcho, licensed AGPL-3.0. This repository only configures, deploys, and operates it. Bugs in Honcho belong upstream. Bugs in this material belong here.

See NOTICE for the full attribution list and LICENSE for this repository's terms.