Pure-Python (+numpy) byte-graph that is a 1-bit ternary LLM: dunder API, autograd, ultra-edges, SVG viz.
  • Python 81.9%
  • Rust 16.7%
  • CMake 0.8%
  • C++ 0.4%
  • Shell 0.1%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
2026-11-17 07:52:33 -05:00
.claude feat: default to bleeding-edge Python 3.13+ with free-threading (v0.17.0) 2026-07-12 22:20:55 +02:00
.devcontainer feat: default to bleeding-edge Python 3.13+ with free-threading (v0.17.0) 2026-07-12 22:20:55 +02:00
.github/workflows chore(ci): bump deprecated actions (checkout@v4→v5) 2026-07-13 23:06:39 +02:00
.obsidian docs: add iCloud sync guide for Obsidian vault 2026-07-14 06:17:39 +02:00
_bmad chore: agent onboarding, gitignore, py.typed, eval cases, README agents section 2026-07-12 10:28:11 +02:00
assets refactor: utilize Python 3.13+ features, simplify and modernize 2026-07-12 22:36:04 +02:00
docs docs: bind lap 14 to its commit 2026-11-17 07:52:33 -05:00
examples feat(trit): eight ternary lanes per word, dotted without a multiply 2026-11-16 23:06:28 -05:00
mcp_server refactor: address review findings in model config, CLI command, MCP, and activation layers 2026-07-13 10:29:12 +02:00
rust/zerocpu perf(neon): dispatch the int8 dot to ARMv8.2 DotProd, not to intrinsics 2026-11-17 07:51:28 -05:00
skills feat: add claude-obsidian knowledge base integration 2026-07-14 05:32:11 +02:00
tests perf(observer): keep the ring's divide in i64, not i128 2026-11-17 07:49:22 -05:00
ultragraph perf(neon): dispatch the int8 dot to ARMv8.2 DotProd, not to intrinsics 2026-11-17 07:51:28 -05:00
vakedc vakedc: ultracrew re-indented parser/graph/emit (compile); document upstream source is lossy (compile_vaked blocked, needs original) 2026-07-11 09:46:31 +02:00
wiki feat: add claude-obsidian knowledge base integration 2026-07-14 05:32:11 +02:00
.editorconfig feat: Hungarian history as an ultra-graph (13 eras, 59 nodes) + timeline viz; mini SSE MCP server (mcp_server/); .devcontainer + .editorconfig; retrain Anonymus GPT (4000 steps) 2026-07-11 18:25:45 +02:00
.gitignore feat(zerocpu): trit quantizer, router, memnet, mesh — the Caveman Ultra loop (0.24.0) 2026-11-16 19:52:22 -05:00
.ignore feat(zerocpu): trit quantizer, router, memnet, mesh — the Caveman Ultra loop (0.24.0) 2026-11-16 19:52:22 -05:00
AGENTS.md feat: default to bleeding-edge Python 3.13+ with free-threading (v0.17.0) 2026-07-12 22:20:55 +02:00
CHANGELOG.md perf(neon): dispatch the int8 dot to ARMv8.2 DotProd, not to intrinsics 2026-11-17 07:51:28 -05:00
CONTRIBUTING.md feat: default to bleeding-edge Python 3.13+ with free-threading (v0.17.0) 2026-07-12 22:20:55 +02:00
justfile feat: default to bleeding-edge Python 3.13+ with free-threading (v0.17.0) 2026-07-12 22:20:55 +02:00
LICENSE docs: add MIT LICENSE and install/license sections in README 2026-07-10 18:01:41 +02:00
pyproject.toml perf(neon): dispatch the int8 dot to ARMv8.2 DotProd, not to intrinsics 2026-11-17 07:51:28 -05:00
README.md perf(memnet): read the slots that exist, not the 64 that might 2026-11-17 07:37:11 -05:00
SETUP_SUMMARY.md feat: add claude-obsidian knowledge base integration 2026-07-14 05:32:11 +02:00
uv.lock fix(cli): detect deployed checkpoints by format, not __meta__ presence 2026-07-18 05:59:47 +02:00
WIKI.md feat: add claude-obsidian knowledge base integration 2026-07-14 05:32:11 +02:00

ultragraph

CI pypi python license ruff

A pure-Python (+ numpy) byte-graph that is a 1-bit (ternary) LLM.

genesis 251e6ea · themed after pocoo.vaked.dev

ultragraph architecture — micro (node/edge, 1 byte each) → meso (tree) → macro (ultra-graph)

Three levels:

level unit storage
micro node / edge 1 byte each — int8 activation / ternary weight {-1,0,+1}
meso tree a whole graph == one net/module (a Linear/MLP block)
macro ultra-edge (===) typed wiring between trees → the ultra-graph = the model

Weights are ternary (BitNet b1.58 style); activations are int8. Full-precision "master" weights live in an ad-hoc side store during training; the byte buffers are the deployed state. Training uses a straight-through estimator (STE).

Illustrations

Real outputs from a trained ternary mini-GPT — regenerate with uv run python assets/make_figures.py:

ultra-graph causal attention ternary weight bytes
architecture attention weights

Left: the model as an ultra-graph — trees wired by ultra-edges (===), with residual skips. Middle: real causal self-attention weights (lower-triangular → no peeking at the future). Right: a trained query projection's weight bytes, each ∈ {−1, 0, +1}.

Install

pip install ultragraph-1bit    # then: import ultragraph
# or from source (Python >=3.13, free-threading ready):
uv sync

Dunder API

>> is overloaded by operand type:

import numpy as np
from ultragraph import Tree, UltraGraph, Tensor, mlp, SGD

# micro-edges inside a sparse tree
g = Tree(4, "g")
g[0] >> g[1]        # node >> node  -> micro-edge
g[2] = 7            # set a node byte
print(len(g), 2 in g, list(g))

# ultra-edges between trees
ug = UltraGraph()
a = ug.add(Tree.dense(8, 16, "a"))
b = ug.add(Tree.dense(16, 4, "b", act="none"))
a >> b              # tree >> tree -> ultra-edge (plain)
a.wire(b, "residual")

Train a tiny ternary net

ug = mlp([4, 16, 2])                 # dense ternary linear trees wired plain
opt = SGD(ug, lr=0.3, momentum=0.9)
x = Tensor(np.random.randn(32, 4).astype("float32"))
for _ in range(300):
    loss = ug.forward(x).cross_entropy(y)
    opt.zero_grad(); loss.backward(); opt.step()   # step() re-quantizes weights

See examples/char_lm.py (MLP LM), examples/transformer_lm.py (single-head attention), examples/mini_gpt.py (batched multi-head attention + RMSNorm + Adam), examples/gpt_lm.py (the whole stack: ByteTokenizer → GPT → train → stream), and examples/mesh_lm.py (a Mesh of GPT experts, gradient accumulation, joint decode) for end-to-end char/byte-level ternary language models.

from ultragraph import Embedding, MultiHeadAttention, RMSNorm, linear_tree, Adam
# pre-norm transformer block over a [B, T, d_model] sequence:
#   x = x + mha(norm1(x));  x = x + ff2(ff1(norm2(x)))

A whole ternary GPT

from ultragraph import GPT

m = GPT(vocab=256, d_model=128, n_layers=4, n_heads=4, max_len=256)  # RoPE + KV-cache
logits = m(ids)                       # ids [B, T] -> logits [B, T, vocab]
out = m.generate([72, 105], n_new=64, temperature=0.8, top_k=40, top_p=0.9,
                 repetition_penalty=1.3, stop=10, seed=0)   # stop on newline byte

for tok in m.generate([72, 105], n_new=64, temperature=0.8, stream=True):
    print(tok, end=" ", flush=True)   # token-by-token
m.save("gpt.npz")                     # fp32 masters; reload onto the same architecture

# a true 1-bit-on-disk checkpoint: bit-packed ternary bytes, no fp32 masters.
m.save_deployed("gpt.q.npz")          # ~10x smaller, inference-only
deployed = GPT.load_deployed("gpt.q.npz")   # byte-exact logits, runs from the trits

The deployed checkpoint stores weights at their true ~1.6 bits/weight density (5 ternary values per byte) plus the tiny fp32 pieces (embedding, norm gains, biases). On an 858k-param model that's 3.4 MB → 334 KB, and deployed(ids) gives logits identical to the trained model — Tree.forward runs straight from the stored bytes.

A mesh of minds

from ultragraph import GPT, Mesh

experts = [GPT(vocab=256, d_model=64, n_layers=2, n_heads=4) for _ in range(4)]
mesh = Mesh(experts, vocab=256, top_k=2)     # a learned router mixes full models
logits = mesh(ids)                           # Σ_e gate(ids)_e · expert_e(ids)
text = mesh.generate([72, 105], n_new=64, temperature=0.8)   # joint KV-cached decode

Mesh lifts nn.MoE's routing to whole networks: a small ternary router reads the sequence and mixes the experts' logits per sequence (soft, or top-k). Router and every expert train together — a graph of minds, still all ternary bytes underneath.

generate decodes with a per-layer KV-cache; since activations are quantized per token, a cached step is byte-for-byte the full-forward result at that position. Positions come from RoPE (rotary embeddings) — relative, and offset-aware so they line up across cached steps.

Guide — common recipes

Goal Command
Install pip install ultragraph-1bit (extras: viz, mcp, wiki)
Train a byte-level GPT uv run python examples/gpt_lm.py
Mixture of full models uv run python examples/mesh_lm.py
1-bit Latin LLM (Anonymus) fetch_gesta.py → anonymus_lm.py
1-bit Hungarian LLM (resumable) fetch_hungarian.py → hungarian_lm.py
Enrich the corpus (no LLM) uv run --extra wiki python examples/enrich_corpus.py
History graph — curated uv run python examples/hungarian_history.py
History graph — live from Wikipedia uv run --extra wiki python examples/hungarian_history_live.py
Serve over MCP (SSE) uv run --extra mcp python mcp_server/server.py
Dev container open in VS Code → Reopen in Container

Ternary language models on real corpora

Two byte-level ternary GPTs trained end-to-end on public-domain text; both deploy to tiny bit-packed checkpoints that run from the trits alone:

  • Latin — examples/anonymus_lm.py on the Gesta Hungarorum of Anonymus (c. 1200), the Hungarian founding chronicle (~94 KB) → 196 KB checkpoint. Sample: GPT.load_deployed("examples/data/anonymus.gpt.npz") → "Almus dux … dux cum patis se … terras suis …".
  • Hungarian — examples/hungarian_lm.py on ~450 KB of public-domain Hungarian literature (Arany János + prose, pulled from Project Gutenberg by fetch_hungarian.py). Training is resumable — it saves the fp32 masters + step state and continues across runs (TOTAL/STEPS env vars, periodic checkpoints), so it converges past any wall-clock cap.
from ultragraph import GPT, ByteTokenizer
tok = ByteTokenizer()
m = GPT.load_deployed("examples/data/hungarian.gpt.npz")
print(tok.decode(m.generate(tok.encode("A magyar "), n_new=90, temperature=0.8, top_p=0.9)))

Corpus tooling — examples/enrich_corpus.py is a non-LLM gatherer: it pulls grounded facts from Hungarian Wikipedia (via ultragraph.wiki), turns them into definition + Kérdés:/Válasz: lines, dedups against the corpus, and appends (idempotent, re-runnable).

Knowledge graphs — curated & live

The byte-graph data model doubles as a knowledge graph: entities are Tree nodes, relations are micro-edges, and higher-level structure is ultra-edges (===).

  • Curated — examples/hungarian_history.py builds Hungarian history as one ultra-graph (13 eras, 59 nodes) and renders a themed timeline SVG.
  • Live — examples/hungarian_history_live.py builds it dynamically from hu.wikipedia via ultragraph.wiki.build_wiki_graph, scraping pages + links (with an on-disk cache) into a sparse Tree.

MCP server

mcp_server/server.py exposes the library over MCP (SSE transport) with tools anonymus_generate, ultragraph_info, and tokenize_preview:

uv run --extra mcp python mcp_server/server.py   # -> http://127.0.0.1:8000/sse

BMad agents

Three BMad agents live under skills/ to help develop and use the library:

Agent Folder What it does
ByteSmith 🔬 skills/ultragraph-dev/ Byte-graph coding agent — writes autograd ops, wires trees, trains models, explains internals
CorpusCrafter 📚 skills/corpus-trainer/ Automates corpus gathering, Wikipedia enrichment, and byte-level GPT training for any language
GraphViz 📊 skills/viz-doc/ Renders SVG/PNG model visualizations and generates model card reports from checkpoints

Each is a stateless skill (SKILL.md + capability references + customize.toml) activated by describing the task.

Tasks

just test        # pytest
just test-fast   # dependency-free runner (stdlib + numpy)
just demo        # char-LM end-to-end
just viz         # render example SVGs

Deterministic kernels — zerocpu

ultragraph.zerocpu is the integer-only half of the library: no float appears in any deterministic path, so the same input produces the same bytes on every machine.

from ultragraph.zerocpu import SpacetimeGraph, parse_pq, apply_pq, emit_ugm, load_ugm, partition

g = SpacetimeGraph()
a, b, c = g.add_node(1, (0, 0, 0)), g.add_node(0, (1 << 16, 0, 0)), g.add_node(0, (2 << 16, 0, 0))
g.add_edge(a, b, +1); g.add_edge(b, c, +1); g.add_edge(c, a, -1)
g.tick(3)                                   # state' = clamp8(state + sum(w * src)), ints only
apply_pq(g, "edge 1 0 +1\nset 2 -7 q 1/2", seed=7)   # exact-rational acceptance, reproducible
p = partition(3, [(0, 1), (1, 2), (2, 0)], k=1)
module role
zerocpu/fixed.py Q16.16 arithmetic, round-half-away-from-zero, symmetric about zero
zerocpu/spacetime.py SpacetimeGraph: ternary causal edges with dt, tick/untick history, light cones, sha256 fingerprint
zerocpu/evolve.py p/q prompt language (edge, flip, prune, set, step, wire) with exact-rational acceptance
zerocpu/emit.py UGMZ artifact v1 — 32-byte header, 24-byte node/edge records, little-endian, byte-stable
zerocpu/vaked_bridge.py lowered .vaked workflow → world → .ugmz (set VAKED_ROOT to a working vakedc)
zerocpu/lod.py quasi-regular partitioning: maximises structure, reports residual defect instead of claiming regularity
zerocpu/pqq.py compiled .pqq programs: rational draws settled once at compile time, replayed with no PRNG
zerocpu/hope.py text → integer heat per lane → seeded world → tick → observer collapse; no floats, no keyword tables
zerocpu/trit.py Q-delta ternary quantizer, A6P-15A bitplanes, and the SWAR packed-ternary dot (eight lanes per word, no multiply)
(was above) Q-delta ternary quantizer (A6P-15A) + 16-plane bit layout that pops straight into NEON; quantize, bitplanes, plane_parity
zerocpu/router.py integer route selection — kernel / batch / model / plugin / deny — from position-weighted heat; explain() shows the counts, not the mood
zerocpu/meshd.py the socket path around mesh.py: MeshListener (loopback by default), announce(), and a python -m CLI — the Python twin of bin/peers.rs
zerocpu/memnet.py sequence-ordered associative memory ((seq, value) beats wall-clock): commutative, associative, idempotent merge, plus RingMemNet — a fixed 64-slot ring with decaying amplitudes and anchors that never fade
zerocpu/mesh.py LAN peer announce frame (UGRD v1, 12-byte fixed header + 32-byte name) and its byte-stable merge
zerocpu/observer.py local clocks ((tick - origin) * rate >> 16), bounded integer decay, the closed-form Padé e^-x, the Mahakala guard, and anchored memory that never fades
rust/zerocpu/ the same kernels in Rust (NEON on aarch64) + world (tick a .ugmz), replay (apply a .pqq) and peers (UDP announce) binaries

Parity is pinned by golden fixtures, so the Python and Rust halves cannot drift apart silently: cargo test replays the committed .ugmz artifacts and LOD partitions that the Python suite regenerates.

Two nodes can read the same world tick differently and still agree on the bytes. A frame is an explicit mapping, not a metaphor:

from ultragraph.zerocpu import ObserverFrame, Witness, relativity

hearth = ObserverFrame("hearth", rate_q16=1 << 15)   # half speed
world_ = ObserverFrame("thunderbird")                 # keeps world time
relativity(hearth, world_, 3)                         # local_a=1 local_b=3 delta=2

memory = Witness()
memory.remember("the stream", 0, span=25)
memory.remember("mumtaz mahal", 0, span=1, anchor=True)
memory.variance("the stream", 40)                     # 22155 -- faded
memory.variance("mumtaz mahal", 40)                   # 6553600 -- untouched

Decay is v * 62339 >> 16 per tick, capped at 4096 ticks, so it cannot loop forever and cannot drift between platforms. Ordering disagreement needs a pair no signal can connect -- ask SpacetimeGraph.lightcone, not a clock.

Everything composes into one runnable transcript — fit a prompt, quantize it, tick a world, remember it, and discover the LAN — with no dependencies beyond the stdlib:

$ python3 examples/caveman_ultra_demo.py

| zerocpu/*.py | Rust twin under rust/zerocpu/src/ |

A text can also be projected straight onto the world. There is no separate cascade engine: the projection sets ternary states and propagation is the ordinary tick, so the resonance is a state you can replay, not a narration.

from ultragraph.zerocpu import load_ugm, resonate

r = resonate(load_ugm("world.ugmz"), "Difference is everywhere.", rounds=2)
r.signal.trits      # (-1, 1, 1, -1)  -- lanes above/below the text's own mean
r.collapse          # +1, 0 or -1: the observer's verdict on the settled state

# Ternary Kleene guard + closed-form decay + a ring memory
tnand(1, 1)                     # -1: two affirmations cannot both hold
tnand(-1, 1)                    # +1: one denial releases the gate
pade_exp_neg(65536)             # 24109 = e^-1 in Q16.16, no exp() anywhere

ring = RingMemNet()
ring.record(3 << 16, 0, anchor=True)
ring.record(1 << 16, 0)
ring.variance(4000, 10, 65536)  # 196608: only the anchor is left
ring.consensus(4000, 10, 65536) # +1: love survives the half-lives
r.trace             # ('0101ffff', '010200ff', '01030101') -- every round, in hex

A p/q prompt can be frozen into a .pqq program that any interpreter replays without a random source — the draw is a compile-time choice, the artifact records it:

from ultragraph.zerocpu import resolve, dump_pqq, write_bundle

write_bundle(g, "diamond", "step 2\nedge 1 3 +1\nset 2 -9 q 1/2", seed=7)  # -> diamond.ugmz + diamond.pqq
print(dump_pqq(resolve("step 3\nedge 0 1 +1", seed=7)))                    # auditable, seed-free text
cargo run --manifest-path rust/zerocpu/Cargo.toml --bin world  -- rust/zerocpu/tests/fixtures/world.ugmz 3
cargo run --manifest-path rust/zerocpu/Cargo.toml --bin replay -- rust/zerocpu/tests/fixtures/vaked_diamond.ugmz \
    rust/zerocpu/tests/fixtures/replay.pqq                                          # state=0e1c387f tick=3 edges=5

Published kernels on g8t.is

Capsules are measured locally (cntvct_el0) and signed; cycles/op is an exact rational, never a float.

capsule signature cycles/op FKN id
round_div_q16 (i64,i64)->i64 1709533/200000 500 24190df2f99d90771ffcd00281129dee
pack5_trits_u8 (i8,i8,i8,i8,i8)->u8 2887733/400000 500 13ddc68f8edc93e28ac25d96efb8b7fe
saturate_i8_i64 (i64)->i64 1324001/400000 500 78c8596dbdfa3ec1d99c44457c7a4a16
trit_at_u8 (u8,u32)->i8 953933/200000 500 325d74c8ecd642a59eb870c7a7ef3607
saturate_trit_i64 (i64)->i8 1323467/400000 500 9bb078d59c2817acb28eb94ad624c0f6
fx_sqrt_q16 (i64)->i64 6471/500 400 46fe68ba0b3218784ddec6a6647bf96d

saturate_trit_i64 compiles to five instructions (test/sets/setg/sub/ret), so it is the cheapest ternary projection on the board; fx_sqrt_q16 is 154, all integer isqrt scaffolding, because exactness costs real work.

A submission must terminate for any input the harness draws: the first trit_at_u8 was rejected (2000 ms timeout) because it looped index times instead of reading a fixed lane table. Bounded-by-construction is part of the contract.

Layout

ultragraph/quant.py     ternary + int8 quantization, STE
ultragraph/autograd.py  numpy autograd tape; ternary_linear (STE); exp/tanh/sigmoid/gelu/silu
ultragraph/core.py      Node/Edge/Tree/UltraEdge/UltraGraph + dunder API
ultragraph/nn.py        linear_tree, mlp, Attention, MultiHeadAttention, RoPE, RMSNorm, LayerNorm, LearnedPositionalEmbedding, MoE, Dropout, Sequential
ultragraph/model.py     TransformerBlock + GPT (RoPE + KV-cache + .generate + save_deployed) + Mesh (mixture of full models)
ultragraph/optim.py     SGD + Adam (grad clip, weight decay, gradient accumulation) + CosineSchedule
ultragraph/pack.py      dense ternary bit-packing (5 values/byte, ~1.58-bit)
ultragraph/tokenize.py  byte-level tokenizer (ByteTokenizer, vocab 256)
ultragraph/vaked.py      optional vaked lowering (lower_graph, compile_vaked via vendored vakedc)
ultragraph/zerocpu/     integer-only kernels: fixed / spacetime / evolve / emit / vaked_bridge / lod
rust/zerocpu/           Rust twin of the same kernels (NEON on aarch64) + UGMZ `world` ticker
ultragraph/viz/         svg.py (pure-SVG) + mpl.py (optional matplotlib) — micro / macro / byte-heatmap
ultragraph/io.py        byte-exact save / load (optional packed weights); save_params/load_params
ultragraph/wiki.py      optional MediaWiki client + build_wiki_graph (live hu.wikipedia -> ultra-graph)
mcp_server/server.py    optional MCP server (SSE) exposing the library as tools
examples/               char/gpt/mesh/anonymus/hungarian LMs, history graphs, corpus fetch + enrich
.devcontainer/          VS Code dev container (Python 3.14 + uv + ruff)

The development loop is written down in docs/ULTRABACKYARDLOOP.md — reflect → improve → wire → polish → push → readme, with the gate a lap must pass and the two standing measurements (world_vs_tick, variance_slots). One machine-readable line per lap lives in docs/lap-ledger.jsonl.

Design spec: docs/superpowers/specs/2026-07-10-ultragraph-design.md. Graph-theory reading list (Erdős classics): docs/references.md.

Install from source

git clone https://codeberg.org/8b-is/ultra-graph   # canonical; mirrors: github.com/8b-is/ultra-graph, github.com/peterlodri-sec/ultra-graph
cd ultra-graph
uv sync
just test

License

MIT — see LICENSE.