- Python 81.9%
- Rust 16.7%
- CMake 0.8%
- C++ 0.4%
- Shell 0.1%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
| .claude | ||
| .devcontainer | ||
| .github/workflows | ||
| .obsidian | ||
| _bmad | ||
| assets | ||
| docs | ||
| examples | ||
| mcp_server | ||
| rust/zerocpu | ||
| skills | ||
| tests | ||
| ultragraph | ||
| vakedc | ||
| wiki | ||
| .editorconfig | ||
| .gitignore | ||
| .ignore | ||
| AGENTS.md | ||
| CHANGELOG.md | ||
| CONTRIBUTING.md | ||
| justfile | ||
| LICENSE | ||
| pyproject.toml | ||
| README.md | ||
| SETUP_SUMMARY.md | ||
| uv.lock | ||
| WIKI.md | ||
ultragraph
A pure-Python (+ numpy) byte-graph that is a 1-bit (ternary) LLM.
genesis
251e6ea· themed after pocoo.vaked.dev
Three levels:
| level | unit | storage |
|---|---|---|
| micro | node / edge | 1 byte each — int8 activation / ternary weight {-1,0,+1} |
| meso | tree | a whole graph == one net/module (a Linear/MLP block) |
| macro | ultra-edge (===) |
typed wiring between trees → the ultra-graph = the model |
Weights are ternary (BitNet b1.58 style); activations are int8. Full-precision "master" weights live in an ad-hoc side store during training; the byte buffers are the deployed state. Training uses a straight-through estimator (STE).
Illustrations
Real outputs from a trained ternary mini-GPT — regenerate with uv run python assets/make_figures.py:
| ultra-graph | causal attention | ternary weight bytes |
|---|---|---|
![]() |
![]() |
![]() |
Left: the model as an ultra-graph — trees wired by ultra-edges (===), with residual skips.
Middle: real causal self-attention weights (lower-triangular → no peeking at the future).
Right: a trained query projection's weight bytes, each ∈ {−1, 0, +1}.
Install
pip install ultragraph-1bit # then: import ultragraph
# or from source (Python >=3.13, free-threading ready):
uv sync
Dunder API
>> is overloaded by operand type:
import numpy as np
from ultragraph import Tree, UltraGraph, Tensor, mlp, SGD
# micro-edges inside a sparse tree
g = Tree(4, "g")
g[0] >> g[1] # node >> node -> micro-edge
g[2] = 7 # set a node byte
print(len(g), 2 in g, list(g))
# ultra-edges between trees
ug = UltraGraph()
a = ug.add(Tree.dense(8, 16, "a"))
b = ug.add(Tree.dense(16, 4, "b", act="none"))
a >> b # tree >> tree -> ultra-edge (plain)
a.wire(b, "residual")
Train a tiny ternary net
ug = mlp([4, 16, 2]) # dense ternary linear trees wired plain
opt = SGD(ug, lr=0.3, momentum=0.9)
x = Tensor(np.random.randn(32, 4).astype("float32"))
for _ in range(300):
loss = ug.forward(x).cross_entropy(y)
opt.zero_grad(); loss.backward(); opt.step() # step() re-quantizes weights
See examples/char_lm.py (MLP LM), examples/transformer_lm.py (single-head attention),
examples/mini_gpt.py (batched multi-head attention + RMSNorm + Adam),
examples/gpt_lm.py (the whole stack: ByteTokenizer → GPT → train → stream), and
examples/mesh_lm.py (a Mesh of GPT experts, gradient accumulation, joint decode)
for end-to-end char/byte-level ternary language models.
from ultragraph import Embedding, MultiHeadAttention, RMSNorm, linear_tree, Adam
# pre-norm transformer block over a [B, T, d_model] sequence:
# x = x + mha(norm1(x)); x = x + ff2(ff1(norm2(x)))
A whole ternary GPT
from ultragraph import GPT
m = GPT(vocab=256, d_model=128, n_layers=4, n_heads=4, max_len=256) # RoPE + KV-cache
logits = m(ids) # ids [B, T] -> logits [B, T, vocab]
out = m.generate([72, 105], n_new=64, temperature=0.8, top_k=40, top_p=0.9,
repetition_penalty=1.3, stop=10, seed=0) # stop on newline byte
for tok in m.generate([72, 105], n_new=64, temperature=0.8, stream=True):
print(tok, end=" ", flush=True) # token-by-token
m.save("gpt.npz") # fp32 masters; reload onto the same architecture
# a true 1-bit-on-disk checkpoint: bit-packed ternary bytes, no fp32 masters.
m.save_deployed("gpt.q.npz") # ~10x smaller, inference-only
deployed = GPT.load_deployed("gpt.q.npz") # byte-exact logits, runs from the trits
The deployed checkpoint stores weights at their true ~1.6 bits/weight density (5
ternary values per byte) plus the tiny fp32 pieces (embedding, norm gains, biases).
On an 858k-param model that's 3.4 MB → 334 KB, and deployed(ids) gives logits
identical to the trained model — Tree.forward runs straight from the stored bytes.
A mesh of minds
from ultragraph import GPT, Mesh
experts = [GPT(vocab=256, d_model=64, n_layers=2, n_heads=4) for _ in range(4)]
mesh = Mesh(experts, vocab=256, top_k=2) # a learned router mixes full models
logits = mesh(ids) # Σ_e gate(ids)_e · expert_e(ids)
text = mesh.generate([72, 105], n_new=64, temperature=0.8) # joint KV-cached decode
Mesh lifts nn.MoE's routing to whole networks: a small ternary router reads the
sequence and mixes the experts' logits per sequence (soft, or top-k). Router and every
expert train together — a graph of minds, still all ternary bytes underneath.
generate decodes with a per-layer KV-cache; since activations are quantized
per token, a cached step is byte-for-byte the full-forward result at that position.
Positions come from RoPE (rotary embeddings) — relative, and offset-aware so
they line up across cached steps.
Guide — common recipes
| Goal | Command |
|---|---|
| Install | pip install ultragraph-1bit (extras: viz, mcp, wiki) |
| Train a byte-level GPT | uv run python examples/gpt_lm.py |
| Mixture of full models | uv run python examples/mesh_lm.py |
| 1-bit Latin LLM (Anonymus) | fetch_gesta.py → anonymus_lm.py |
| 1-bit Hungarian LLM (resumable) | fetch_hungarian.py → hungarian_lm.py |
| Enrich the corpus (no LLM) | uv run --extra wiki python examples/enrich_corpus.py |
| History graph — curated | uv run python examples/hungarian_history.py |
| History graph — live from Wikipedia | uv run --extra wiki python examples/hungarian_history_live.py |
| Serve over MCP (SSE) | uv run --extra mcp python mcp_server/server.py |
| Dev container | open in VS Code → Reopen in Container |
Ternary language models on real corpora
Two byte-level ternary GPTs trained end-to-end on public-domain text; both deploy to
tiny bit-packed checkpoints that run from the trits alone:
- Latin —
examples/anonymus_lm.pyon the Gesta Hungarorum of Anonymus (c. 1200), the Hungarian founding chronicle (~94 KB) → 196 KB checkpoint. Sample:GPT.load_deployed("examples/data/anonymus.gpt.npz")→ "Almus dux … dux cum patis se … terras suis …". - Hungarian —
examples/hungarian_lm.pyon ~450 KB of public-domain Hungarian literature (Arany János + prose, pulled from Project Gutenberg byfetch_hungarian.py). Training is resumable — it saves the fp32 masters + step state and continues across runs (TOTAL/STEPSenv vars, periodic checkpoints), so it converges past any wall-clock cap.
from ultragraph import GPT, ByteTokenizer
tok = ByteTokenizer()
m = GPT.load_deployed("examples/data/hungarian.gpt.npz")
print(tok.decode(m.generate(tok.encode("A magyar "), n_new=90, temperature=0.8, top_p=0.9)))
Corpus tooling — examples/enrich_corpus.py is a non-LLM gatherer: it pulls
grounded facts from Hungarian Wikipedia (via ultragraph.wiki), turns them into
definition + Kérdés:/Válasz: lines, dedups against the corpus, and appends
(idempotent, re-runnable).
Knowledge graphs — curated & live
The byte-graph data model doubles as a knowledge graph: entities are Tree nodes,
relations are micro-edges, and higher-level structure is ultra-edges (===).
- Curated —
examples/hungarian_history.pybuilds Hungarian history as one ultra-graph (13 eras, 59 nodes) and renders a themed timeline SVG. - Live —
examples/hungarian_history_live.pybuilds it dynamically from hu.wikipedia viaultragraph.wiki.build_wiki_graph, scraping pages + links (with an on-disk cache) into a sparseTree.
MCP server
mcp_server/server.py exposes the library over MCP (SSE transport) with tools
anonymus_generate, ultragraph_info, and tokenize_preview:
uv run --extra mcp python mcp_server/server.py # -> http://127.0.0.1:8000/sse
BMad agents
Three BMad agents live under skills/ to help develop and use the library:
| Agent | Folder | What it does |
|---|---|---|
| ByteSmith 🔬 | skills/ultragraph-dev/ |
Byte-graph coding agent — writes autograd ops, wires trees, trains models, explains internals |
| CorpusCrafter 📚 | skills/corpus-trainer/ |
Automates corpus gathering, Wikipedia enrichment, and byte-level GPT training for any language |
| GraphViz 📊 | skills/viz-doc/ |
Renders SVG/PNG model visualizations and generates model card reports from checkpoints |
Each is a stateless skill (SKILL.md + capability references + customize.toml) activated by describing the task.
Tasks
just test # pytest
just test-fast # dependency-free runner (stdlib + numpy)
just demo # char-LM end-to-end
just viz # render example SVGs
Deterministic kernels — zerocpu
ultragraph.zerocpu is the integer-only half of the library: no float appears in any
deterministic path, so the same input produces the same bytes on every machine.
from ultragraph.zerocpu import SpacetimeGraph, parse_pq, apply_pq, emit_ugm, load_ugm, partition
g = SpacetimeGraph()
a, b, c = g.add_node(1, (0, 0, 0)), g.add_node(0, (1 << 16, 0, 0)), g.add_node(0, (2 << 16, 0, 0))
g.add_edge(a, b, +1); g.add_edge(b, c, +1); g.add_edge(c, a, -1)
g.tick(3) # state' = clamp8(state + sum(w * src)), ints only
apply_pq(g, "edge 1 0 +1\nset 2 -7 q 1/2", seed=7) # exact-rational acceptance, reproducible
p = partition(3, [(0, 1), (1, 2), (2, 0)], k=1)
| module | role |
|---|---|
zerocpu/fixed.py |
Q16.16 arithmetic, round-half-away-from-zero, symmetric about zero |
zerocpu/spacetime.py |
SpacetimeGraph: ternary causal edges with dt, tick/untick history, light cones, sha256 fingerprint |
zerocpu/evolve.py |
p/q prompt language (edge, flip, prune, set, step, wire) with exact-rational acceptance |
zerocpu/emit.py |
UGMZ artifact v1 — 32-byte header, 24-byte node/edge records, little-endian, byte-stable |
zerocpu/vaked_bridge.py |
lowered .vaked workflow → world → .ugmz (set VAKED_ROOT to a working vakedc) |
zerocpu/lod.py |
quasi-regular partitioning: maximises structure, reports residual defect instead of claiming regularity |
zerocpu/pqq.py |
compiled .pqq programs: rational draws settled once at compile time, replayed with no PRNG |
zerocpu/hope.py |
text → integer heat per lane → seeded world → tick → observer collapse; no floats, no keyword tables |
zerocpu/trit.py |
Q-delta ternary quantizer, A6P-15A bitplanes, and the SWAR packed-ternary dot (eight lanes per word, no multiply) |
| (was above) | Q-delta ternary quantizer (A6P-15A) + 16-plane bit layout that pops straight into NEON; quantize, bitplanes, plane_parity |
zerocpu/router.py |
integer route selection — kernel / batch / model / plugin / deny — from position-weighted heat; explain() shows the counts, not the mood |
zerocpu/meshd.py |
the socket path around mesh.py: MeshListener (loopback by default), announce(), and a python -m CLI — the Python twin of bin/peers.rs |
zerocpu/memnet.py |
sequence-ordered associative memory ((seq, value) beats wall-clock): commutative, associative, idempotent merge, plus RingMemNet — a fixed 64-slot ring with decaying amplitudes and anchors that never fade |
zerocpu/mesh.py |
LAN peer announce frame (UGRD v1, 12-byte fixed header + 32-byte name) and its byte-stable merge |
zerocpu/observer.py |
local clocks ((tick - origin) * rate >> 16), bounded integer decay, the closed-form Padé e^-x, the Mahakala guard, and anchored memory that never fades |
rust/zerocpu/ |
the same kernels in Rust (NEON on aarch64) + world (tick a .ugmz), replay (apply a .pqq) and peers (UDP announce) binaries |
Parity is pinned by golden fixtures, so the Python and Rust halves cannot drift apart
silently: cargo test replays the committed .ugmz artifacts and LOD partitions that
the Python suite regenerates.
Two nodes can read the same world tick differently and still agree on the bytes. A frame is an explicit mapping, not a metaphor:
from ultragraph.zerocpu import ObserverFrame, Witness, relativity
hearth = ObserverFrame("hearth", rate_q16=1 << 15) # half speed
world_ = ObserverFrame("thunderbird") # keeps world time
relativity(hearth, world_, 3) # local_a=1 local_b=3 delta=2
memory = Witness()
memory.remember("the stream", 0, span=25)
memory.remember("mumtaz mahal", 0, span=1, anchor=True)
memory.variance("the stream", 40) # 22155 -- faded
memory.variance("mumtaz mahal", 40) # 6553600 -- untouched
Decay is v * 62339 >> 16 per tick, capped at 4096 ticks, so it cannot loop
forever and cannot drift between platforms. Ordering disagreement needs a pair
no signal can connect -- ask SpacetimeGraph.lightcone, not a clock.
Everything composes into one runnable transcript — fit a prompt, quantize it, tick a world, remember it, and discover the LAN — with no dependencies beyond the stdlib:
$ python3 examples/caveman_ultra_demo.py
| zerocpu/*.py | Rust twin under rust/zerocpu/src/ |
A text can also be projected straight onto the world. There is no separate
cascade engine: the projection sets ternary states and propagation is the ordinary
tick, so the resonance is a state you can replay, not a narration.
from ultragraph.zerocpu import load_ugm, resonate
r = resonate(load_ugm("world.ugmz"), "Difference is everywhere.", rounds=2)
r.signal.trits # (-1, 1, 1, -1) -- lanes above/below the text's own mean
r.collapse # +1, 0 or -1: the observer's verdict on the settled state
# Ternary Kleene guard + closed-form decay + a ring memory
tnand(1, 1) # -1: two affirmations cannot both hold
tnand(-1, 1) # +1: one denial releases the gate
pade_exp_neg(65536) # 24109 = e^-1 in Q16.16, no exp() anywhere
ring = RingMemNet()
ring.record(3 << 16, 0, anchor=True)
ring.record(1 << 16, 0)
ring.variance(4000, 10, 65536) # 196608: only the anchor is left
ring.consensus(4000, 10, 65536) # +1: love survives the half-lives
r.trace # ('0101ffff', '010200ff', '01030101') -- every round, in hex
A p/q prompt can be frozen into a .pqq program that any interpreter replays
without a random source — the draw is a compile-time choice, the artifact records it:
from ultragraph.zerocpu import resolve, dump_pqq, write_bundle
write_bundle(g, "diamond", "step 2\nedge 1 3 +1\nset 2 -9 q 1/2", seed=7) # -> diamond.ugmz + diamond.pqq
print(dump_pqq(resolve("step 3\nedge 0 1 +1", seed=7))) # auditable, seed-free text
cargo run --manifest-path rust/zerocpu/Cargo.toml --bin world -- rust/zerocpu/tests/fixtures/world.ugmz 3
cargo run --manifest-path rust/zerocpu/Cargo.toml --bin replay -- rust/zerocpu/tests/fixtures/vaked_diamond.ugmz \
rust/zerocpu/tests/fixtures/replay.pqq # state=0e1c387f tick=3 edges=5
Published kernels on g8t.is
Capsules are measured locally (cntvct_el0) and signed; cycles/op is an exact
rational, never a float.
| capsule | signature | cycles/op | FKN | id |
|---|---|---|---|---|
round_div_q16 |
(i64,i64)->i64 |
1709533/200000 | 500 | 24190df2f99d90771ffcd00281129dee |
pack5_trits_u8 |
(i8,i8,i8,i8,i8)->u8 |
2887733/400000 | 500 | 13ddc68f8edc93e28ac25d96efb8b7fe |
saturate_i8_i64 |
(i64)->i64 |
1324001/400000 | 500 | 78c8596dbdfa3ec1d99c44457c7a4a16 |
trit_at_u8 |
(u8,u32)->i8 |
953933/200000 | 500 | 325d74c8ecd642a59eb870c7a7ef3607 |
saturate_trit_i64 |
(i64)->i8 |
1323467/400000 | 500 | 9bb078d59c2817acb28eb94ad624c0f6 |
fx_sqrt_q16 |
(i64)->i64 |
6471/500 | 400 | 46fe68ba0b3218784ddec6a6647bf96d |
saturate_trit_i64 compiles to five instructions (test/sets/setg/sub/ret),
so it is the cheapest ternary projection on the board; fx_sqrt_q16 is 154, all
integer isqrt scaffolding, because exactness costs real work.
A submission must terminate for any input the harness draws: the first trit_at_u8
was rejected (2000 ms timeout) because it looped index times instead of reading a
fixed lane table. Bounded-by-construction is part of the contract.
Layout
ultragraph/quant.py ternary + int8 quantization, STE
ultragraph/autograd.py numpy autograd tape; ternary_linear (STE); exp/tanh/sigmoid/gelu/silu
ultragraph/core.py Node/Edge/Tree/UltraEdge/UltraGraph + dunder API
ultragraph/nn.py linear_tree, mlp, Attention, MultiHeadAttention, RoPE, RMSNorm, LayerNorm, LearnedPositionalEmbedding, MoE, Dropout, Sequential
ultragraph/model.py TransformerBlock + GPT (RoPE + KV-cache + .generate + save_deployed) + Mesh (mixture of full models)
ultragraph/optim.py SGD + Adam (grad clip, weight decay, gradient accumulation) + CosineSchedule
ultragraph/pack.py dense ternary bit-packing (5 values/byte, ~1.58-bit)
ultragraph/tokenize.py byte-level tokenizer (ByteTokenizer, vocab 256)
ultragraph/vaked.py optional vaked lowering (lower_graph, compile_vaked via vendored vakedc)
ultragraph/zerocpu/ integer-only kernels: fixed / spacetime / evolve / emit / vaked_bridge / lod
rust/zerocpu/ Rust twin of the same kernels (NEON on aarch64) + UGMZ `world` ticker
ultragraph/viz/ svg.py (pure-SVG) + mpl.py (optional matplotlib) — micro / macro / byte-heatmap
ultragraph/io.py byte-exact save / load (optional packed weights); save_params/load_params
ultragraph/wiki.py optional MediaWiki client + build_wiki_graph (live hu.wikipedia -> ultra-graph)
mcp_server/server.py optional MCP server (SSE) exposing the library as tools
examples/ char/gpt/mesh/anonymus/hungarian LMs, history graphs, corpus fetch + enrich
.devcontainer/ VS Code dev container (Python 3.14 + uv + ruff)
The development loop is written down in
docs/ULTRABACKYARDLOOP.md — reflect → improve →
wire → polish → push → readme, with the gate a lap must pass and the two
standing measurements (world_vs_tick, variance_slots). One machine-readable
line per lap lives in docs/lap-ledger.jsonl.
Design spec: docs/superpowers/specs/2026-07-10-ultragraph-design.md.
Graph-theory reading list (Erdős classics): docs/references.md.
Install from source
git clone https://codeberg.org/8b-is/ultra-graph # canonical; mirrors: github.com/8b-is/ultra-graph, github.com/peterlodri-sec/ultra-graph
cd ultra-graph
uv sync
just test
License
MIT — see LICENSE.



