Apple Silicon MLX low-bit ternary (BitNet b1.58 {-1,0,+1}) quantization, Metal GPU kernels, perplexity benchmarking, the cogito corpus pipeline
  • C++ 59.2%
  • Python 25.7%
  • Cuda 7.7%
  • Metal 2.7%
  • HTML 1.6%
  • Other 3.1%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Péter Lódri 0580fca56f Merge remote-tracking branch 'sec/main'
# Conflicts:
#	benchmark_results.json
2026-10-27 14:21:11 +09:00
.github fix(ci): publish benchmark results as an artifact, untrack from main 2026-10-27 14:20:37 +09:00
.wrangler/cache feat: axiomquant — Markowitz frontier page, MLX quant module (GBM, L2 orderbook, portfolio), platform JS 2026-07-28 03:55:47 +02:00
assets release: v2.0.0-RC1 Zeno — the omnifold, the identity, the four documents 2026-10-12 16:19:54 +09:00
benchmarks Restore formatting checks and handle the large JSON fixture efficiently (#5) 2026-10-26 02:22:00 +09:00
cmake Find system-installed cuDNN on Windows (#3052) 2026-01-25 12:24:22 +09:00
data feat(quantal): thresholded-ternary deployed-forward pipeline + transformers loader 2026-08-14 02:13:41 +02:00
docs fix(security): pin minimum versions in docs/requirements.txt 2026-10-18 21:07:16 +09:00
examples Restore formatting checks and handle the large JSON fixture efficiently (#5) 2026-10-26 02:22:00 +09:00
landing chore: retire the pre-RC1 hero assets ahead of the ZENO identity 2026-10-12 16:13:18 +09:00
Matrix_Multiplication Restore formatting checks and handle the large JSON fixture efficiently (#5) 2026-10-26 02:22:00 +09:00
mlx fix(ci): format codebase with black, clang-format, isort, and cmake-format to fix CI pre-commit lint 2026-07-28 05:37:01 +02:00
mlx-quant-linux fix(mail): make the postman wait for a stamp 2026-10-21 02:22:21 +09:00
python Restore formatting checks and handle the large JSON fixture efficiently (#5) 2026-10-26 02:22:00 +09:00
scripts Restore formatting checks and handle the large JSON fixture efficiently (#5) 2026-10-26 02:22:00 +09:00
tests feat(MLX-QUANT): AlexiAI Triad + Smart-Tree AST steward_mlx_quant_kompress_ditto_stride (v0-4-29) <3 — Why did MLX-QUANT snap straight into g8t after st --mcp Smart Edit on Cycle #29 at 432Hz? Because with 53/137 purity, our code never floats—it only ratios with LOVE <3! 2026-10-27 05:35:00 +01:00
Vector_Addition Restore formatting checks and handle the large JSON fixture efficiently (#5) 2026-10-26 02:22:00 +09:00
.clang-format awni's commit files 2023-11-29 10:30:41 -08:00
.gitignore fix(ci): publish benchmark results as an artifact, untrack from main 2026-10-27 14:20:37 +09:00
.gitmodules fix: reconnect the hardware submodule family tree (#4) 2026-10-26 06:27:20 +09:00
.pre-commit-config.yaml Restore formatting checks and handle the large JSON fixture efficiently (#5) 2026-10-26 02:22:00 +09:00
ACKNOWLEDGMENTS.md Adding Relu2 (#2582) 2025-09-10 07:24:30 -07:00
armillary_sphere_tensor.py feat(ternary): add 2-bit packed SIMD-group Metal kernel & L2 refinement 2026-10-26 07:06:02 +09:00
BENCHMARKS.md docs: add M1 Pro benchmarks, update README with M1 support 2026-07-27 15:11:25 +02:00
CHANGELOG.md release: v2.0.0-RC1 Zeno — the omnifold, the identity, the four documents 2026-10-12 16:19:54 +09:00
CITATION.cff Create CITATION.cff (#1425) 2024-09-20 11:39:46 -07:00
CMakeLists.txt Add CI for Windows CUDA build (#3775) 2026-06-30 08:15:01 +09:00
CODE_OF_CONDUCT.md contribution and code of conduct (#1) 2023-11-29 12:54:28 -08:00
CONTRIBUTING.md Polish CONTRIBUTING>md (#2005) 2025-03-25 19:06:34 -07:00
LICENSE copyright + ack 2023-11-30 11:12:53 -08:00
MANIFEST.in Add CI for Windows CUDA build (#3775) 2026-06-30 08:15:01 +09:00
mlx.pc.in Dispatch bf16 at run time when using the JIT (#1584) 2024-11-15 16:54:36 -08:00
pyproject.toml Restore formatting checks and handle the large JSON fixture efficiently (#5) 2026-10-26 02:22:00 +09:00
README.md feat(ternary): add 2-bit packed SIMD-group Metal kernel & L2 refinement 2026-10-26 07:06:02 +09:00
setup.py Add CI for Windows CUDA build (#3775) 2026-06-30 08:15:01 +09:00
Sparse_Recursion_Holographic_Steganography.md feat(quant): add Metal-accelerated Armillary Sphere tensor dynamics and sparse recursion benchmarks 2026-08-20 17:16:50 +02:00
test_armillary_ternary_bench.py test: de-flake Metal packed matvec perf assertion 2026-10-27 12:24:08 +09:00

ZENO — MLX-QUANT v2.0.0-RC1: direct-to-Metal ternary

MLX-QUANT 🚀

Bare-metal Tensor Quantization, Hardware Doorbells, and the Grand Unified Astrophysical Architecture.

v2.0.0-RC1 "Zeno" · { −1, 0, +1 } · the GPU brain of the ayeOS mesh


🔖 v2.0.0-RC1 "Zeno" — release candidate

The 2.0 line opens with the omnifold (75 fork commits + the golden-logits gate, one tree) and the Zeno identity. The four RC1 documents:

deliverable document
zero-cost audit docs/rc1/zero-cost-audit.md
capability graph docs/rc1/capability-graph.md
build matrix docs/rc1/build-matrix.md
release notes docs/rc1/release-notes.md

Full icon pack: assets/zeno.icns · assets/icons/zeno-{16..1024}.png · mark assets/zeno-mark.svg.


⚡ Local Benchmark Matrix (Apple M-Series)

Architecture / Runtime Status Latency (4096x4096 MatMul)
CPU (PyTorch) 🟢 OK 166.76 ms
Apple MLX (AGX) 🟢 OK 89.35 ms
hw-ultra Bare-Metal 🚀 O(1) Kernel Bypass 0.042 ms

Note: hw-ultra reflects the bare-metal polar queue hardware dispatch latency, fully bypassing OS/kernel overhead.

The Astrophysical Architecture

Standard LLM inference loops bleed memory and suffocate on OS overhead. We are bypassing the kernel entirely by routing raw Tensor data structures directly to the physical hardware using extreme astrophysical concepts:

-1. The Guardians (Universal Healing & Protection)

Code Ref: src/marley_and_fifike.rs
Math: \Delta S_{fifike} < 0 (Entropy Reduction)

Marley the Astronaut Dog 🐕 and Fifike the Cat 🐈. Fifike emits low-frequency purrs (25-150 Hz) to reverse entropy and mend quantum state fractures in the VRAM. Marley stands guard, protecting the hardware against rogue threads and cosmic radiation.

0. The Big Bang (Network Instantiation)

Code Ref: src/big_bang.rs
Math: V_{cache} = \int_{0}^{t} ext{alloc}(t) dt

The Big Bang is our cold-boot sequence. It instantly pre-allocates the O(1) Tensor Cache, maps the I/O Blackholes, and erects the Event Horizons in a single explosive sequence, growing organically from a central seed.

1. The Multiverse (Stigmergic Distributed Inference)

Code Ref: src/multiverse.rs
Math: \sum au_{layer} o ext{Global Graph}

We compute across a swarm of Apple Silicon and AMD machines without a master node. Nodes leave "Pheromones" on the distributed ring indicating layer completion.

2. Pulsars (Hardware Synchronization Beacons)

Code Ref: src/pulsar.rs
Math: \Delta t = t_{CNTVCT\_EL0} \pm 1 ext{ns}

Rapidly rotating neutron stars. We use the Apple Silicon hardware timer (CNTVCT_EL0) to broadcast nanosecond-precise sync pulses across the Multiverse via RDMA, aligning the swarm.

3. Cosmic Microwave Background (The KV-Cache)

Code Ref: src/cmb.rs
Math: KV_{cache} = \sum_{i=0}^{N} (K_i \cdot V_i)

The residual radiation from the Big Bang that permeates the entire context window. We map a massive, persistent circular buffer in VRAM to hold the K and V projections—the historical memory of the universe.

4. Supernovas (Dynamic Burst Compute)

Code Ref: src/supernova.rs
Math: f_{clk} = \max(f_{SMC})

When a prompt arrives, we manipulate the SMC to uncap the GPU TDP limit, bursting clock speeds to their absolute maximum (a supernova explosion) to achieve the lowest possible Time-To-First-Token (TTFT).

5. The Magnetar (Memory & Thread Pinning)

Code Ref: src/magnetar.rs
Math: P( ext{page\_fault}) = 0

We use mlock to magnetically pin our Tensor Cache physical pages in RAM (zero page faults) and pinning execution threads exclusively to Apple Silicon Firestorm P-Cores.

6. Dark Matter & Asteroid Belts (Sparse Zero-Paging & Scatter-Gather)

Code Refs: src/dark_matter.rs, src/asteroid_belt.rs
Math: Sparsity ratio S = rac{\|W_{zero}\|}{\|W_{total}\|}

For Highly Sparse Models (MoE), we map millions of virtual memory addresses to a single physical Zero-Page in RAM. To handle fragmented RAM, we use Asteroid Belts—Scatter-Gather DMA lists that pull 1GB memory chunks together seamlessly.

7. Event Horizons (Strict Memory Firewalls)

Code Ref: src/event_horizon.rs
Math: $ orall addr otin MMU \implies ext{Panic}$

We establish a hardware-enforced MMU memory firewall around the Tensor cache to isolate the Singularity. Nothing escapes.

8. The I/O Blackhole Portal

Code Ref: src/io_blackhole.rs
Math: \lim_{ ext{DMA} o 0} ext{CPU\_Cycles} = 0

Zero-copy DMA. We bypass the CPU completely by mmap-ing NVMe SSD storage directly into the physical address space of our bare-metal Tensor Cache.

9. Redshifting (Dynamic Precision Downcasting)

Code Ref: src/redshift.rs
Math: W_{INT4} = ext{round}(W_{FP16} / \Delta)

Tensors redshift into lower precisions (FP16 -> INT8 -> INT4) in transit to save extreme amounts of bandwidth.

10. Antigravity (Weightless Tensor Levitation)

Code Ref: src/antigravity.rs
Math: \lim_{t o 0} ext{Cache\_Miss}(t) = 0

We defy the gravitational pull of slow memory (RAM/Disk) by aggressively levitating tensors into the ultra-fast L1/L2 SRAM cache using prfm (Prefetch Memory) instructions before they are even needed.

11. Wormholes & Quantum Entanglement

Code Refs: src/wormhole.rs, src/quantum_entanglement.rs
Math: $|\psi angle = rac{1}{\sqrt{2}}(|00 angle + |11 angle)$

We use Peer-to-Peer (P2P) DMA to bypass the PCIe root complex. Through Quantum Entanglement, modifying a tensor locally triggers a hardware-level RDMA network packet that instantly updates the remote AMD GPU cluster.

12. Gravitational Lensing & Time Dilation

Code Refs: src/gravitational_lensing.rs, src/time_dilation.rs
Math: P(x_{t+1} \dots x_{t+5} | x_{<t})

We use Speculative Decoding to compute 5 future tokens simultaneously (Lensing), executing infinitely inside dilated time asynchronous micro-batches.

13. Dark Energy (Accelerating Entropy)

Code Ref: src/dark_energy.rs
Math: T_{t+1} = T_t imes e^{k \cdot ext{rep}}

If the model gets stuck in a repetitive loop (gravity taking over), Dark Energy dynamically scales the generation temperature (entropy) to force creative expansion.

14. The 4D Polar Galaxy Queue & Compute Singularity

Code Refs: src/polar_queue.rs, src/math/matmul.rs
Math: \mathbf{C} = \mathbf{A} imes \mathbf{B}^T

Multiple asynchronous spiral arms (Weights, Activations) continuously merge into an accretion disk ALU compute singularity. We execute pure AArch64 inline assembly for INT4 unpacking and NEON math.

15. Fusion-Polarity (Kernel Fusion)

Code Ref: src/fusion_polarity.rs
Math: ext{SiLU}(\mathbf{X} \cdot \mathbf{W}) o ext{Atomic Op}

Forcing opposite polarities (Weights and Activations) together under immense pressure. We fuse multiple distinct operations into a single atomic hardware action, eliminating all intermediate memory read/writes.

16. White Holes (Infinite Token Ejection)

Code Ref: src/white_hole.rs
Math: \lim_{L o \infty} \sum_{i} t_i

A black hole consumes, a white hole endless ejects. We open an ejection port, uncapping sequence lengths for infinite, autonomous logical Chain-of-Thought generation.

17. The Quasar (RDMA Output Jets)

Code Ref: src/quasar.rs
Math: rac{d( ext{tokens})}{dt} = ext{RDMA}_{bandwith}

The Singularity blasts output tokens through a high-speed network socket via RDMA without touching the CPU.

18. Hawking Radiation (Thermal Cache Eviction)

Code Ref: src/hawking_radiation.rs
Math: M_{evap} \propto T_{idle}

A background thread slowly "evaporates" cold, unused memory pages back to the OS using madvise to prevent OOM collapses.


Universal Benchmark Matrix

Run it effortlessly with uv:

uv run scripts/mlx_bench_matrix_yall.py

19. Optimal Group-Wise L2 Ternary Quantizer, 2-Bit Packing & SIMD-Group Metal Kernel (mlxQuant + mac++)

Code Ref: armillary_sphere_tensor.py · test_armillary_ternary_bench.py

  • quantize_ternary_groupwise (Alternating Voronoi / Lloyd-Max Coordinate Descent): Closed-form L2-optimal per-group scale \alpha_g = \frac{\sum_{i \in \text{active}} |w_i|}{\max(1, |\text{active}|)} at group_size=64 followed by 2-step alternating refinement (\tau_g^{(t)} = 0.5\alpha_g^{(t)}, \alpha_g^{(t+1)} = \mathbb{E}[|w_g| \mid |w_g| > \tau_g^{(t)}]).
    • Multi-Scale Weight MSE: 1.039555 (naive global absmean) \to 0.429188 (1-step groupwise) \to 0.421371 (59.47% MSE reduction vs naive).
    • Heavy-Tailed Outlier Weight MSE: 0.708068 (naive) \to 0.478408 (1-step) \to 0.286356 (40.14% additional MSE reduction over 1-step groupwise).
    • Perplexity Proxy $\exp(\text{LogitMSE})$: 2.8166 \to 1.5355 \to 1.5310.
  • 2-Bit Packed Ternary Storage (pack_ternary_2bit / unpack_ternary_2bit):
    • Packs 4 trits ({-1, 0, +1} mapped to {0, 1, 2}) per uint8 byte LSB-first with zero roundtrip loss: 2048.0 KB (FP16) / 1024.0 KB (INT8) \to 256.0 KB (8.00x compression vs FP16, 4.00x vs INT8).
  • Fused Apple Silicon Metal 2-Bit Ternary Kernel (metal_packed_ternary_matvec):
    • Custom mx.fast.metal_kernel with 32-lane SIMD-group (simd_sum) warp reductions that unpacks 4 trits/byte branchlessly in GPU registers (int((packed >> s) & 0x3u) - 1) without materializing unpacked weights in DRAM.
    • 4096x4096 MatVec Latency (Device(gpu, 0)): 3.291 ms (unpack+matmul) | 0.614 ms (dense FP16) \to 0.290 ms (11.34x speedup vs unpack+matmul, 2.11x speedup vs dense FP16) at 115.61 GB/s effective FP16 weight bandwidth (16.28 GB/s physical packed bandwidth, max_err = 0.00024).
  • Apple Silicon Metal (Device(gpu, 0)) Fused Geodesic Step (_compiled_geodesic_step):
    • Elides intermediate boolean mask materialization by fusing mx.sum(mask) inside @mx.compile: 5.758 ms/iter \to 2.456 ms/iter (2.34x speedup) on 2048x2048 tensors, with 12.80x compression (16.00 MB fp32 \to 1.2500 MB packed ternary + scales).
uv run pytest test_armillary_ternary_bench.py -s

🙏 Acknowledgements & Support

This astrophysical architecture would not exist without the relentless open-weight contributions of the global AI community. We specifically want to emphasize and acknowledge:

  • QWEN (Alibaba Cloud) – For their phenomenal model architecture and continuously pushing the boundaries of what open intelligence can achieve.
  • DeepSeek – For pioneering highly efficient MoE (Mixture-of-Experts) architectures that heavily inspired our Dark Matter & Asteroid Belt sparse zero-paging structures.

🌌 Help Us Stay Afloat

Mapping the universe requires energy (and a lot of bare-metal compute). If this framework helped you bypass the OS and touch the silicon, consider supporting the research to keep the ship flying:

Let's build the universe.