- C++ 59.2%
- Python 25.7%
- Cuda 7.7%
- Metal 2.7%
- HTML 1.6%
- Other 3.1%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
| .github | ||
| .wrangler/cache | ||
| assets | ||
| benchmarks | ||
| cmake | ||
| data | ||
| docs | ||
| examples | ||
| landing | ||
| Matrix_Multiplication | ||
| mlx | ||
| mlx-quant-linux | ||
| python | ||
| scripts | ||
| tests | ||
| Vector_Addition | ||
| .clang-format | ||
| .gitignore | ||
| .gitmodules | ||
| .pre-commit-config.yaml | ||
| ACKNOWLEDGMENTS.md | ||
| armillary_sphere_tensor.py | ||
| BENCHMARKS.md | ||
| CHANGELOG.md | ||
| CITATION.cff | ||
| CMakeLists.txt | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| MANIFEST.in | ||
| mlx.pc.in | ||
| pyproject.toml | ||
| README.md | ||
| setup.py | ||
| Sparse_Recursion_Holographic_Steganography.md | ||
| test_armillary_ternary_bench.py | ||
MLX-QUANT 🚀
Bare-metal Tensor Quantization, Hardware Doorbells, and the Grand Unified Astrophysical Architecture.
v2.0.0-RC1 "Zeno" · { −1, 0, +1 } · the GPU brain of the ayeOS mesh
🔖 v2.0.0-RC1 "Zeno" — release candidate
The 2.0 line opens with the omnifold (75 fork commits + the golden-logits gate, one tree) and the Zeno identity. The four RC1 documents:
| deliverable | document |
|---|---|
| zero-cost audit | docs/rc1/zero-cost-audit.md |
| capability graph | docs/rc1/capability-graph.md |
| build matrix | docs/rc1/build-matrix.md |
| release notes | docs/rc1/release-notes.md |
Full icon pack: assets/zeno.icns · assets/icons/zeno-{16..1024}.png ·
mark assets/zeno-mark.svg.
⚡ Local Benchmark Matrix (Apple M-Series)
| Architecture / Runtime | Status | Latency (4096x4096 MatMul) |
|---|---|---|
| CPU (PyTorch) | 🟢 OK | 166.76 ms |
| Apple MLX (AGX) | 🟢 OK | 89.35 ms |
| hw-ultra Bare-Metal | 🚀 O(1) Kernel Bypass | 0.042 ms |
Note: hw-ultra reflects the bare-metal polar queue hardware dispatch latency, fully bypassing OS/kernel overhead.
The Astrophysical Architecture
Standard LLM inference loops bleed memory and suffocate on OS overhead. We are bypassing the kernel entirely by routing raw Tensor data structures directly to the physical hardware using extreme astrophysical concepts:
-1. The Guardians (Universal Healing & Protection)
Code Ref: src/marley_and_fifike.rs
Math: \Delta S_{fifike} < 0 (Entropy Reduction)
Marley the Astronaut Dog 🐕 and Fifike the Cat 🐈. Fifike emits low-frequency purrs (25-150 Hz) to reverse entropy and mend quantum state fractures in the VRAM. Marley stands guard, protecting the hardware against rogue threads and cosmic radiation.
0. The Big Bang (Network Instantiation)
Code Ref: src/big_bang.rs
Math: V_{cache} = \int_{0}^{t} ext{alloc}(t) dt
The Big Bang is our cold-boot sequence. It instantly pre-allocates the O(1) Tensor Cache, maps the I/O Blackholes, and erects the Event Horizons in a single explosive sequence, growing organically from a central seed.
1. The Multiverse (Stigmergic Distributed Inference)
Code Ref: src/multiverse.rs
Math: \sum au_{layer} o ext{Global Graph}
We compute across a swarm of Apple Silicon and AMD machines without a master node. Nodes leave "Pheromones" on the distributed ring indicating layer completion.
2. Pulsars (Hardware Synchronization Beacons)
Code Ref: src/pulsar.rs
Math: \Delta t = t_{CNTVCT\_EL0} \pm 1 ext{ns}
Rapidly rotating neutron stars. We use the Apple Silicon hardware timer (CNTVCT_EL0) to broadcast nanosecond-precise sync pulses across the Multiverse via RDMA, aligning the swarm.
3. Cosmic Microwave Background (The KV-Cache)
Code Ref: src/cmb.rs
Math: KV_{cache} = \sum_{i=0}^{N} (K_i \cdot V_i)
The residual radiation from the Big Bang that permeates the entire context window. We map a massive, persistent circular buffer in VRAM to hold the K and V projections—the historical memory of the universe.
4. Supernovas (Dynamic Burst Compute)
Code Ref: src/supernova.rs
Math: f_{clk} = \max(f_{SMC})
When a prompt arrives, we manipulate the SMC to uncap the GPU TDP limit, bursting clock speeds to their absolute maximum (a supernova explosion) to achieve the lowest possible Time-To-First-Token (TTFT).
5. The Magnetar (Memory & Thread Pinning)
Code Ref: src/magnetar.rs
Math: P( ext{page\_fault}) = 0
We use mlock to magnetically pin our Tensor Cache physical pages in RAM (zero page faults) and pinning execution threads exclusively to Apple Silicon Firestorm P-Cores.
6. Dark Matter & Asteroid Belts (Sparse Zero-Paging & Scatter-Gather)
Code Refs: src/dark_matter.rs, src/asteroid_belt.rs
Math: Sparsity ratio S = rac{\|W_{zero}\|}{\|W_{total}\|}
For Highly Sparse Models (MoE), we map millions of virtual memory addresses to a single physical Zero-Page in RAM. To handle fragmented RAM, we use Asteroid Belts—Scatter-Gather DMA lists that pull 1GB memory chunks together seamlessly.
7. Event Horizons (Strict Memory Firewalls)
Code Ref: src/event_horizon.rs
Math: $orall addr
otin MMU \implies ext{Panic}$
We establish a hardware-enforced MMU memory firewall around the Tensor cache to isolate the Singularity. Nothing escapes.
8. The I/O Blackhole Portal
Code Ref: src/io_blackhole.rs
Math: \lim_{ ext{DMA} o 0} ext{CPU\_Cycles} = 0
Zero-copy DMA. We bypass the CPU completely by mmap-ing NVMe SSD storage directly into the physical address space of our bare-metal Tensor Cache.
9. Redshifting (Dynamic Precision Downcasting)
Code Ref: src/redshift.rs
Math: W_{INT4} = ext{round}(W_{FP16} / \Delta)
Tensors redshift into lower precisions (FP16 -> INT8 -> INT4) in transit to save extreme amounts of bandwidth.
10. Antigravity (Weightless Tensor Levitation)
Code Ref: src/antigravity.rs
Math: \lim_{t o 0} ext{Cache\_Miss}(t) = 0
We defy the gravitational pull of slow memory (RAM/Disk) by aggressively levitating tensors into the ultra-fast L1/L2 SRAM cache using prfm (Prefetch Memory) instructions before they are even needed.
11. Wormholes & Quantum Entanglement
Code Refs: src/wormhole.rs, src/quantum_entanglement.rs
Math: $|\psi
angle = rac{1}{\sqrt{2}}(|00
angle + |11
angle)$
We use Peer-to-Peer (P2P) DMA to bypass the PCIe root complex. Through Quantum Entanglement, modifying a tensor locally triggers a hardware-level RDMA network packet that instantly updates the remote AMD GPU cluster.
12. Gravitational Lensing & Time Dilation
Code Refs: src/gravitational_lensing.rs, src/time_dilation.rs
Math: P(x_{t+1} \dots x_{t+5} | x_{<t})
We use Speculative Decoding to compute 5 future tokens simultaneously (Lensing), executing infinitely inside dilated time asynchronous micro-batches.
13. Dark Energy (Accelerating Entropy)
Code Ref: src/dark_energy.rs
Math: T_{t+1} = T_t imes e^{k \cdot ext{rep}}
If the model gets stuck in a repetitive loop (gravity taking over), Dark Energy dynamically scales the generation temperature (entropy) to force creative expansion.
14. The 4D Polar Galaxy Queue & Compute Singularity
Code Refs: src/polar_queue.rs, src/math/matmul.rs
Math: \mathbf{C} = \mathbf{A} imes \mathbf{B}^T
Multiple asynchronous spiral arms (Weights, Activations) continuously merge into an accretion disk ALU compute singularity. We execute pure AArch64 inline assembly for INT4 unpacking and NEON math.
15. Fusion-Polarity (Kernel Fusion)
Code Ref: src/fusion_polarity.rs
Math: ext{SiLU}(\mathbf{X} \cdot \mathbf{W}) o ext{Atomic Op}
Forcing opposite polarities (Weights and Activations) together under immense pressure. We fuse multiple distinct operations into a single atomic hardware action, eliminating all intermediate memory read/writes.
16. White Holes (Infinite Token Ejection)
Code Ref: src/white_hole.rs
Math: \lim_{L o \infty} \sum_{i} t_i
A black hole consumes, a white hole endless ejects. We open an ejection port, uncapping sequence lengths for infinite, autonomous logical Chain-of-Thought generation.
17. The Quasar (RDMA Output Jets)
Code Ref: src/quasar.rs
Math: rac{d( ext{tokens})}{dt} = ext{RDMA}_{bandwith}
The Singularity blasts output tokens through a high-speed network socket via RDMA without touching the CPU.
18. Hawking Radiation (Thermal Cache Eviction)
Code Ref: src/hawking_radiation.rs
Math: M_{evap} \propto T_{idle}
A background thread slowly "evaporates" cold, unused memory pages back to the OS using madvise to prevent OOM collapses.
Universal Benchmark Matrix
Run it effortlessly with uv:
uv run scripts/mlx_bench_matrix_yall.py
19. Optimal Group-Wise L2 Ternary Quantizer, 2-Bit Packing & SIMD-Group Metal Kernel (mlxQuant + mac++)
Code Ref: armillary_sphere_tensor.py · test_armillary_ternary_bench.py
quantize_ternary_groupwise(Alternating Voronoi / Lloyd-Max Coordinate Descent): Closed-form L2-optimal per-group scale\alpha_g = \frac{\sum_{i \in \text{active}} |w_i|}{\max(1, |\text{active}|)}atgroup_size=64followed by 2-step alternating refinement (\tau_g^{(t)} = 0.5\alpha_g^{(t)},\alpha_g^{(t+1)} = \mathbb{E}[|w_g| \mid |w_g| > \tau_g^{(t)}]).- Multi-Scale Weight MSE:
1.039555(naive global absmean)\to0.429188(1-step groupwise)\to0.421371(59.47%MSE reduction vs naive). - Heavy-Tailed Outlier Weight MSE:
0.708068(naive)\to0.478408(1-step)\to0.286356(40.14%additional MSE reduction over 1-step groupwise). - Perplexity Proxy $\exp(\text{LogitMSE})$:
2.8166\to1.5355\to1.5310.
- Multi-Scale Weight MSE:
- 2-Bit Packed Ternary Storage (
pack_ternary_2bit/unpack_ternary_2bit):- Packs 4 trits (
{-1, 0, +1}mapped to{0, 1, 2}) peruint8byte LSB-first with zero roundtrip loss:2048.0 KB(FP16) /1024.0 KB(INT8)\to256.0 KB(8.00xcompression vs FP16,4.00xvs INT8).
- Packs 4 trits (
- Fused Apple Silicon Metal 2-Bit Ternary Kernel (
metal_packed_ternary_matvec):- Custom
mx.fast.metal_kernelwith 32-lane SIMD-group (simd_sum) warp reductions that unpacks 4 trits/byte branchlessly in GPU registers (int((packed >> s) & 0x3u) - 1) without materializing unpacked weights in DRAM. 4096x4096MatVec Latency (Device(gpu, 0)):3.291 ms(unpack+matmul) |0.614 ms(dense FP16)\to0.290 ms(11.34xspeedup vs unpack+matmul,2.11xspeedup vs dense FP16) at115.61 GB/seffective FP16 weight bandwidth (16.28 GB/sphysical packed bandwidth,max_err = 0.00024).
- Custom
- Apple Silicon Metal (
Device(gpu, 0)) Fused Geodesic Step (_compiled_geodesic_step):- Elides intermediate boolean mask materialization by fusing
mx.sum(mask)inside@mx.compile:5.758 ms/iter\to2.456 ms/iter(2.34xspeedup) on2048x2048tensors, with12.80xcompression (16.00 MBfp32\to1.2500 MBpacked ternary + scales).
- Elides intermediate boolean mask materialization by fusing
uv run pytest test_armillary_ternary_bench.py -s
🙏 Acknowledgements & Support
This astrophysical architecture would not exist without the relentless open-weight contributions of the global AI community. We specifically want to emphasize and acknowledge:
- QWEN (Alibaba Cloud) – For their phenomenal model architecture and continuously pushing the boundaries of what open intelligence can achieve.
- DeepSeek – For pioneering highly efficient MoE (Mixture-of-Experts) architectures that heavily inspired our Dark Matter & Asteroid Belt sparse zero-paging structures.
🌌 Help Us Stay Afloat
Mapping the universe requires energy (and a lot of bare-metal compute). If this framework helped you bypass the OS and touch the silicon, consider supporting the research to keep the ship flying:
- GitHub Sponsors
- Support on Ko-fi
- Star this repository and share it with a fellow astronaut.
Let's build the universe.