Sovereign 8s.is mirror of 8b-is/aurora-8b-pq
- Python 86.8%
- Rust 13.2%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
Core pipeline:
- Model: encoder → product quantization (10×256 codebook) → decoder
42 bytes per 80ms frame = 4.2 kbps, magic 0xA8B0
- Training: multi-resolution spectral loss + commitment loss
- Quantization: post-training ternary {-1,0,+1} with per-channel scales
- Export: C headers with ternary weights + PQ codebooks for ESP32-S3
- Data: LibriSpeech + Common Voice dataset preparation
- Evaluation: SNR, PESQ, STOI metrics
Deployment:
- Rust/C validation binary (packet round-trip)
- ESP32-S3 no_std runner (xtensa-esp32s3-none-elf)
- C headers in deploy/include/ for C-based ESP32-S3 inference
Integration with wa-stream aurora-esp32 WebSocket gateway.
|
||
| configs | ||
| data | ||
| deploy | ||
| model | ||
| utils | ||
| .gitignore | ||
| Cargo.toml | ||
| eval.py | ||
| export.py | ||
| LICENSE | ||
| pyproject.toml | ||
| quantize.py | ||
| README.md | ||
| train.py | ||
aurora-8b-pq — Quantal Audio Encoder-Decoder Training Pipeline
Tiny model: 42 bytes per 80ms packet (4.2 kbps)
Target: ternary quantized ESP32-S3 deployment
Format: magic 0xA8B0, product-quantized latent codes
Pipeline: data/ → train.py → quantize.py → export.py → deploy/
Quick start
pip install -e .
python train.py --config configs/default.yaml
python quantize.py --checkpoint checkpoints/best.pt
python export.py --checkpoint checkpoints/best.pt --out deploy/include/
Project structure
| Path | Purpose |
|---|---|
model/ |
Encoder, decoder, PQ layers, blocks |
data/ |
Dataset download, transforms, 80ms framing |
train.py |
Main training loop |
quantize.py |
Post-training ternary quantization |
export.py |
C/Rust header generation for ESP32-S3 |
eval.py |
SNR, PESQ, STOI evaluation |
deploy/ |
Rust inference + ESP32-S3 no_std runner |
configs/ |
YAML training configurations |
Architecture
Audio (16 kHz, 80ms = 1280 samples)
│
▼
[Encoder] ──► PQ Latent (42B/frame)
│ │
│ [Quantize → ternary]
│ │
▼ ▼
[Decoder] ESP32-S3 inference
│
▼
Audio out
Quantal packet format (42 bytes / 80ms)
┌────────┬──────────┬──────────────────┐
│ magic │ PQ code │ residual (opt.) │
│ 0xA8B0 │ 40 bytes │ 2 bytes │
├────────┼──────────┴──────────────────┤
│ 2 B │ 40 B (10×4B sub-vectors) │
└────────┴─────────────────────────────┘
Dependencies
- Python 3.10+ with PyTorch 2.0+
- Rust (for validation binary, optional)
- ESP-IDF or esp-hal for ESP32-S3 (target:
xtensa-esp32s3-none-elf)
License
MIT — see LICENSE