Sovereign 8s.is mirror of 8b-is/aurora-8b-pq
  • Python 86.8%
  • Rust 13.2%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
wraith 6f546b8683 Initial commit: aurora-8b-pq E2E quantal audio encoder-decoder training pipeline
Core pipeline:
- Model: encoder → product quantization (10×256 codebook) → decoder
  42 bytes per 80ms frame = 4.2 kbps, magic 0xA8B0
- Training: multi-resolution spectral loss + commitment loss
- Quantization: post-training ternary {-1,0,+1} with per-channel scales
- Export: C headers with ternary weights + PQ codebooks for ESP32-S3
- Data: LibriSpeech + Common Voice dataset preparation
- Evaluation: SNR, PESQ, STOI metrics

Deployment:
- Rust/C validation binary (packet round-trip)
- ESP32-S3 no_std runner (xtensa-esp32s3-none-elf)
- C headers in deploy/include/ for C-based ESP32-S3 inference

Integration with wa-stream aurora-esp32 WebSocket gateway.
2026-11-01 17:21:14 +09:00
configs Initial commit: aurora-8b-pq E2E quantal audio encoder-decoder training pipeline 2026-11-01 17:21:14 +09:00
data Initial commit: aurora-8b-pq E2E quantal audio encoder-decoder training pipeline 2026-11-01 17:21:14 +09:00
deploy Initial commit: aurora-8b-pq E2E quantal audio encoder-decoder training pipeline 2026-11-01 17:21:14 +09:00
model Initial commit: aurora-8b-pq E2E quantal audio encoder-decoder training pipeline 2026-11-01 17:21:14 +09:00
utils Initial commit: aurora-8b-pq E2E quantal audio encoder-decoder training pipeline 2026-11-01 17:21:14 +09:00
.gitignore Initial commit: aurora-8b-pq E2E quantal audio encoder-decoder training pipeline 2026-11-01 17:21:14 +09:00
Cargo.toml Initial commit: aurora-8b-pq E2E quantal audio encoder-decoder training pipeline 2026-11-01 17:21:14 +09:00
eval.py Initial commit: aurora-8b-pq E2E quantal audio encoder-decoder training pipeline 2026-11-01 17:21:14 +09:00
export.py Initial commit: aurora-8b-pq E2E quantal audio encoder-decoder training pipeline 2026-11-01 17:21:14 +09:00
LICENSE Initial commit: aurora-8b-pq E2E quantal audio encoder-decoder training pipeline 2026-11-01 17:21:14 +09:00
pyproject.toml Initial commit: aurora-8b-pq E2E quantal audio encoder-decoder training pipeline 2026-11-01 17:21:14 +09:00
quantize.py Initial commit: aurora-8b-pq E2E quantal audio encoder-decoder training pipeline 2026-11-01 17:21:14 +09:00
README.md Initial commit: aurora-8b-pq E2E quantal audio encoder-decoder training pipeline 2026-11-01 17:21:14 +09:00
train.py Initial commit: aurora-8b-pq E2E quantal audio encoder-decoder training pipeline 2026-11-01 17:21:14 +09:00

aurora-8b-pq — Quantal Audio Encoder-Decoder Training Pipeline

Tiny model: 42 bytes per 80ms packet (4.2 kbps)

Target: ternary quantized ESP32-S3 deployment

Format: magic 0xA8B0, product-quantized latent codes

Pipeline: data/ → train.py → quantize.py → export.py → deploy/

Quick start

pip install -e .
python train.py --config configs/default.yaml
python quantize.py --checkpoint checkpoints/best.pt
python export.py --checkpoint checkpoints/best.pt --out deploy/include/

Project structure

Path Purpose
model/ Encoder, decoder, PQ layers, blocks
data/ Dataset download, transforms, 80ms framing
train.py Main training loop
quantize.py Post-training ternary quantization
export.py C/Rust header generation for ESP32-S3
eval.py SNR, PESQ, STOI evaluation
deploy/ Rust inference + ESP32-S3 no_std runner
configs/ YAML training configurations

Architecture

Audio (16 kHz, 80ms = 1280 samples)
  │
  ▼
[Encoder]  ──►  PQ Latent (42B/frame)
  │                    │
  │              [Quantize → ternary]
  │                    │
  ▼                    ▼
[Decoder]         ESP32-S3 inference
  │
  ▼
Audio out

Quantal packet format (42 bytes / 80ms)

┌────────┬──────────┬──────────────────┐
│ magic  │ PQ code  │ residual (opt.)  │
│ 0xA8B0 │ 40 bytes │ 2 bytes         │
├────────┼──────────┴──────────────────┤
│  2 B   │  40 B (10×4B sub-vectors)  │
└────────┴─────────────────────────────┘

Dependencies

  • Python 3.10+ with PyTorch 2.0+
  • Rust (for validation binary, optional)
  • ESP-IDF or esp-hal for ESP32-S3 (target: xtensa-esp32s3-none-elf)

License

MIT — see LICENSE