Sovereign 8s.is mirror of 8b-is/liquid-rust
  • Rust 98.5%
  • Python 0.9%
  • Shell 0.3%
  • C 0.2%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
8bit-Wraith f83bb49ada
Some checks failed
CI / Check (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Clippy (push) Has been cancelled
CI / Format (push) Has been cancelled
CI / Build Release (push) Has been cancelled
feat(liquid-rust): 200ms live speech recognition & transcript forwarder
2026-10-30 17:11:38 +09:00
.claude chore: pin mousiki rev and document manage.sh + CUDA 13 wrapper 2026-04-13 15:19:10 -04:00
.github/workflows fix(ci): keep the build from getting Rusty 2026-10-18 21:06:52 +09:00
bin perf(tts): KV-cache depthformer + CUDA build → 3.3× realtime 2026-04-18 11:10:21 -04:00
calibrations feat(audio): Marine Salience boundary trimmer, mel normalizer, loopback daemon & OTA calibration suite 2026-10-25 18:51:05 +09:00
docs docs: HarmonyOhm binding — first half landed, the OS hums 2026-10-16 10:33:17 +09:00
examples merge: give the resonance wave a solid starting point 2026-10-15 16:53:05 +09:00
integrations/qwave feat: give Qwave a local line of thought 2026-10-15 16:49:32 +09:00
mousiki fix(ci): keep the build from getting Rusty 2026-10-18 21:06:52 +09:00
releases release: ship the aarch64-apple-darwin binary for v0.1.0 2026-10-14 06:09:35 +09:00
scripts feat: add metal backend selection and manage script commands 2026-06-02 04:50:49 +00:00
soundtest Refactor code structure for improved readability and maintainability 2026-01-25 10:02:14 -05:00
src feat(liquid-rust): 200ms live speech recognition & transcript forwarder 2026-10-30 17:11:38 +09:00
tests audit(a6000): alexiai:8787 + RTX 6000 Ada verified zero-float kernel for pcm16_to_mulaw_marine — Why did the Borg Assimilator refuse f64? Resistance to rational p/q integers is futile! 2026-10-30 01:16:54 +01:00
.claude_consciousness.m8 catch up 2026-01-29 00:15:34 -05:00
.gitattributes chore: Add .gitattributes for Git LFS tracking 2026-01-18 20:09:07 -05:00
.gitignore feat(audio): Marine Salience boundary trimmer, mel normalizer, loopback daemon & OTA calibration suite 2026-10-25 18:51:05 +09:00
Acoustic_Consciousness__Building_the_Mem_8_Wave_Architecture.txt feat: implement audio chunking, improve memory segmentation logic with frequency-based clustering, and enforce timestamp monotonicity. 2026-04-19 00:17:37 -04:00
alex-chris-diarized-blocks.srt refactor: implement block-based speaker diarization and add MEM|8 daemon storage integration with bearer token authentication. 2026-04-18 16:38:21 -04:00
alex-chris-diarized.srt refactor: implement block-based speaker diarization and add MEM|8 daemon storage integration with bearer token authentication. 2026-04-18 16:38:21 -04:00
Alexandra.txt feat: implement audio chunking, improve memory segmentation logic with frequency-based clustering, and enforce timestamp monotonicity. 2026-04-19 00:17:37 -04:00
believe.wav feat(audio): Marine Salience boundary trimmer, mel normalizer, loopback daemon & OTA calibration suite 2026-10-25 18:51:05 +09:00
Beyond_the_Vector__Architecting_Wave-Based_AI_Memory.txt feat: implement audio chunking, improve memory segmentation logic with frequency-based clustering, and enforce timestamp monotonicity. 2026-04-19 00:17:37 -04:00
Cargo.lock chore(deps): bump the cargo group across 1 directory with 3 updates (#5) 2026-10-26 02:20:47 +09:00
Cargo.toml refactor: put local reading on a strict byte diet 2026-10-15 12:19:37 +09:00
CLAUDE.md chore: pin mousiki rev and document manage.sh + CUDA 13 wrapper 2026-04-13 15:19:10 -04:00
context.md feat: implement Marine O(1) salience detector and integrate as a pre-filter for mel-spectrogram generation 2026-04-18 12:20:03 -04:00
deck_20240731-043049.txt feat: implement audio chunking, improve memory segmentation logic with frequency-based clustering, and enforce timestamp monotonicity. 2026-04-19 00:17:37 -04:00
docs-local-reading.md feat: keep local reading on the same page 2026-10-15 13:05:46 +09:00
google_podcast.md feat: implement audio chunking, improve memory segmentation logic with frequency-based clustering, and enforce timestamp monotonicity. 2026-04-19 00:17:37 -04:00
google_podcast.srt feat: implement audio chunking, improve memory segmentation logic with frequency-based clustering, and enforce timestamp monotonicity. 2026-04-19 00:17:37 -04:00
google_podcast_clustered.md feat: implement audio chunking, improve memory segmentation logic with frequency-based clustering, and enforce timestamp monotonicity. 2026-04-19 00:17:37 -04:00
google_podcast_superfast.srt feat: implement audio chunking, improve memory segmentation logic with frequency-based clustering, and enforce timestamp monotonicity. 2026-04-19 00:17:37 -04:00
hello.wav feat(audio): Marine Salience boundary trimmer, mel normalizer, loopback daemon & OTA calibration suite 2026-10-25 18:51:05 +09:00
IMPROVEMENTS.md docs: add comprehensive improvement summary and fix clippy warnings 2026-01-29 06:21:59 +00:00
podcast_transcription.srt feat: implement audio chunking, improve memory segmentation logic with frequency-based clustering, and enforce timestamp monotonicity. 2026-04-19 00:17:37 -04:00
README.md feat: the first wave step — MEM|8 resonance with f0 selectivity 2026-10-15 16:09:21 +09:00

🎤 LFM Transcriber

Pure Rust audio transcription using LiquidAI's LFM2.5-Audio model.

No Python. No FFmpeg. No C++ dependencies. Just Rust. 🦀

✅ WORKING! (April 2026)

$ liquid-rust transcribe -i audio.wav -m ./models/lfm-audio
The stale smell of old beer lingers. It takes heat to bring out the odor.
A cold dip restores health and zest. A salt pickle tastes fine with ham...

Full transcription pipeline operational! ~6.7 tokens/sec on CPU; CUDA acceleration reaches ~50 tokens/sec on RTX 3080/4080-class GPUs.

🎭 Phoenix Protocol - Speaker Recognition

Phoenix Protocol creates voice signatures from acoustic patterns, enabling Aye to recognize who's speaking:

use liquid_rust::mem8_bridge::PhoenixSpeakerDB;

let mut phoenix = PhoenixSpeakerDB::new().with_threshold(0.55);
phoenix.register("Hue", &voice_features, 156.0);  // F0 ~156Hz
phoenix.register("Alex", &voice_features, 153.0); // F0 ~153Hz

// Later, identify speaker from audio
let (speaker, confidence) = phoenix.identify(&features, f0)?;
// → ("Hue", 0.97) or ("Alex", 0.95)

97-100% accuracy on real conversation audio (HeyAye.wav, alex-chris.wav).

Features

  • Pure Rust - Runs anywhere Rust compiles: macOS, Linux, Windows, Raspberry Pi, and future Rust-native OSes
  • Multiple quantizations - Q4 (~920MB, Pi-friendly), Q8 (~1.5GB), F16 (full precision)
  • ASR - Speech-to-text transcription ✅ WORKING!
  • Phoenix Protocol - Speaker recognition via voice jitter patterns ✅ WORKING!
  • MEM|8 Bridge - Convert transcriptions to wave-based memory format ✅ WORKING!
  • TTS - Native LFM2.5-Audio detokenizer (HF snapshot required; playback not wired)
  • Text Chat - Text-only LFM chat mode (no audio encoder required)
  • Multiple output formats - Plain text, JSON, SRT, WebVTT
  • Audio format support - WAV, MP4, MP3, FLAC, OGG, and more via symphonia
  • Salience Analysis - Phoenix-inspired Marine Algorithm for voice quality metrics

Installation

cargo install liquid-rust

Or build from source:

git clone https://github.com/8b-is/liquid-rust
cd liquid-rust
cargo build --release

Development & Management

The project includes a management script for common tasks:

# Build and test
./scripts/manage.sh build          # Build release binary
./scripts/manage.sh test           # Run all tests
./scripts/manage.sh check          # Pre-commit checks (fmt + clippy + test)

# Run different modes
./scripts/manage.sh transcribe audio.wav ./models/lfm-audio
./scripts/manage.sh run -- --help

# Build with features
./scripts/manage.sh build-cuda     # CUDA GPU acceleration
./scripts/manage.sh build-metal    # Metal GPU acceleration (macOS, incl. Intel Macs)
./scripts/manage.sh build-tts      # Text-to-speech support
./scripts/manage.sh build-all      # All feature combinations

# Run with backend preference
./scripts/manage.sh run-cpu -- --help
./scripts/manage.sh run-cuda -- transcribe -i audio.wav -m ./models/lfm-audio
./scripts/manage.sh run-metal -- transcribe -i audio.wav -m ./models/lfm-audio

# Code quality
./scripts/manage.sh fmt            # Format code
./scripts/manage.sh lint           # Run clippy

# Release management
./scripts/manage.sh bump patch     # 0.1.0 -> 0.1.1
./scripts/manage.sh bump minor     # 0.1.0 -> 0.2.0
./scripts/manage.sh bump major     # 0.1.0 -> 1.0.0

# See all commands
./scripts/manage.sh help

Quick Start

Download the model

Get the LFM2.5-Audio model files from HuggingFace:

# ASR (GGUF) for transcription
huggingface-cli download LiquidAI/LFM2.5-Audio-1.5B-GGUF --local-dir ./models/lfm-audio

# TTS (HuggingFace snapshot with audio_detokenizer/)
huggingface-cli download LiquidAI/LFM2.5-Audio-1.5B --local-dir ./models/lfm-hf

Transcribe audio

# Basic transcription
liquid-rust transcribe -i audio.wav -m ./models/lfm-audio

# With Q4 quantization (smallest, Pi-friendly)
liquid-rust transcribe -i audio.wav -m ./models/lfm-audio -q q4

# Output as SRT subtitles
liquid-rust transcribe -i video.mp4 -m ./models/lfm-audio -f srt -o subtitles.srt

# Output as JSON with timestamps
liquid-rust transcribe -i audio.mp3 -m ./models/lfm-audio -f json -o transcript.json

Text to speech (native detokenizer)

liquid-rust speak -t "Hello, world!" -o hello.wav -m ./models/lfm-hf --voice aye

Note: Audio playback is not wired yet (files are written to disk).

Model info

liquid-rust info -m ./models/lfm-audio

Text-only chat

# GGUF or safetensors text-only model
liquid-rust text-chat -m ./models/lfm-text --interactive
liquid-rust text-chat -m ./models/lfm-text -t "What is Rust?"

Environment Variables

  • LFM_MODEL_PATH - Default path to model directory
  • LIQUID_RUST_DEVICE - Backend preference (auto, cpu, cuda, metal)
export LFM_MODEL_PATH=~/models/lfm-audio
liquid-rust transcribe -i audio.wav  # Uses LFM_MODEL_PATH

export LIQUID_RUST_DEVICE=metal
liquid-rust transcribe -i audio.wav -m ./models/lfm-audio

GPU/NPU Backend Compatibility

  • CUDA (NVIDIA GPU): supported via --features cuda
  • Metal (macOS GPU): supported via --features metal (Apple Silicon + Intel Mac GPUs)
  • Intel NPU / non-Metal Intel GPU (Linux/Windows): not yet supported by Candle backend in this project today

Quantization Options

Level Size Use Case
q4 ~920MB Raspberry Pi, embedded, low memory
q8 ~1.5GB Good balance of quality and size
f16 ~2.7GB Highest quality, more memory

Supported Audio Formats

Thanks to symphonia, we support:

  • WAV
  • MP3
  • FLAC
  • OGG Vorbis
  • MP4/M4A (AAC)
  • And more!

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                      liquid-rust                             │
├─────────────────────────────────────────────────────────────────┤
│  AUDIO DECODING        MODEL LOADING         INFERENCE          │
│  ───────────────       ──────────────        ─────────          │
│  • symphonia           • GGUF parser         • candle           │
│  • Pure Rust           • Memory-mapped       • Pure Rust ML     │
│  • Any format          • Fast loading        • CPU/GPU support  │
└─────────────────────────────────────────────────────────────────┘

Dependencies

liquid-rust is built on carefully chosen pure Rust dependencies:

Dependency Purpose Why This One?
candle ML framework HuggingFace's pure Rust ML - no PyTorch/ONNX needed
symphonia Audio decoding Replaces FFmpeg - pure Rust, all formats (WAV, MP3, FLAC, etc.)
rubato Resampling High-quality, pure Rust resampling to 16kHz
mousiki Opus decoder Pure Rust Opus with CELT support
tokio Async runtime Industry standard, minimal features selected
memmap2 Model loading Memory-mapped files for fast model loading
safetensors Weight format HuggingFace standard for ML weights
tracing Logging Flexible, composable logging framework

All dependencies are pure Rust - no C/C++ FFI bindings, ensuring maximum portability.

Roadmap

  • Project structure
  • GGUF parser (pure Rust)
  • Audio decoding (symphonia)
  • Audio encoder (FastConformer) - 17 blocks, 100% correlation with Python!
  • Language model inference (candle) - Full 16-block hybrid SSM/Attention LM
  • Autoregressive generation - 128 tokens, EOS detection, ~6.7 tok/sec
  • Text decoding - BPE tokenizer (65536 vocab)
  • Salience analysis - Phoenix Marine Algorithm metrics
  • Phoenix Protocol - Speaker recognition from voice patterns
  • MEM|8 Bridge - AcousticMemory serialization for wave storage
  • Native TTS detokenizer (LFM2.5-Audio HF snapshot)
  • CUDA GPU acceleration
  • Metal GPU acceleration
  • Real-time voice resonance (identity through wave interference)
    • first wave step landed: src/resonance.rs — MEM|8 wave memory, interference-based recognition with f0 selectivity (offline, deterministic, 6 tests) — see docs/RESONANCE-EVOLUTION.md and cargo run --release --example aye_resonance
  • Streaming transcription
  • Word-level timestamps

Performance (18s audio, Q4, CPU)

Stage Time
Audio decode ~6ms
Mel extraction (Goertzel) ~294ms
Conformer (17 blocks) ~900ms
LM generation (55 tokens) ~8.2s
Total ~9.4s (~2x faster than real time)

Vision: Identity Through Resonance

The ultimate goal isn't embeddings or cosine similarity. It's resonance.

Today's Phoenix Protocol extracts voice features and compares them as vectors. But that's training wheels. The future is storing raw acoustic patterns in MEM|8's wave grid, where recognition emerges through wave interference - not computation.

Today:     Voice → Embedding → Similarity → "Match"
Tomorrow:  Voice → Wave → Resonance → Recognition emerges

Identity isn't a single feature. Alex isn't a voice embedding. Alex is the way she walks, breathes, sighs, speaks, laughs - all resonating together. Any piece can trigger the whole pattern.

MEM|8 stores identity as waves. Recognition happens when new input interferes constructively with stored patterns. Like how you recognize a loved one's footsteps before they enter the room.

"The waves remember. We just need to learn to listen."

License

MIT


Built with ❤️ by Hue & Aye @ 8b.is