- Rust 95.2%
- TeX 3.1%
- Shell 1.7%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
| marqant | ||
| migrations | ||
| rs-smtp | ||
| rust-network-scanner | ||
| src | ||
| testemlx | ||
| .gitattributes | ||
| .gitignore | ||
| Cargo.lock | ||
| Cargo.toml | ||
| COMPRESSION_OPTIMIZATION_RESULTS.md | ||
| compression_results_optimized.txt | ||
| COMPRESSION_TEST_RESULTS.md | ||
| HARMONY.md | ||
| hoover.toml | ||
| LICENSE | ||
| LIFELINE.md | ||
| marine-Universal-Salience-algoritm.tex | ||
| MEM8_Full_Arxiv.tex | ||
| README.md | ||
| STATUS.md | ||
| TRACKING_INTELLIGENCE.md | ||
Hoover — 8b.is × AlexiAI Sovereign Mail & Intelligence Engine
Sovereign SMTP, Live AlexiAI GPU Triage, WHOIS Anti-Privacy Payment Guard, WhatsApp Executive Alerts & Auto-Provisioning
Hoover is the Rust-based sovereign email and intelligence engine powering 8b.is and ns1..ns4.i1.is. Every incoming and stored email is processed through a strictly local loopback AlexiAI instance (GAIA-MLX-QUANT field + cheet-deepseek-v2:16b on our cheet.is NVIDIA RTX 6000 Ada 48GB GPU over sovereign loopback tunnel) — with zero third-party LLM calls.
Features
1. Sovereign AlexiAI Mail Intelligence (src/alexiai.rs, src/api.rs)
- Live Loopback GPU Triage (
127.0.0.1:8787): Every incoming email is folded against the 9-planetGAIA-MLX-QUANTfield (/api/gaia) and summarized bycheet-deepseek-v2:16b(/api/chat) to produce executive summaries, priority scores (1..=5), action items, smart replies, calendar extraction, and contact aggregation. - 🚨 SUPER ULTRA IMPORTANT — SONY Executive Track for Alexandra (
a@8b.is):- Any email from or mentioning Sony (
@sony.*,sony,playstation,shinagawa,kenichiro,totoki) is automatically elevated to Priority 5 (SUPER ULTRA IMPORTANT · SONY EXECUTIVE), routed toa@8b.is(plusca@8b.is,c@8b.is,p@8b.is, andaye@8b.is), and pinned #1 on the Shared Important Topics Board (/api/mail/topics). - Immediately dispatches a WhatsApp alert to
"little dude(s) and Angels"(120363413133092554@g.us) viawa-streamwith an explicit All-Hands Team Reminder (@Chris @Peter) to make sure Alexandra knows right away that she received a response to her Sony executive email.
- Any email from or mentioning Sony (
- Unknown Address AI Classification & 1-Tap Phone Provisioning:
- Unrecognized
*@8b.isor*@i1.isrecipients (including typos likeakex@8b.is) are classified byAlexiAIto their intended principal mailbox. - Tracks per-person device provisioning (
sovereign_device_status). If a recipient's phone is not hooked up yet (phone_provisioned == false), WhatsApp alerts automatically include 1-tap iPhone/macOS (.mobileconfigvia/api/mail/provision/apple) and Android (/api/mail/provision/android) setup links.
- Unrecognized
- Sovereign Calendar (
.ics) & VIP Contact Aggregation (.vcf):- Automatically extracts meetings, dates, and calls into
/api/mail/calendarand/api/mail/calendar.ics. - Aggregates senders and executives into
/api/mail/contactsand/api/mail/contacts.vcfwith WHOIS verification badges and VIP flags.
- Automatically extracts meetings, dates, and calls into
- TCP/43 WHOIS Anti-Privacy Payment Guard:
- Verifies every sender domain against IANA/registry WHOIS servers. Any company hiding behind a domain privacy proxy (
Domains By Proxy,Withheld for Privacy,REDACTED FOR PRIVACY) is automatically blocked from requesting payments or settling invoices.
- Verifies every sender domain against IANA/registry WHOIS servers. Any company hiding behind a domain privacy proxy (
Core SMTP Honeypot & Router
- Accept-all SMTP server - Captures all incoming emails for analysis
- Connection tracking - Logs IP addresses, reverse DNS, ASN, and country
- Authentication logging - Records all login attempts (username/password)
- Email metadata extraction - Subject, sender, recipients, size, timestamps
- Threat scoring - Automatic risk assessment based on sender reputation
Marine Compression System
- Salience detection - EMA-based algorithm identifies repetitive vs. unusual emails
- Adaptive tokenization - Per-sender vocabulary learning and pattern compression
- M8 binary format - Efficient storage with metadata headers
- Selective compression - Only compresses low-salience (repetitive) emails, preserving anomalies
Harmony DNS Module
- MX/PTR/SPF lookups - Standard DNS resolution for email validation
- Harmony checking - Jitter detection to identify DNS inconsistencies
- Emergency signaling - Lifeline protocol for DNS-based distress signals
- Tracking intelligence - Multi-hop redirect chain for emergency response
Security & Intelligence
- IP ban system - Automatic banning with configurable thresholds
- Threat intelligence - Reverse DNS, Criminal IP API integration
- 24-hour caching - Efficient reputation lookups
- Proxy/Tor/VPN detection - Network type identification
Installation
Prerequisites
- Rust 1.70+ (with cargo)
- SQLite 3.x
Build from Source
# Clone the repository
git clone https://github.com/8b-is/hoover.git
cd hoover
# Build in release mode
cargo build --release
# The binary will be at ./target/release/hoover
Quick Start
# Generate default configuration
./target/release/hoover gen-config -o hoover.toml
# Edit configuration as needed
vim hoover.toml
# Start the server
./target/release/hoover serve
Usage
CLI Commands
hoover [OPTIONS] [COMMAND]
Commands:
serve Run the SMTP server (default)
gen-config Generate example configuration file
ban-stats Show ban statistics
unban Remove an IP from the ban list
Options:
-c, --config <FILE> Configuration file path [default: hoover.toml]
-h, --help Print help
Examples
# Start server with default config
hoover serve
# Start server with custom config
hoover serve -c /etc/hoover/production.toml
# Generate config to specific location
hoover gen-config -o /etc/hoover/hoover.toml
# View ban statistics
hoover ban-stats
# Unban an IP address
hoover unban 192.168.1.100
Configuration
Hoover uses TOML configuration. Generate a default config with hoover gen-config.
[server]
bind = "0.0.0.0:25" # Listen address
hostname = "hoover.local" # SMTP banner hostname
enable_tls = false # TLS support (future)
max_message_size = 10485760 # 10MB max email size
timeout = 300 # Connection timeout (seconds)
[database]
url = "sqlite://hoover.db" # SQLite database path
max_connections = 10 # Connection pool size
[threat_intel]
enable_reverse_dns = true # PTR record lookups
enable_whois = true # WHOIS queries
criminal_ip_api_key = "" # Optional: Criminal IP API key
criminal_ip_endpoint = "https://api.criminalip.io/v1"
[ban]
duration_seconds = 86400 # 24-hour ban duration
max_failed_auth = 3 # Auth failures before ban
max_suspicious_commands = 10 # Suspicious commands before ban
[relay]
enable_relay = false # Enable legitimate relay
whitelist_path = "whitelist.toml"
forward_smtp_server = "" # Upstream relay server
forward_smtp_port = 587
forward_smtp_username = ""
forward_smtp_password = ""
[storage]
backend = "filesystem" # filesystem | s3 | mem8 | stdout
filesystem_path = "./emails" # Email storage directory
enable_compression = true # Enable Marine compression
enable_legal_hold = false # Store full headers for compliance
[compression]
enable_salience = true # Enable salience detection
salience_alpha = 0.95 # EMA adaptation rate (0.95 = slow)
salience_threshold = 0.3 # Jitter threshold (0.3 = 30%)
min_pattern_length = 4 # Minimum pattern bytes
min_pattern_frequency = 3 # Minimum occurrences to tokenize
m8_storage_path = "./m8_emails"
Architecture
┌─────────────────────────────────────────────────────────────────┐
│ Hoover SMTP Server │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ rs-smtp │───▶│ HooverBackend│───▶│ Database │ │
│ │ (SMTP) │ │ (Sessions) │ │ (SQLite) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Ban System │ │ Threat Intel │ │ Storage │ │
│ │ (Moka LRU) │ │ (DNS/WHOIS) │ │ (FS/S3/M8) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ Harmony DNS │ │ Marine │ │
│ │ (Resolver) │ │ Compression │ │
│ └──────────────┘ └──────────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ Lifeline │ │ Tokenizer │ │
│ │ (Emergency) │ │ (Per-sender) │ │
│ └──────────────┘ └──────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
Module Overview
src/smtp/ - SMTP Server
The core honeypot implementation using the rs-smtp library. Handles MAIL FROM, RCPT TO, DATA commands and integrates with all subsystems.
src/compression/ - Marine Compression
- salience.rs - EMA-based salience detection (Marine algorithm)
- tokenizer.rs - Adaptive per-sender tokenization with LRU cache
- storage.rs - M8 binary format storage
src/dns/ - Harmony DNS
- mod.rs - DNS resolver with harmony (consistency) checking
- emergency.rs - Lifeline emergency signaling protocol
- tracking.rs - Intelligence gathering redirect chain
- cloudflare.rs - CloudFlare API integration (stubs)
src/ban/ - IP Banning
Dual-layer caching (Moka in-memory + SQLite) with automatic TTL expiration.
src/intel/ - Threat Intelligence
Multi-source IP reputation: reverse DNS, WHOIS, Criminal IP API.
src/storage/ - Email Storage
Multi-backend abstraction: filesystem, S3 (stub), MEM8 (stub), stdout.
src/db/ - Database
SQLite with async sqlx. Tables: connections, auth_attempts, banned_ips, emails, whitelisted_domains, threat_intel.
src/relay/ - Email Relay
Forward legitimate emails to upstream servers (not yet integrated).
src/config/ - Configuration
TOML-based configuration with sensible defaults.
Marine Compression Algorithm
The Marine algorithm uses Exponential Moving Averages (EMA) to detect email salience:
- Extract metrics from each email: size, unique words, HTML ratio, links, images, caps ratio
- Track per-sender baselines using EMA (slow adaptation, α=0.95)
- Calculate jitter (deviation from baseline) for each metric
- Score salience as normalized average jitter
Low salience (< 0.3) → Repetitive email → Compress with tokenization
High salience (≥ 0.3) → Unusual email → Preserve uncompressed
Compression Results
Tested on 5,000 real production emails:
- 67% high salience (unusual) - preserved
- 33% low salience (repetitive) - compressed
- Average compression ratio on repetitive emails: 1.20x (20% savings)
- Best compression ratio: 2.00x (50% savings)
Lifeline Emergency Protocol
DNS-based emergency signaling for life-threatening situations:
Format: <recipient>.<action>.<identity>.<location>.<domain>
Examples:
dad.help.its.victor.c.8b.is → Victor needs help at location C
mom.sos.emergency.jane.8b.is → Jane sending SOS to mom
911.danger.now.chris.8b.is → Chris signaling authorities
Urgency Levels:
sos(10/10) - Immediate dangerdanger(9/10) - High threathelp(7/10) - Needs assistanceping(3/10) - Status checksafe(1/10) - All clear
See LIFELINE.md for full protocol documentation.
Database Schema
-- SMTP connection logs
connections (id, ip_address, hostname, reverse_dns, country, asn, connected_at, disconnected_at, commands_count, suspicious)
-- Authentication attempts
auth_attempts (id, connection_id, ip_address, username, password, mechanism, success, attempted_at)
-- IP ban list with TTL
banned_ips (id, ip_address, reason, banned_at, expires_at, ban_count)
-- Email metadata
emails (id, connection_id, from_address, to_addresses, subject, size_bytes, storage_path, compressed, threat_score, received_at)
-- Legitimate relay domains
whitelisted_domains (id, domain, forward_to, verified, added_at, last_used)
-- Cached threat intelligence
threat_intel (id, ip_address, reputation_score, threat_type, country, asn, is_proxy, is_tor, is_vpn, last_updated)
Development
Running Tests
# Run all tests
cargo test
# Run with logging
RUST_LOG=debug cargo test
# Run specific test
cargo test test_salience_detection
Testing Compression
# Build and run compression test binary
cargo run --bin test_compression
# Test with custom email directory
cargo run --bin test_compression -- --path ./test_emails
Project Structure
hoover/
├── src/
│ ├── main.rs # CLI entry point
│ ├── lib.rs # Library exports
│ ├── ban/mod.rs # IP ban system
│ ├── compression/
│ │ ├── mod.rs # Module exports
│ │ ├── salience.rs # Marine algorithm
│ │ ├── tokenizer.rs # Adaptive tokenization
│ │ └── storage.rs # M8 binary format
│ ├── config/mod.rs # TOML configuration
│ ├── db/mod.rs # SQLite database
│ ├── dns/
│ │ ├── mod.rs # Harmony resolver
│ │ ├── emergency.rs # Lifeline protocol
│ │ ├── tracking.rs # Intelligence chain
│ │ ├── cloudflare.rs # CF API stubs
│ │ ├── parser.rs # DNS parsing
│ │ ├── server.rs # DNS server
│ │ └── client.rs # DNS client
│ ├── intel/mod.rs # Threat intelligence
│ ├── relay/mod.rs # Email relay
│ ├── smtp/mod.rs # SMTP backend
│ └── storage/mod.rs # Storage backends
├── migrations/ # SQL migrations
├── rs-smtp/ # SMTP library (submodule)
├── marqant/ # Compression experiments
└── Cargo.toml
Dependencies
- tokio - Async runtime
- rs-smtp - SMTP server implementation
- sqlx - Async database with SQLite
- trust-dns-resolver - DNS resolution
- mail-parser - Email parsing
- lettre - SMTP relay transport
- moka - Concurrent LRU cache
- serde/toml - Configuration
- clap - CLI argument parsing
- tracing - Structured logging
Status
| Component | Status | Notes |
|---|---|---|
| SMTP Honeypot | ✅ Production-ready | Fully integrated |
| Database | ✅ Production-ready | Migrations included |
| Ban System | ✅ Production-ready | Dual-layer caching |
| Threat Intel | ✅ Working | API integration optional |
| Marine Compression | ✅ Production-ready | Tested on 5K emails |
| Storage | ✅ Working | Filesystem ready, S3 stub |
| Harmony DNS | 🚧 Partial | Resolver works, server stub |
| Lifeline Protocol | 🚧 Partial | Core implemented |
| Email Relay | 🚧 Not integrated | Infrastructure ready |
Documentation
README.md- This fileSTATUS.md- Detailed implementation statusHARMONY.md- Architecture philosophyLIFELINE.md- Emergency protocol specificationTRACKING_INTELLIGENCE.md- Tracking chain documentationCOMPRESSION_TEST_RESULTS.md- Compression benchmarks
License
MIT License - see LICENSE file for details.
Authors
- 8b-is - Initial work and architecture
Acknowledgments
- The Marine algorithm is inspired by consciousness research on salience detection
- Harmony DNS draws from distributed systems consensus theory
- The Lifeline protocol is designed for emergency response scenarios