Sovereign 8s.is mirror of 8b-is/Xtract
  • Rust 85.9%
  • Python 10%
  • Java 3.3%
  • Dockerfile 0.3%
  • Shell 0.3%
  • Other 0.2%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
2026-10-30 22:52:38 +01:00
bindings/extractous-python Refactor code structure for improved readability and maintainability 2025-11-21 09:01:17 -05:00
ds-ocr-example chore(ds-ocr-example): track Cargo.lock 2026-10-30 22:52:38 +01:00
extractous-core Add comprehensive entity extraction and Companies House specialized features 2025-11-21 14:36:16 +00:00
test_files Refactor code structure for improved readability and maintainability 2025-11-21 09:01:17 -05:00
.gitignore Refactor code structure for improved readability and maintainability 2025-11-21 09:01:17 -05:00
COMPANIES_HOUSE_GUIDE.md Add comprehensive entity extraction and Companies House specialized features 2025-11-21 14:36:16 +00:00
EXTRACTION_PIPELINE_ANALYSIS.md Add comprehensive extraction pipeline analysis 2025-11-21 14:14:15 +00:00
LICENSE Refactor code structure for improved readability and maintainability 2025-11-21 09:01:17 -05:00
README.md Refactor code structure for improved readability and maintainability 2025-11-21 09:01:17 -05:00

XTract8r

Overview

XTract8r is a native content extraction engine that delivers high-performance text and metadata extraction without relying on external services. The project began with a simple goal: provide a self-contained alternative to heavyweight frameworks that depend on Python ecosystems and remote APIs.

By contrast, libraries such as unstructured-io wrap numerous dependencies, inherit the limitations of the Python Global Interpreter Lock (GIL), and often require server-side infrastructure. XTract8r keeps the focus on efficient, in-process execution and scales across multiple CPU cores without GIL contention.

Key Capabilities

  • Native Rust core for predictable performance, safe concurrency, and low memory overhead.
  • Seamless fallback to Apache Tika compiled via GraalVM for broad file-format coverage.
  • Python bindings published on PyPI with a builder-style API that bypasses GIL constraints for parallel workloads.
  • Built-in OCR integration through Tesseract to extract text from scanned documents and images.
  • Extensible architecture with forthcoming bindings for additional languages.

Quick Start

Install the Python package or add the Rust crate, then choose the workflow that best fits your environment. The examples below demonstrate the core usage patterns.

Python

Extract a file to a string:

from xTract8r import Extractor

extractor = Extractor().set_extract_string_max_length(1000)
text, metadata = extractor.extract_file_to_string("README.md")

print(text)
print(metadata)

Stream large files, URLs, or byte arrays:

from xTract8r import Extractor

extractor = Extractor()

reader, metadata = extractor.extract_file("tests/quarkus.pdf")

result = []
chunk = reader.read(4096)
while chunk:
  result.append(chunk.decode("utf-8"))
  chunk = reader.read(4096)

print("".join(result))
print(metadata)

Run OCR-enabled extraction (requires Tesseract and language data, for example sudo apt install tesseract-ocr tesseract-ocr-deu on Debian-based systems):

from xTract8r import Extractor, TesseractOcrConfig

extractor = Extractor().set_ocr_config(TesseractOcrConfig().set_language("deu"))
text, metadata = extractor.extract_file_to_string("../../test_files/documents/eng-ocr.pdf")

print(text)
print(metadata)

Rust

Read a file into a string:

use xTract8r::{Extractor, PdfOcrStrategy, PdfParserConfig, TesseractOcrConfig};

fn main() {
  let mut extractor = Extractor::new().set_extract_string_max_length(1000);
  let (text, metadata) = extractor.extract_file_to_string("README.md").unwrap();

  println!("{}", text);
  println!("{:?}", metadata);
}

Stream results for large files, URLs, or in-memory buffers:

use std::io::{BufReader, Read};
use xTract8r::Extractor;

fn main() {
  let args: Vec<String> = std::env::args().collect();
  let file_path = &args[1];

  let extractor = Extractor::new()
    .set_ocr_config(TesseractOcrConfig::new().set_language("deu"))
    .set_pdf_config(PdfParserConfig::new().set_ocr_strategy(PdfOcrStrategy::OCR_ONLY));
  let mut reader = BufReader::new(stream);
  let mut buffer = Vec::new();
  reader.read_to_end(&mut buffer).unwrap();

  println!("{}", String::from_utf8(buffer).unwrap());
  println!("{:?}", metadata);
}

Perform OCR-based PDF extraction (requires Tesseract with the relevant language pack):

use xTract8r::Extractor;

fn main() {
  let file_path = "../test_files/documents/deu-ocr.pdf";

  let extractor = Extractor::new()
    .set_ocr_config(TesseractOcrConfig::new().set_language("deu"))
    .set_pdf_config(PdfParserConfig::new().set_ocr_strategy(PdfOcrStrategy::OCR_ONLY));

  let (content, metadata) = extractor.extract_file_to_string(file_path).unwrap();
  println!("{}", content);
  println!("{:?}", metadata);
}

Performance

Independent benchmarks confirm that XTract8r delivers substantial performance gains over unstructured-io while maintaining extraction quality. When processing SEC Form 10 filings, XTract8r achieved roughly 18x faster runtimes and used approximately 11x less memory.

xTract8r_speedup_relative_to_unstructured

xTract8r_memory_efficiency_relative_to_unstructured

xTract8r_memory_efficiency_relative_to_unstructured

Supported File Formats

Category Supported Formats Notes
Microsoft Office DOC, DOCX, PPT, PPTX, XLS, XLSX, RTF Legacy and modern Office formats
OpenOffice ODT, ODS, ODP OpenDocument standards
PDF PDF Includes embedded content extraction and OCR
Spreadsheets CSV, TSV Plain text spreadsheet formats
Web Documents HTML, XML Structured web content
E-Books EPUB Electronic book format
Text TXT, Markdown Plain text formats
Images PNG, JPEG, TIFF, BMP, GIF, ICO, PSD, SVG OCR support for embedded text
Email EML, MSG, MBOX, PST Body, headers, and attachment extraction
HAR HAR HTTP Archive format for web traffic analysis

//: # (| Other | ICS (Calendar), VCF (vCard) | Supports calendar and contact file formats |) //: # (| Geospatial | KML, KMZ, GeoJSON | Extracts geospatial data and metadata |) //: # (| Fonts | TTF, OTF | Extracts metadata from font files |)

Contributing

Contributions are welcome. Please open an issue or submit a pull request to propose improvements or new capabilities.

License

XTract8r is distributed under the Apache License 2.0. Refer to the LICENSE file for the full text.