Compression Engine|
In Development

Meridian

Middle-out compression. Zero compromises.

Source Code

Workspace

Architecture

Middle-out

Parallelism

mmap

I/O Model

Overview

Meridian is a lossless compression engine that produces seekable .mrd files. It splits input into blocks, compresses them in parallel using a middle-out strategy, and applies adaptive context mixing for optimal compression ratios. The workspace architecture separates the core library from the CLI, with criterion benchmarks for continuous performance validation.

Stack

Rust 1.85rayonxxhash-rustmemmap2bincodecriterion

Features

What it does.

Compression

  • Parallel middle-out block compression
  • Adaptive context mixing for optimal ratios
  • LZ hash tables for pattern matching
  • XXH32 checksums for data integrity
  • Seekable .mrd block format

Performance

  • Work-stealing parallelism via rayon
  • Memory-mapped file I/O for zero-copy access
  • Criterion benchmarks for regression detection
  • Block descriptor table for random access

Format (.mrd)

  • File header with magic bytes and version
  • Block descriptor table for seeking
  • Compressed block data with checksums
  • Footer with metadata and integrity verification

Architecture

How it's built.

1

meridian-core

Compression engine: block splitting, entropy encoding, mixer, models, pipeline

2

meridian-cli

Command-line interface with progress display via indicatif

3

Rayon thread pool

Work-stealing parallelism across CPU cores

4

memmap2

Zero-copy memory-mapped file access for large inputs

Interested in this project?

Get in touch →
← Back to all projects