CipherFS
In development · unreleased · format may change

ADR-001: Use redb instead of SQLite

Status

Accepted

Context

CipherFS needs an embedded database to store:

  • File metadata (paths, sizes, hashes, etc.)
  • Chunk indices
  • Version history
  • Operation logs

Initially considered using SQLite (the industry standard), but there are some issues in the Rust ecosystem.

Decision

Use redb as the metadata storage.

Considered Options

1. SQLite

Pros:

  • Industry standard, battle-tested
  • Rich SQL query capabilities
  • Extensive documentation and community support

Cons:

  • Requires C compiler and FFI bindings
  • Complex cross-platform compilation (especially Android/iOS)
  • rusqlite bindings increase compilation time
  • Complex multi-database sharding solution

2. sled

Pros:

  • Pure Rust
  • fetch_and_update atomic operations
  • watch_prefix subscription feature
  • Concurrent write support

Cons:

  • High memory usage
  • Version 1.0 not yet released
  • Stability concerns

3. redb (Selected)

Pros:

  • Pure Rust, no C dependencies
  • Single-file storage
  • ACID transaction guarantees
  • Zero-copy reads, excellent performance
  • Clean API, type-safe
  • Simple cross-platform compilation

Cons:

  • Relatively new
  • Single-writer, multi-reader (no concurrent writes)
  • No SQL query capabilities

4. RocksDB / LMDB

Pros:

  • Extremely high performance
  • Battle-tested

Cons:

  • Requires C/C++ dependencies
  • Complex configuration
  • Too heavyweight

Rationale

  1. Pure Rust ecosystem: Project goal is pure Rust implementation, avoiding C dependencies simplifies compilation and cross-platform support
  2. Single-file storage: Easy to backup, migrate, and synchronize
  3. ACID transactions: Ensure atomicity of operations like reference counting
  4. Sufficient performance: For metadata storage scenarios, redb performance is more than adequate

Consequences

Pros

  • Simple compilation, just cargo build
  • Cross-platform support (Linux/macOS/Windows/Android/iOS)
  • Single-file database, easy to manage
  • Cross-table transactions ensure data consistency

Cons

  • No concurrent writes (single-writer, multi-reader)
  • No SQL queries, complex queries need manual implementation
  • Relatively small community

Mitigation

  • Concurrent writes: Queue handling at application layer
  • Complex queries: Design reasonable table structures and indexes
  • [Deprecated] multi_database_sharding.md - SQLite sharding solution (deprecated)

Source file in the repository