Open SourceIn Development

Rustqlite

A faithful, from-scratch reimplementation of SQLite3 in Rust — byte-compatible on-disk format, a C-API-mirroring library, and a sqlite3-style shell.

View on GitHub
Rustqlite
RoleCreator & Maintainer
TimelineIn Development
Technologies
RustTokioSQLitepestclap

Overview

Rustqlite is a faithful, from-scratch reimplementation of SQLite3 in Rust — not bindings to libsqlite3. The goal is an engine whose internal architecture mirrors upstream SQLite, whose public library API mirrors the SQLite C API, whose CLI mirrors the sqlite3 shell, and whose on-disk format is byte-compatible: it opens and correctly reads and writes .db files created by C SQLite.

The compatibility target is SQLite 3.53.1, and the on-disk format is stable across all of SQLite 3.x.

Architecture

The project is a Cargo workspace of three crates with a strict dependency direction — rustqlite (CLI) → rustsqlite-core → rustqlite-parser:

  • rustqlite-parser — SQL text to AST. A pest PEG grammar ported from SQLite’s parse.y, with expression precedence handled by pest’s PrattParser. No engine dependency.
  • rustsqlite-core — the core engine and the public, C-API-mirroring library. Async on tokio.
  • rustqlite — the shell (binary rustsqlite), using clap derive for flags and dot-commands dispatched in the REPL.

Rustqlite deliberately mirrors SQLite’s internal layering — tokenizer, VDBE bytecode VM, B-tree, pager and WAL, record codecs, VFS — so the implementation can be checked against the upstream C source file-by-file.

Async model

The VFS and pager I/O are async on tokio, but the sqlite3_* C-API functions keep their synchronous signatures and drive the async engine to completion via a process-global runtime. The public surface stays C-API-faithful while I/O is async underneath. Concurrency stays sqlite3-compatible — many readers, single writer — with tokio adding async I/O and parallel connections rather than new SQL semantics.

Technical Details

  • Written almost entirely in Rust (with a small amount of shell)
  • Built bottom-up so each layer is verified against real SQLite before the next: the read query path and write path (with rollback journal and crash recovery) both land before indexes and the planner
  • Differential tests run the engine’s output against C SQLite, and SQLite’s own sqllogictest corpora are run out-of-tree
  • Licensed under Apache-2.0

Rustqlite reflects my fascination with systems programming, database internals, and rebuilding foundational software in Rust.