phg/fuse-mail-dir-by-tag
0
0
Fork 0
Code Issues 10 Pull requests Projects Releases Packages Wiki Activity Actions

docs: research Linux FUSE implementation stack

Browse source
This commit is contained in:
Philip Henning 2026-07-23 00:20:12 +02:00
commit b6048de04f
1 changed files with 139 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

139
docs/research/linux-fuse-message-stack.md Normal file
View file

@ -0,0 +1,139 @@
# Linux FUSE and message-parsing implementation stack
Research date: 2026-07-23
## Decision
Implement the daemon in stable Rust and use this stack:
| Concern | Choice | Baseline examined |
| --- | --- | --- |
| Linux FUSE protocol | [`fuser`](https://docs.rs/fuser/0.17.0/fuser/) with default features (no `libfuse` feature) | 0.17.0 |
| RFC message headers and address lists | [`mailparse`](https://docs.rs/mailparse/0.16.1/mailparse/) | 0.16.1 |
| Filesystem change notifications | [`inotify`](https://docs.rs/inotify/0.11.4/inotify/) directly, on a dedicated blocking thread | 0.11.x |
| Persistent index | [`rusqlite`](https://github.com/rusqlite/rusqlite) linked to the Nix-provided SQLite library | current compatible release |
| Concurrency | `fuser` worker threads plus one bounded, single-consumer state coordinator; immutable in-memory read snapshots | Rust standard library primitives |
| Build and deployment | Cargo lockfile; Nixpkgs `rustPlatform.buildRustPackage`; Linux-only package and NixOS module | stable Rust supported by the selected `fuser` release |
Pin exact crate versions and checksums in `Cargo.lock`, but accept compatible patch updates after the integration suite passes. Keep each third-party API behind a small internal adapter so that parser, watcher, FUSE, or database bindings can be replaced independently.
## Why this fits
### Rust and `fuser`
`fuser` exposes the inode-oriented FUSE operations needed here, including `lookup`, `read`, `readdir`, `rename`, `setattr`, and `unlink`; its `Filesystem` implementation must be `Send + Sync`, matching a daemon that serves concurrent kernel requests ([`Filesystem` trait](https://docs.rs/fuser/0.17.0/fuser/trait.Filesystem.html)). On Linux it can replace the userspace portion of libfuse and can be built without the optional `libfuse` feature, reducing the runtime/build surface to the kernel FUSE device and mount tooling ([upstream README](https://github.com/cberner/fuser/blob/v0.17.0/README.md)).
Version 0.17 has explicit worker-count and cloned-FUSE-fd configuration. Multiple event-loop threads are supported on Linux, and cloned fds require Linux 4.5 or later ([`Config`](https://docs.rs/fuser/0.17.0/fuser/struct.Config.html)). This allows blocking local file reads to proceed concurrently without adding an async runtime. Start with a small fixed worker count (for example four), make it configurable, and measure before increasing it.
External deliveries and flag renames change entries behind the mounted view. `fuser::Notifier` supports invalidating a directory entry or inode and reporting deletion to the kernel, which is required in addition to publishing a new userspace snapshot ([`Notifier`](https://docs.rs/fuser/0.17.0/fuser/struct.Notifier.html)). Use short, conservative entry/attribute TTLs as a fallback; notifications are the primary cache-coherency mechanism.
Rust is a good fit because the design combines concurrent FUSE callbacks, borrowed header buffers, path and inode identity, and serialized mutation. Its ownership and `Send`/`Sync` checking make accidental unsynchronized sharing a compile-time problem, while the standard library supplies multi-producer/single-consumer channels and read/write locks ([Rust shared-state concurrency](https://doc.rust-lang.org/book/ch16-03-shared-state.html), [`std::sync`](https://doc.rust-lang.org/std/sync/)). This does not eliminate filesystem logic errors or deadlocks, so integration testing remains mandatory.
### `mailparse` for a header-only scan
The projection needs only the RFC message header block, not MIME bodies. `mailparse::parse_headers` consumes raw bytes through the header/body separator, returns every header plus the body offset, and explicitly permits callers that only care about headers to ignore the body offset ([`parse_headers`](https://docs.rs/mailparse/0.16.1/mailparse/fn.parse_headers.html)). Its header-map API performs case-insensitive lookup and can return every repeated value in source order ([`MailHeaderMap`](https://docs.rs/mailparse/0.16.1/mailparse/trait.MailHeaderMap.html)).
For every `Delivered-To`, `X-Original-To`, `To`, and `Cc` header selected by the specification's precedence rule, call `addrparse_header` on the original `MailHeader`; upstream documents that this avoids correctness problems caused by decoding encoded words before splitting an address list ([`addrparse_header` source and documentation](https://docs.rs/mailparse/0.16.1/mailparse/fn.addrparse_header.html), [`addrparse` caveat](https://docs.rs/mailparse/0.16.1/mailparse/fn.addrparse.html)). Flatten both individual and group addresses, then apply the project's local-part rule: everything after the first `+`, with an empty suffix rejected.
`mailparse` is intentionally tolerant of real Maildir data and states that it may accept non-strict input such as LF-only messages ([upstream README](https://github.com/staktrace/mailparse#api)). That matches the requirement to omit and log malformed messages rather than fail the mount, but it means tests—not a claim of strict rejection—must define the accepted edge cases.
Read only until the first blank line and impose a configurable maximum header size before parsing. This prevents a malformed or hostile message with no terminator from causing unbounded allocation. Never parse or retain message bodies during indexing; reads from the mounted file should stream from the source file.
### Direct `inotify`
The source is a Linux Maildir with a small, known directory set (`cur` and `new`; `tmp` is ignored), so the cross-platform normalization of a higher-level watcher is unnecessary. The Rust `inotify` crate is an idiomatic wrapper that offers blocking reads and exposes the kernel event mask, filename, and rename cookie ([crate documentation](https://docs.rs/inotify/0.11.4/inotify/), [`Event`](https://docs.rs/inotify/0.11.4/inotify/struct.Event.html)). Run it on one dedicated thread and submit compact change commands to the state coordinator.
Watch both `cur` and `new` for create, close-write, delete, and move events. The Linux API gives matching `IN_MOVED_FROM`/`IN_MOVED_TO` events a shared cookie, including moves between watched directories ([inotify(7)](https://man7.org/linux/man-pages/man7/inotify.7.html#DESCRIPTION)). Do not assume the pair is adjacent or atomically enqueued; the kernel documentation explicitly warns that matching rename events is racy and one side may be absent ([inotify rename caveats](https://man7.org/linux/man-pages/man7/inotify.7.html#Dealing_with_rename()_events)). Treat unmatched or timed-out rename halves as removal/addition and reconcile the affected directories.
`IN_Q_OVERFLOW` means events were dropped, and the Linux documentation recommends rebuilding the cache when this happens. Inotify is non-recursive and can miss changes on remote/network filesystems ([inotify limitations](https://man7.org/linux/man-pages/man7/inotify.7.html#NOTES)). Therefore:
- reject or clearly document unsupported network-backed source Maildirs in version 1;
- on overflow, watch invalidation, an unmatched event, or any stat/index contradiction, keep serving the last complete snapshot, rebuild from `cur` and `new`, then publish the new snapshot atomically;
- periodically run a low-frequency reconciliation as defense in depth, because the kernel manual recommends consistency checking even for carefully written monitors ([inotify cache guidance](https://man7.org/linux/man-pages/man7/inotify.7.html#DESCRIPTION)).
### SQLite as a disposable, persistent index
The source Maildir is authoritative. Store only derived metadata: source identity and current relative path, stat fingerprint, parsed tag values, reversible projected names, stable projected inode assignments, and the parser/schema version. Do not store message bodies. A normalized `messages`, `tags`, and `message_tags` schema supports 100,000 messages and 10,000 tags without loading bodies and permits indexed consistency checks.
Use one `rusqlite::Connection`, owned exclusively by the state coordinator. FUSE read callbacks consult an immutable in-memory snapshot rather than querying SQLite. This makes request latency independent of database locks, while the database preserves parser results and stable inode assignments between restarts. `rusqlite` normally finds a system SQLite through `pkg-config`; its `bundled` feature instead compiles an embedded SQLite copy ([upstream build notes](https://github.com/rusqlite/rusqlite#notes-on-building-rusqlite-and-libsqlite3-sys)). For Nix, prefer the system library so Nixpkgs controls security fixes and there is only one SQLite version in the closure.
Use SQLite's rollback-journal mode (`journal_mode=DELETE`), `synchronous=FULL`, `foreign_keys=ON`, and explicit transactions. SQLite documents that transactions appear atomic even across an operating-system crash or power failure ([SQLite atomic commit](https://sqlite.org/atomiccommit.html#_introduction)). WAL's extra reader/writer concurrency is unnecessary because only the coordinator touches the database; WAL also adds checkpoint management and extra `-wal`/`-shm` files ([SQLite WAL overview](https://sqlite.org/wal.html#_overview)). If a later design introduces multiple SQLite connections and WAL, require a SQLite release containing the 2026 WAL-reset fix (3.51.3 or the documented backports) and test busy/checkpoint behavior ([SQLite WAL-reset advisory](https://sqlite.org/wal.html#walreset)).
At startup:
1. Open/migrate the database.
2. Scan `cur` and `new` completely. Reuse cached parser results only when the stored fingerprint still matches; otherwise re-read and parse the bounded header block.
3. Reconcile all rows inside a transaction and construct the complete immutable in-memory snapshot.
4. Only after commit and snapshot construction succeeds, mount FUSE and signal systemd readiness.
For an incremental event or FUSE mutation, the state coordinator performs the authoritative Maildir rename/unlink first, commits the corresponding index change second, publishes one replacement snapshot third, and finally sends kernel invalidations. A crash can occur between the source operation and the index commit, so cross-filesystem atomicity is not claimed; the next reconciliation repairs the disposable index from the authoritative Maildir.
## Concurrency and component boundary
Use this ownership model:
```text
fuser worker callbacks ----\
> bounded command queue -> state coordinator -> source Maildir
inotify watcher thread ----/ | \
| -> SQLite connection
v
immutable view snapshot
|
fuser readers + Notifier
```
- FUSE lookup/read/readdir/getattr operations take a short-lived read handle to the current immutable snapshot and access source files through validated file handles.
- FUSE rename/unlink/setattr operations send a command and wait for the coordinator's result before replying to the kernel. This serializes global flag and deletion semantics across every projected tag path.
- The watcher uses the same command queue, so external Dovecot/Postfix changes and writes through the projection have one ordering point.
- Use a bounded channel for backpressure; Rust's `sync_channel` is explicitly bounded and supports cloned producers with one receiver ([`sync_channel`](https://doc.rust-lang.org/std/sync/mpsc/fn.sync_channel.html)). If it fills or the watcher cannot submit without jeopardizing liveness, mark the state dirty and schedule reconciliation rather than silently dropping semantic work.
- Do not hold the snapshot lock or a FUSE inode lock while waiting for the coordinator or sending a kernel notification. This rule must be covered by concurrency tests.
The immutable snapshot should contain compact identifiers, paths, flags, file size/timestamps, tag-to-message membership, and directory enumeration order. Memory is therefore proportional to messages plus tag memberships, not message bytes. Verify the 100,000-message/10,000-tag target with a generated corpus and measure startup time, steady-state resident memory, concurrent `readdir`, flag-renames, and deletion.
## Nix and systemd implications
Package with a checked-in `Cargo.lock` and Nixpkgs `rustPlatform.buildRustPackage`; link `rusqlite` to the Nix-provided SQLite and leave `fuser`'s `libfuse` feature disabled. The runtime still needs Linux FUSE support and appropriate mount setup. Current NixOS releases make the `programs.fuse` module opt-in, so the NixOS module must enable/provide the required `fusermount3` tooling or mount with the service's explicitly granted privileges ([NixOS release note](https://nixos.org/manual/nixos/unstable/release-notes#sec-release-26.11)).
The service should use a per-instance systemd state directory for the SQLite file, start only after source and mountpoint paths exist, remain in the foreground, log to stderr/journald, unmount on orderly shutdown, and report ready only after startup reconciliation and successful mount. systemd/NixOS already models ordering between services and mount units, and custom NixOS services can declare `after`, `requires`, `before`, and `wantedBy` relationships ([NixOS service manual](https://nixos.org/manual/nixos/stable/#sec-systemd)).
## Viable alternatives considered
### `fuse3` (Rust async FUSE)
`fuse3` 0.9.0 is actively released and supports unprivileged `fusermount3`, `readdirplus`, POSIX locks, and async direct I/O; upstream still lists `ioctl` and fuseblk as unsupported and `poll` as unstable ([upstream README](https://github.com/Sherlock-Holo/fuse3)). It is viable, but its async-first API and runtime are unnecessary for a local Maildir/SQLite workload whose mutations are deliberately serialized. At the examined release, docs.rs also failed to build its 0.9.0 API documentation ([docs.rs release page](https://docs.rs/crate/fuse3/0.9.0)), increasing adoption risk. Reconsider it if measurements show that blocking FUSE workers are a bottleneck or the backend later becomes remote.
### `go-fuse/v2` (Go)
`go-fuse/v2` is the strongest language-level alternative. It has raw and higher-level inode APIs, normally handles each request in its own goroutine, supports kernel invalidation, and its higher-level API explicitly supports hard links and immutable node IDs ([raw package](https://pkg.go.dev/github.com/hanwen/go-fuse/v2/fuse), [higher-level `fs` package](https://pkg.go.dev/github.com/hanwen/go-fuse/v2/fs)). Choose it instead if the maintainers are substantially stronger in Go. It is not selected because the Rust stack already supplies the exact header-only parser and direct inotify APIs required, avoids a garbage-collected runtime in a long-lived metadata-heavy daemon, and allows one language-level ownership model across borrowed input, inode state, and FUSE callbacks. This is a preference, not a claim that `go-fuse` is incapable.
### C/C++ with libfuse
libfuse is the reference userspace implementation and offers a synchronous high-level API plus an asynchronous low-level API ([libfuse API](https://libfuse.github.io/doxygen/)). It is technically complete for this task. It is rejected because implementing the same parser, concurrent inode state, and crash-safe ownership model in a memory-unsafe language adds avoidable risk, while `fuser` exposes the required low-level operations and cache notifications. libfuse's own README also says it currently has no active regular contributors beyond a maintainer applying pull requests and making releases, so using the C reference does not remove upstream-maintenance risk ([libfuse README](https://github.com/libfuse/libfuse#development-status)).
### `mail-parser` instead of `mailparse`
Stalwart's `mail-parser` is a strong maintained parser: upstream describes RFC 5322/MIME coverage, zero-copy strings, fuzzing, and use against millions of messages ([crate documentation](https://docs.rs/mail-parser/0.11.5/mail_parser/)). It is preferable for applications that need a rich parsed MIME model. This daemon needs repeated arbitrary header lookup and header-only address parsing; `mailparse::parse_headers`, `get_all_headers`, and `addrparse_header` directly match that narrower job. Keep corpus tests parser-neutral so this choice can be revisited if `mailparse` fails on deployed messages.
### `notify` instead of direct `inotify`
`notify` is maintained and its Linux backend translates queue overflow into a `Rescan` event ([backend source](https://docs.rs/notify/latest/src/notify/inotify.rs.html)). It is the right default for cross-platform applications. This service is explicitly Linux-only and benefits from direct access to rename cookies and exact overflow/watch-invalidated masks, so the lower-level `inotify` wrapper is simpler at the semantic boundary despite requiring more careful code.
### No persistence, flat files, or an embedded key-value store
An in-memory-only index would force reparsing every message on every boot. A custom JSON/CBOR snapshot can be atomically replaced, but schema migration, referential checks, partial-update handling, and indexed lookups become application code. RocksDB is far heavier than the dataset and introduces a C++ dependency. SQLite supplies transactions, integrity checks, compact indexes, and mature recovery in one system package; treating it as a rebuildable cache limits the consequence of any database failure.
## Risks and acceptance work
1. **Dovecot exercises Maildir behavior, not a crate API.** Add black-box tests with Dovecot against the mounted filesystem for `new` to `cur` transitions, `S/F/R/T/D` flag renames, concurrent views of the same source message, copy-then-delete moves, expunge, restart, and forced unmount.
2. **FUSE cache invalidation can make correct state look stale.** Test notifier calls and TTL behavior under an already-open Dovecot indexer; never publish new state without invalidating every affected projected dentry/inode.
3. **Duplicate projected paths can race.** Route all mutations through the coordinator and make operations idempotent. A second delete should return the documented stale/not-found error, not delete an unrelated reused inode.
4. **Inotify is advisory, not a transaction log.** Test queue overflow, missing rename halves, watch deletion, burst delivery, and changes while reconciliation is scanning. Every such path must converge through a full reconciliation.
5. **Parser tolerance is part of observable behavior.** Maintain a corpus containing folded headers, repeated delivery headers, address groups, comments, quoted local parts, UTF-8/encoded display names, LF-only messages, malformed addresses, missing separators, and oversized headers. Only the extracted routing address—not the display name—feeds tag extraction.
6. **Maildir and SQLite cannot commit atomically together.** Preserve the source-first/disposable-index rule and inject crashes between each step to prove restart repair.
7. **Upstream churn remains possible.** Pin the lockfile, run dependency/security updates routinely, and isolate `fuser`, parser, watcher, and storage adapters. The complete Dovecot integration suite is the upgrade gate.
8. **The scale target needs measurement.** Acceptance requires 100,000 messages and 10,000 distinct tags, bounded headers, no body cache, and recorded startup latency/RSS. A complexity argument alone is insufficient.
## Resulting implementation constraint
The implementation specification should name this stack and ownership model, but should not expose crate types in the domain model. The stable internal seam is: a header scanner yields raw tags; a source catalog yields authoritative messages; an index persists derived metadata; an immutable projection answers reads; and one mutation coordinator owns all source/index changes. That boundary keeps the route open if any selected library needs replacement.