Struct VolumeIndex 

pub struct VolumeIndex {

    pub(super) lower_pool: Vec<u8>,
    pub(super) orig_pool: Vec<u8>,
    pub(super) orig_off: Vec<u32>,
    pub(super) name_off: Vec<u32>,
    pub(super) name_len: Vec<u16>,
    pub(super) parent: Vec<EntryId>,
    pub(super) size_lo: Vec<u32>,
    pub(super) size_ovf: FxHashMap<EntryId, u64>,
    pub(super) mtime: Vec<i64>,
    pub(super) frn: Vec<u64>,
    pub(super) flag: Vec<u8>,
    pub(super) frn_index: FrnIndex,
    pub(super) perm_name: Vec<EntryId>,
    pub(super) content_generation: u64,
    pub(super) structural_generation: u64,
    pub(super) dir_topology_generation: u64,
    pub(super) tombstones: u32,
    pub(super) dead_name_bytes: u64,
    pub(super) derived_cache: Mutex<Option<DerivedCache>>,
}

In-memory per-volume index.

Struct-of-arrays entry columns, two string pools sharing one offset/length table, an FRN map, and the always-sorted name permutation (docs/ARCHITECTURE.md). One instance per indexed volume.

Fields

lower_pool: Vec<u8>

The one contiguous, sweepable pool: every entry’s folded name at name_off..name_off+name_len. Most names fold to themselves (ADR-0004), so the original spelling is stored only where it differs — in orig_pool at orig_off, same length (the fold is length-preserving, wtf8.rs). orig_off == u32::MAX means the folded bytes are the original.

orig_pool: Vec<u8>

orig_off: Vec<u32>

name_off: Vec<u32>

name_len: Vec<u16>

parent: Vec<EntryId>

size_lo: Vec<u32>

File sizes < u32::MAX, 4 bytes per entry; u32::MAX is the sentinel for the overflow map (≥4GiB files, ADR-0007). Read through VolumeIndex::size.

size_ovf: FxHashMap<EntryId, u64>

mtime: Vec<i64>

frn: Vec<u64>

flag: Vec<u8>

frn_index: FrnIndex

perm_name: Vec<EntryId>

The one always-maintained permutation: name order is the default sort and the merge target of every USN batch. Size/mtime orders are lazily derived caches (query::memo::{SizePerm, MtimePerm}) — built on the first sorted query, extended per generation, never persisted.

content_generation: u64

structural_generation: u64

dir_topology_generation: u64

Bumped whenever an existing directory’s name or parent changes — the two mutations that invalidate memoized descendant paths in ways an append-only extension cannot express. Plain appends/deletes/stat updates leave it untouched.

tombstones: u32

dead_name_bytes: u64

Abandoned name bytes across both pools (tombstoned rows and in-place dir renames leave their old bytes behind: folded copy always, the original copy when one existed). Compaction-trigger input. Not persisted — recomputed from tombstones on restore, so rename gaps make it a lower bound there.

derived_cache: Mutex<Option<DerivedCache>>

Query-independent caches derived from index content (dir-path memo, pool offset table, …) keyed by content_generation and value type. Type-erased so the index stays ignorant of query-module types.

Implementations

impl VolumeIndex

pub fn compaction_due(&self) -> bool

Should this index be compacted? (Policy entry point for the volume thread, once per applied USN batch.)

fn compaction_due_past(&self, min_entries: usize) -> bool

pub fn compacted(&self) -> Self

A compacted copy: live entries only, pools rebuilt without garbage, permutation and FRN index remapped without re-sorting. Children of a tombstoned directory attach to the root — the same orphan policy as push_raw. The copy starts at generation zero on all three counters; install_index carries the structural generation forward.

Call only at a batch boundary (the FRN index must cover every entry — merge_new_into_permutations just ran).

impl VolumeIndex

pub const ROOT: EntryId = 0

The volume root’s EntryId (always slot 0).

pub const fn len(&self) -> usize

Total entry slots, live plus tombstoned (the column length).

pub const fn is_empty(&self) -> bool

True when no entries have ever been appended.

pub const fn live_len(&self) -> usize

Live entry count: total slots minus tombstones.

pub fn name(&self, id: EntryId) -> &[u8]

The original-spelling name. Fold-identical entries (most of them) borrow straight from the folded pool — the bytes are the same.

pub fn lower_name(&self, id: EntryId) -> &[u8]

The case-folded name bytes of id (ADR-0004), straight from the folded pool — the form every matcher compares against.

pub fn is_live(&self, id: EntryId) -> bool

True while id is a real entry — false once it has been tombstoned.

pub fn is_excluded(&self, id: EntryId) -> bool

Hidden/system (or under such a branch) — skipped by default queries.

pub fn is_dir(&self, id: EntryId) -> bool

True when id is a directory rather than a file.

pub fn is_reparse(&self, id: EntryId) -> bool

True when id is a reparse point (symlink, junction, mount point).

pub fn size(&self, id: EntryId) -> u64

File size of id in bytes, read through the u32 column and the overflow map for ≥4 GiB files (ADR-0007).

pub(super) fn set_size(&mut self, id: EntryId, v: u64)

The single write path for sizes — keeps the column and the overflow map consistent in both directions (a file can shrink back under the sentinel).

pub(super) fn push_size(&mut self, v: u64)

Append form of Self::set_size (column construction).

pub fn mtime(&self, id: EntryId) -> i64

Last-modification time of id as a Windows FILETIME tick count.

pub fn parent(&self, id: EntryId) -> EntryId

The EntryId of id’s parent directory (NO_PARENT at the root).

pub fn frn(&self, id: EntryId) -> Frn

The NTFS File Reference Number of id.

pub fn entry_by_record(&self, record: impl Into<RecordNo>) -> Option<EntryId>

The live entry for a record number, if any. Pass a RecordNo (or a raw record-number u64); derive one from a full reference with Frn::record — the type stops a full FRN being mistaken for a key.

pub(crate) fn name_off_of(&self, id: EntryId) -> u32

pub(crate) fn name_len_of(&self, id: EntryId) -> usize

pub(crate) fn is_fold_identical(&self, id: EntryId) -> bool

True when the entry’s original spelling is its folded form — the case-exact matchers’ fast path: such a name can never contain a needle with fold-unstable characters, and for fold-stable needles the folded comparison is the exact comparison.

pub(crate) fn lower_pool_bytes(&self) -> &[u8]

pub const fn content_generation(&self) -> u64

The content generation — bumped by every USN batch; open result handles stay readable across it (docs/ARCHITECTURE.md, generation 2-tier).

pub const fn structural_generation(&self) -> u64

The structural generation — bumped only by compaction/rebuild, which hard-stales open result handles (docs/ARCHITECTURE.md, generation 2-tier).

pub(crate) const fn dir_topology_generation(&self) -> u64

pub(crate) const fn bump_structural_from(&mut self, prev: u64)

Carry the structural generation across a rebuild: a freshly built index replacing one whose generation was prev must read as strictly newer, so open result handles go hard-stale (docs/ARCHITECTURE.md, generation 2-tier). Compaction (M2) will reuse this.

pub(crate) fn cached_derived<T, F>(&self, build: F) -> Arc<T>

where T: Any + Send + Sync, F: FnOnce() -> T,

Return the cached content-derived value of type T, rebuilding it with build when the content generation moved. All cached types are invalidated together on a generation change.

pub(crate) fn cached_derived_or_update<T, F>(&self, build: F) -> Arc<T>

where T: Any + Send + Sync, F: FnOnce(Option<Arc<T>>) -> T,

Like Self::cached_derived, but on a generation change build receives the previous generation’s value so it can extend it incrementally instead of rebuilding from scratch.

pub(crate) fn derived_probe<T: Any + Send + Sync>(&self) -> Option<Arc<T>>

Read-only probe of the current generation’s cached T — never builds. For memory accounting (IndexStats.derived_cache_bytes).

fn with_derived<T, F>(&self, build: F) -> Arc<T>

where T: Any + Send + Sync, F: FnOnce(Option<Arc<T>>) -> T,

pub fn stats(&self, volume: &str) -> IndexStats

Per-column memory accounting for the perf panel / fmf stats. The map size is an estimate (hashbrown control bytes + slot padding).

pub fn shrink_to_fit(&mut self)

Trim over-allocated columns after a bulk build.

pub fn name_permutation(&self) -> &[EntryId]

The always-maintained name-sorted permutation: entry ids in default (folded-name) sort order, the merge target of every USN batch.

pub fn append_path(&self, id: EntryId, out: &mut Vec<u8>)

Append the full WTF-8 path of id (“C:\dir\file.txt”) to out. Built lazily from the parent chain — paths are never stored.

pub fn append_parent_path(&self, id: EntryId, out: &mut Vec<u8>)

Append the path of id’s parent directory, including a trailing \.

pub fn tombstone_ratio(&self) -> f64

Fraction of slots that are tombstones (0.0–1.0) — the compaction trigger input. 0.0 for an empty index.

pub(crate) fn cmp_by(&self, key: SortKey, a: EntryId, b: EntryId) -> Ordering

The one definition of each sort key’s strict total order (id tie-break) — pub(crate) so the lazy permutation caches in the query layer sort by exactly the same order the merge maintains.

pub(super) fn sort_columns(&self) -> SortColumns<'_>

impl VolumeIndex

fn owned_name_bytes(&self, id: EntryId) -> u64

Pool bytes id owns that compaction could reclaim: the folded copy always, plus the original copy when one exists.

pub fn upsert(&mut self, e: &RawEntry<'_>) -> EntryId

Insert or replace an entry for record. Replacement (same record number) tombstones the old entry — this is how renames work. Returns the new id. Caller must finish the batch with Self::merge_new_into_permutations.

pub fn delete(&mut self, record: impl Into<RecordNo>) -> Option<EntryId>

Tombstoning is the whole deletion: the FRN index never finds dead entries (liveness filter), so there is nothing to unmap.

pub fn reparent( &mut self, record: impl Into<RecordNo>, new_parent_record: impl Into<RecordNo>, ) -> Option<EntryId>

Move record under a new parent. Cheap: no permutation depends on the path, and child paths rebuild lazily. A corrupt record naming itself as parent keeps its current parent (no self-cycles).

pub fn rename_dir_in_place( &mut self, record: impl Into<RecordNo>, name_utf16: &[u16], new_parent_record: impl Into<RecordNo>, ) -> Option<EntryId>

Rename/move a directory in place. Directories must keep their EntryId stable — children’s parent fields point at it — so instead of tombstone+new (the file path), the name is swapped and the entry is repositioned inside perm_name. O(len) per rename; directory renames are rare enough that this beats invalidating every child.

pub fn update_stat( &mut self, record: impl Into<RecordNo>, size: u64, mtime: i64, ) -> Option<EntryId>

Update size/mtime in place (USN_REASON_DATA_* without a name change).

pub fn merge_new_into_permutations(&mut self, first_new: EntryId)

Merge entries first_new..len (already appended, unsorted) into the name permutation (in place — see merge_sorted_tail), then bump the content generation. Call once per USN batch. The lazy size/mtime permutation caches catch up on their next sorted query (the generation bump is their invalidation signal).

fn push_orig_if_differs(&mut self, lower_off: usize, orig: &[u8]) -> u32

Store the original spelling only when it differs from the folded bytes just appended at lower_off — the fold-identical majority costs nothing beyond the sentinel (ADR-0004).

pub(super) fn push_raw(&mut self, e: &RawEntry<'_>, parent: EntryId) -> EntryId

Append with a pre-resolved parent: the USN path resolves against the live index (see Self::upsert); the initial-scan builder passes a provisional ROOT because finish() re-resolves every parent anyway — a per-push lookup against the unmerged FRN tail would be O(n²) there.

pub(super) fn push_encoded( &mut self, e: &EncodedEntry<'_>, parent: EntryId, ) -> EntryId

fn push_columns( &mut self, off: usize, orig_off: u32, parent: EntryId, frn: Frn, size: u64, mtime: i64, is_dir: bool, is_reparse: bool, is_hidden: bool, is_system: bool, ) -> EntryId

Shared column append after the name bytes already landed in the pools at off/orig_off. The flag/parent logic must stay identical between the utf16 (push_raw) and pre-encoded (push_encoded) entry points.

pub(super) fn recompute_excluded(&mut self, id: EntryId)

Re-derive EXCLUDED for id from its own H/S bits and current parent.

pub fn update_attrs( &mut self, record: impl Into<RecordNo>, is_hidden: bool, is_system: bool, ) -> Option<EntryId>

Update raw attribute bits (USN BASIC_INFO_CHANGE) and the derived EXCLUDED bit.

impl VolumeIndex

pub fn write_snapshot<W: Write>( &self, w: &mut W, journal_id: u64, next_usn: i64, ) -> Result<()>

Serialize the index plus the USN checkpoint (journal_id, next_usn).

Errors

Propagates any I/O error from writing to w.

pub fn read_snapshot<R: Read>(r: &mut R) -> Result<(Self, u64, i64)>

Load a snapshot; returns the index and the persisted (journal_id, next_usn) checkpoint. Any structural or checksum mismatch is an error — callers fall back to a full rescan.

Errors

Returns an std::io::Error: any underlying read error, or InvalidData on a bad magic, checksum mismatch, or any structural inconsistency in the (untrusted) stream.

Panics

Panics only on an internal invariant violation: the fixed-size header conversions assume the 32-byte header buffer, which is always read in full before they run.

pub fn save_to(&self, path: &Path, journal_id: u64, next_usn: i64) -> Result<()>

Atomic save: write to <path>.tmp, then rename over the target.

Errors

Propagates any I/O error from creating the parent directory, writing the temporary file, or renaming it over the target.

pub fn load_from(path: &Path) -> Result<(Self, u64, i64)>

Open path and load the snapshot it holds.

Errors

Propagates any I/O error from opening the file, or any error from Self::read_snapshot (a corrupt or incompatible snapshot).