Struct Pipeline 

pub struct Pipeline<'src, 'a, S> {
    source: &'src str,
    arena: &'a Arena,
    diagnostics: Vec<Diagnostic>,
    state: S,
    _arena: PhantomData<&'a Arena>,
}

Type-state lex pipeline. Each state’s transition method consumes self, materialises its phase output into the next state struct, and returns a new pipeline in the next state.

Fields

source: &'src str

arena: &'a Arena

diagnostics: Vec<Diagnostic>

state: S

_arena: PhantomData<&'a Arena>

Implementations

impl<'src, 'a> Pipeline<'src, 'a, Source>

pub fn new(source: &'src str, arena: &'a Arena) -> Pipeline<'src, 'a, Source>

Wrap a source string for type-state-driven lex. Phase 0 has not yet run; only source and arena are set.

pub fn run_to_completion( source: &'src str, arena: &'a Arena, ) -> BorrowedLexOutput<'a>

One-shot driver: run every phase and return the final BorrowedLexOutput. Equivalent to crate::lex_into_arena.

pub fn source(&self) -> &'src str

Borrow the original source text.

pub fn sanitize(self) -> Pipeline<'src, 'a, Sanitized<'a>>

Run Phase 0 (sanitize). Materialises the sanitized text in the arena so downstream phases borrow from the arena, not from the Pipeline struct (which would be self-referential).

impl<'src, 'a> Pipeline<'src, 'a, Sanitized<'a>>

pub fn sanitized_text(&self) -> &'a str

Sanitized text (arena-allocated).

pub fn diagnostics(&self) -> &[Diagnostic]

Diagnostics accumulated through Phase 0.

pub fn tokenize(self) -> Pipeline<'src, 'a, Tokenized<'a>>

Run Phase 1 (tokenize). Materialises the full BumpVec<'a, Token> inside arena via tokenize_in.

impl<'src, 'a> Pipeline<'src, 'a, Tokenized<'a>>

pub fn tokens(&self) -> &[Token]

Borrow the materialised token list. Useful for instrumentation.

pub fn pair(self) -> Pipeline<'src, 'a, Paired<'a>>

Run Phase 2 (pair). Materialises a paired-event stream inside arena via pair_in. Phase 2’s diagnostics are drained into the pipeline’s diagnostic accumulator immediately.

impl<'a> Pipeline<'_, 'a, Paired<'a>>

pub fn events(&self) -> &[PairEvent]

Borrow the materialised pair-event list. Useful for inspection before .build().

pub fn links(&self) -> &[PairLink]

Borrow the resolved (open, close) pair side-table. Useful for inspection before .build().

pub fn build(self) -> BorrowedLexOutput<'a>

Drive Phase 3 + the arena normalizer fold and return the final BorrowedLexOutput. Terminal transition because &mut BorrowedAllocator cannot be safely held across an external pause without locking the pipeline into a single thread for the allocator’s lifetime.

Diagnostic order

Sanitize (Phase 0) → Pair (Phase 2 unclosed/unmatched) → Classify (Phase 3 unknown annotations etc.). Matches the pre-Pipeline lex_into_arena ordering.

Panics

Panics if the sanitized source exceeds u32::MAX bytes (the lexer’s Span width contract). In practice unreachable; Phase 0 caps source length at the same boundary.

Trait Implementations

impl<'src, 'a, S> Debug for Pipeline<'src, 'a, S>

where S: Debug,

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.