Package sxir/pkg/runtime/vm

import "syntaxir.dev/sxir/pkg/runtime/vm"

Overview

Package vm runs grammars compiled to bytecode on the SyntaxIR virtual machine. LoadParser loads a compiled grammar (a .sxirc file, written by sxir compile or compiler.Marshal) without parsing, checking or compiling the grammar again, so a program that only loads parsers links no compiler. compiler.Compile returns a *Parser too, for a grammar compiled in the same program.

A Parser runs on a VM that uses Go recursion for rule calls; Iterative returns one that keeps rule calls on its own stack, so deeply nested input does not consume the Go stack. Both produce the same results. A Document parses a text incrementally; besides runtime.Document it has versions, batch edits, immutable snapshots and change notifications.

Index

Functions

func IsCompiled source

func IsCompiled(data []byte) bool

IsCompiled reports whether data is a compiled grammar written by compiler.Marshal or sxir compile.

Types

type CompiledMode source

type CompiledMode uint8

CompiledMode is the mode of a compiled grammar file, chosen when it is saved. A file holds the module of one mode and serves only the parses of that mode; the closure backend, Grammar and OutputSchema need the grammar source.

const (
	// CompiledParse serves Parse, ParsePrefix, ParseStream and Iterate without WithTrace or
	// WithProfile.
	CompiledParse CompiledMode = iota
	// CompiledRecognize serves Recognize without WithTrace or WithProfile.
	CompiledRecognize
	// CompiledDocument serves NewDocument, and Parse, ParsePrefix, ParseStream and Iterate with
	// WithTrace or WithProfile.
	CompiledDocument
)

func (CompiledMode) String source

func (m CompiledMode) String() string

String returns the name of the mode: "parse", "recognize" or "document".

type Document source

type Document struct {
	// contains filtered or unexported fields
}

Document is a text that is edited repeatedly. Parsing after an edit reuses the results of the previous parse that the edit did not affect (incremental parsing), as needed by editors. A Document is not safe for concurrent use.

Reused results are shared with the trees returned by earlier parses, and the nodes after an edit are moved to their new positions in place: an earlier tree changes when Parse runs again, and is then a mix of nodes at new positions (those reused) and at old ones. Parse writes it, so it must not be read concurrently with Parse. Use Node.Clone to keep a tree as it was.

func (*Document) CacheStats source

func (d *Document) CacheStats() runtime.CacheStats

CacheStats inspects retained incremental history without changing it. Document methods, including this one, require exclusive ownership.

func (*Document) DrainChanges source

func (d *Document) DrainChanges() []DocumentChange

DrainChanges transfers queued events. Overflow produces one full-refresh marker; draining resets the queue but retains the last comparison snapshot.

func (*Document) Edit source

func (d *Document) Edit(start, end int, text string) error

Edit replaces the range [start, end) of the text, in the document's position unit, with text. With byte positions, start and end must be on character boundaries. Each invalid UTF-8 byte is a separate character; edits may join bytes into a valid UTF-8 character.

func (*Document) EditBatch source

func (d *Document) EditBatch(base DocumentVersion, edits ...sxirinput.TextEdit) error

EditBatch atomically applies ordered, nonoverlapping edits in base coordinates. It rejects stale or foreign tokens, repeated starts, invalid ranges and byte boundaries. An empty batch validates base but changes nothing. A nonempty batch advances one revision and clears parse caches without parsing.

func (*Document) EnableChanges source

func (d *Document) EnableChanges(queueLimit int) error

EnableChanges opts into bounded notifications and one retained comparison snapshot. queueLimit bounds record count, not replacement or snapshot bytes. It must be positive; enabling twice is rejected. Future Parse and Snapshot calls capture and compare immutable outcomes inside their cumulative spend. Comparison is conservative about aliases and nil-versus-empty collections; this policy does not define language value equality. Interrupted invocations return nil and notify unavailable, breaking the comparison chain.

func (*Document) Parse source

func (d *Document) Parse(opts ...runtime.ParseOption) (*value.Node, error)

Parse parses the current text. opts apply once over construction defaults and may change context, compute limits, depth and trace, but not unit, effective backend or recognition mode. Overrides do not change future defaults. Pre-execution interruption preserves caches and statistics; execution or publication interruption discards reusable state while preserving text for retry. Parser-produced syntax diagnostics belong to that call: mutating them does not affect later parses. Returned Nodes may share the mutable Document graph.

func (*Document) ResetHistory source

func (d *Document) ResetHistory()

ResetHistory discards incremental reuse history while preserving text, construction options, version and the statistics of the last parse. Earlier result trees are detached from later node movement.

func (*Document) Snapshot source

func (d *Document) Snapshot(opts ...runtime.ParseOption) (*Snapshot, error)

Snapshot parses and freezes this revision using construction defaults plus opts. Cancellation and compute limits cover parsing, text, tree and diagnostics; interruption returns nil. Ordinary syntax errors retain the failed version.

func (*Document) Stats source

func (d *Document) Stats() runtime.ParseStats

Stats returns the evaluation and reuse counts of the last Parse.

func (*Document) Text source

func (d *Document) Text() string

Text returns the current text.

func (*Document) Unit source

func (d *Document) Unit() sxirinput.PositionUnit

Unit returns the position unit selected when the document was constructed.

func (*Document) Version source

func (d *Document) Version() DocumentVersion

Version returns the current text revision, initially sequence one.

type DocumentChange source

type DocumentChange struct {
	Kind        sxirinput.DocumentChangeKind
	From, To    DocumentVersion
	Unit        sxirinput.PositionUnit
	Edits       []sxirinput.TextEdit
	FullRefresh bool
	// Invalidated is conservative: an accepted edit may affect the whole output.
	// Reevaluation remains unknown; invalidation is not evidence of execution.
	Invalidated  bool
	Reevaluation string
	Output       sxirinput.OutputChange
}

DocumentChange owns an edit operation or an outcome comparison. Edit From/To advance source revisions; outcome From/To identify compared snapshots. Consumers track those streams separately. FullRefresh requires rebuilding current state.

type DocumentVersion source

type DocumentVersion struct {
	// contains filtered or unexported fields
}

DocumentVersion identifies one text revision of a Document. Its zero value is invalid; tokens are comparable and do not retain document contents.

func (DocumentVersion) Sequence source

func (v DocumentVersion) Sequence() uint64

Sequence returns the text revision number, starting at one.

type Parser source

type Parser struct {
	// contains filtered or unexported fields
}

Parser parses with a grammar compiled to bytecode, from one start rule. It is safe for concurrent use.

func LoadParser source

func LoadParser(data []byte) (*Parser, error)

LoadParser loads a parser saved by compiler.Marshal or sxir compile. Corruption is detected by a checksum and structural validation, but types are not checked again: load only data from trusted sources, such as the output of sxir compile or compiler.Marshal. The parser runs on the bytecode backends and serves the parses of the file's mode only; any other parse returns an error that names the mode it needs.

func (*Parser) CompiledMode source

func (p *Parser) CompiledMode() (CompiledMode, bool)

CompiledMode returns the mode of a parser loaded with LoadParser and true, or false for a parser compiled from a grammar, which serves every mode.

func (*Parser) DefaultStart source

func (p *Parser) DefaultStart() string

DefaultStart returns the default start rule of the grammar.

func (*Parser) Iterate source

func (p *Parser) Iterate(r io.Reader, opts ...runtime.ParseOption) (*runtime.StreamIterator, error)

Iterate validates a #stream parser and returns a lazy pull iterator. Reader access begins on Next; lookahead and buffering may still read ahead. With a diagnostic callback, Next reports committed recoveries before their element.

func (*Parser) Iterative source

func (p *Parser) Iterative() *Parser

Iterative returns a parser that shares the program and the start rule and runs on the VM that keeps rule calls on its own stack, so deeply nested input does not consume the Go stack. Its default depth limit is larger (runtime.Limits.Depth).

func (*Parser) Marshal source

func (p *Parser) Marshal(mode CompiledMode) ([]byte, error)

Marshal encodes the parser in the .sxirc format: the bytecode of mode and the start rule. LoadParser restores it without parsing, analyzing, type checking or compiling the grammar again. A loaded parser is saved only in its own mode (CompiledMode), which reproduces the file it was loaded from; saving another mode needs the grammar source.

func (*Parser) NewDocument source

func (p *Parser) NewDocument(text string, opts ...runtime.ParseOption) (runtime.Document, error)

NewDocument creates a Document for text. opts select its fixed unit and backend and the defaults for later Parse calls. Construction and each Parse use fresh budget counters. Edit uses the same unit and is outside parse budget controls. It returns an error if the backend is unavailable (Closure on a loaded parser) or the parser is loaded from a compiled grammar that is not in document mode.

func (*Parser) OutputSchema source

func (p *Parser) OutputSchema(opts ...runtime.SchemaOption) (*schema.Output, error)

OutputSchema exports the output schema of the start rule, or of every rule and type with runtime.WithAllSchemaRules. It does not inspect parse results or change parsing. A parser loaded from a compiled grammar has no grammar metadata and returns an error that wraps runtime.ErrSchemaUnavailable.

func (*Parser) Parse source

func (p *Parser) Parse(input string, opts ...runtime.ParseOption) (*value.Node, error)

Parse parses the whole input and returns the value of the start rule. If the input does not match, it returns a *value.SyntaxError. If the parse recovered from errors with #recover, it returns both the node and the value.SyntaxErrors.

func (*Parser) ParsePrefix source

func (p *Parser) ParsePrefix(name, input string, opts ...runtime.ParseOption) (PrefixResult, error)

ParsePrefix parses the named start rule at the beginning of input without requiring whole-input consumption; an empty name selects the grammar's default start rule. EOF assertions still refer to the complete input. Positions remain local to input. For a source slice, add its starting byte offset to ConsumedBytes to find the consumed endpoint in the original source. Recovered success returns the result and SyntaxErrors. Unrecovered syntax failures and fatal errors return a zero result. A zero-width success returns zero consumption; a nil Node does not imply failure. Parse remains whole-input. WithDiagnosticCallback and positive WithMissingTokens limits are invalid for this operation. Grammars containing #missing are unsupported even when insertion is disabled.

func (*Parser) ParseStream source

func (p *Parser) ParseStream(r io.Reader, emit func(*value.Node) error, opts ...runtime.ParseOption) error

ParseStream parses input read from r and passes each element of the #stream repetition at the top level of the start rule to emit as soon as it matches. Emitted elements and the input before them are not retained, so the input never has to fit in memory. If emit returns an error, parsing stops and ParseStream returns that error. An element without a value (for example, one that is discarded with -) is passed as nil. As in any repetition, an element that matches without consuming input ends it. WithContext and WithLimits apply to this invocation. Cancellation cannot interrupt a blocked reader or callback; the caller must arrange for it to return. A returned reader or emit error wins over cancellation first observed after that call. Delivered elements remain caller-owned, and retrying an interrupted stream requires caller-managed input replay. WithDiagnosticCallback delivers committed recoveries immediately before their elements; its error stops parsing before that element is emitted. Recovered errors are still returned at EOF. WithDiagnosticLimit bounds the retained recovery tail and explicitly marks a truncated result.

func (*Parser) Recognize source

func (p *Parser) Recognize(input string, opts ...runtime.ParseOption) error

Recognize decides whether the input matches the grammar without building a tree. It returns the same success, failure and syntax errors as Parse, recovered errors included. Predicates, and the actions they read, run; the run-time errors of other actions and typed conversions are not checked. A parser loaded in document mode does not recognize.

func (*Parser) Start source

func (p *Parser) Start() string

Start returns the name of the parser's start rule.

func (*Parser) Starts source

func (p *Parser) Starts() []string

Starts returns the start rules of the grammar, the rules marked @start where parsing may begin, in declaration order. The one marked @start(default), or the only one, is the default.

func (*Parser) WithStart source

func (p *Parser) WithStart(name string) (*Parser, error)

WithStart returns a parser that starts at the rule name, which must be a start rule; an empty name selects the grammar's default start rule. The compiled grammar is shared.

type PrefixResult source

type PrefixResult struct {
	Node          *value.Node
	Consumed      int // Selected parser Unit; defaults to CodePoints.
	ConsumedBytes int // Exact UTF-8 byte endpoint in the supplied input.
}

PrefixResult reports a rule's value and successful cursor. Consumption comes from the runtime cursor, even when Node is nil or its span differs.

type Snapshot source

type Snapshot struct {
	// contains filtered or unexported fields
}

Snapshot is an immutable document parse outcome for one text revision. Readers may use it concurrently with later edits of its Document.

func (*Snapshot) Error source

func (s *Snapshot) Error() error

Error returns a copy of the captured syntax diagnostic, or nil.

func (*Snapshot) Materialize source

func (s *Snapshot) Materialize() *value.Node

Materialize makes an independent mutable tree, preserving sharing, cycles, nil child slots, and nil versus allocated-empty slices.

func (*Snapshot) Root source

func (s *Snapshot) Root() SnapshotNode

Root returns the captured root, or the zero handle for a nil result.

func (*Snapshot) Stats source

func (s *Snapshot) Stats() runtime.ParseStats

Stats returns the evaluation and memo counts of the captured parse.

func (*Snapshot) Text source

func (s *Snapshot) Text() string

Text returns the captured text.

func (*Snapshot) Unit source

func (s *Snapshot) Unit() sxirinput.PositionUnit

Unit returns the position unit of the captured tree.

func (*Snapshot) Version source

func (s *Snapshot) Version() DocumentVersion

Version returns the captured text revision.

type SnapshotField source

type SnapshotField struct {
	Name  string
	Value any
}

SnapshotField is a field of a SnapshotNode. Its Value is an immutable scalar or a SnapshotNode, never a mutable Node.

type SnapshotNode source

type SnapshotNode struct {
	// contains filtered or unexported fields
}

SnapshotNode is a comparable read-only node handle; its zero value is nil.

func (SnapshotNode) Child source

func (n SnapshotNode) Child(i int) SnapshotNode

Child returns an indexed child; an invalid index panics like slice indexing.

func (SnapshotNode) ChildCount source

func (n SnapshotNode) ChildCount() int

ChildCount returns the number of child slots, nil slots included.

func (SnapshotNode) End source

func (n SnapshotNode) End() int32

End returns the exclusive end position.

func (SnapshotNode) Field source

func (n SnapshotNode) Field(name string) (any, bool)

Field returns the value of the named field, a scalar or a SnapshotNode, and whether the field is present: a present field may hold nil.

func (SnapshotNode) Fields source

func (n SnapshotNode) Fields() []SnapshotField

Fields returns the fields in stored order, in a slice the caller owns. A nil node value is a zero SnapshotNode.

func (SnapshotNode) IsTerminal source

func (n SnapshotNode) IsTerminal() bool

IsTerminal reports whether the node is terminal.

func (SnapshotNode) Kind source

func (n SnapshotNode) Kind() value.NodeKind

Kind returns what the node stands for, as Node.Kind does. Like Node.Kind, it panics on a nil node.

func (SnapshotNode) Len source

func (n SnapshotNode) Len() int

Len returns the number of elements of a tuple or a list (KindTuple, KindList, KindTupleValue, KindListValue), counting nil elements, and 0 for other nodes and for nil, as Node.Len does.

func (SnapshotNode) Rule source

func (n SnapshotNode) Rule() string

Rule returns the originating rule, or an empty string for nil.

func (SnapshotNode) Start source

func (n SnapshotNode) Start() int32

Start returns the inclusive start position.

func (SnapshotNode) Text source

func (n SnapshotNode) Text() string

Text returns terminal text.

func (SnapshotNode) Type source

func (n SnapshotNode) Type() string

Type returns the node type, or an empty string for nil.

func (SnapshotNode) Valid source

func (n SnapshotNode) Valid() bool

Valid reports whether the handle denotes a non-nil node.

func (SnapshotNode) Value source

func (n SnapshotNode) Value() any

Value returns the value of a KindValue node (such as an int, a string or a bool) and the string of a KindString node; it returns nil for other nodes and for nil, as Node.Value does.