Package sxir/pkg/bytecode

import "syntaxir.dev/sxir/pkg/bytecode"

Overview

Package bytecode defines the module that a SyntaxIR grammar compiles to, which the bytecode VM runs, and its compiled-file format: the tables of a module (Module and the Info types), Flow bodies (Flow) and their instruction set (FlowOp, FlowInstr), action code (Action), the encoding of the compiled-file format (Encoder, Decoder, WriteModule, ReadModule, FileMagic) and the load-time validation of a module (ValidateModule). spec/bytecode/README.md specifies all of them; the instruction set changes with its version, ISAVersion.

Package compiler makes modules (compiler.Marshal writes one as a compiled grammar file) and package runtime/vm loads and runs them; most programs use those packages and not this one. Like all of the module API at v0.x, this package may change in any release.

Index

Constants

const (
	ConstNil = iota
	ConstInt
	ConstString
	ConstBool
)

Kinds of an action expression constant.

const (
	VarInt = iota
	VarString
	VarBool
)

Variable types

const (
	OperatorPrefix = iota
	OperatorPostfix
	OperatorInfix
)

Operators Rule operator kinds and associativity

const (
	AssocLeft = iota
	AssocRight
	AssocNone
)
const FileMagic = "SXC\x00"

FileMagic begins every compiled grammar file (.sxirc), whose format the File format section of spec/bytecode/README.md specifies.

const FixedExps = 5

FixedExps is the number of fixed expectations at the start of the parser's expectation table (any character and the four anchors), which the expectation IDs of FlowExpectOf count before the strings of the module.

const ISAVersion = 38

ISAVersion is the instruction set of the value model, the only one supported. Earlier versions built the previous syntax trees, so their files are not loaded and need to be compiled again.

Variables

var FlowShapes = [...]FlowShape{
	FlowConst:         {Name: "CONST", Dst: 1, Imm: FlowImmConst},
	FlowMove:          {Name: "MOVE", Dst: 1, Args: 1},
	FlowNot:           {Name: "NOT", Dst: 1, Args: 1},
	FlowPos:           {Name: "POS", Dst: 1},
	FlowSetPos:        {Name: "SETPOS", Args: 1},
	FlowJump:          {Name: "JUMP", Target: true},
	FlowJumpFalse:     {Name: "JUMPFALSE", Args: 1, Target: true},
	FlowReturn:        {Name: "RETURN", Args: 2},
	FlowMark:          {Name: "MARK", Dst: 1},
	FlowReset:         {Name: "RESET", Args: 1},
	FlowPeek:          {Name: "PEEK", Dst: 3},
	FlowGuard:         {Name: "GUARD", Dst: 1, Args: 2, Imm: FlowImmGuard},
	FlowExpect:        {Name: "EXPECT", Args: 1, Imm: FlowImmExpect},
	FlowUnfresh:       {Name: "UNFRESH", Args: 1},
	FlowCall:          {Name: "CALL", Dst: 2, Imm: FlowImmCall},
	FlowAdd:           {Name: "ADD", Dst: 1, Args: 2},
	FlowSub:           {Name: "SUB", Dst: 1, Args: 2},
	FlowEq:            {Name: "EQ", Dst: 1, Args: 2},
	FlowNe:            {Name: "NE", Dst: 1, Args: 2},
	FlowLt:            {Name: "LT", Dst: 1, Args: 2},
	FlowLe:            {Name: "LE", Dst: 1, Args: 2},
	FlowStep:          {Name: "STEP"},
	FlowSilent:        {Name: "SILENT", Imm: FlowImmSilence},
	FlowAdvance:       {Name: "ADVANCE", Args: 1},
	FlowInClass:       {Name: "INCLASS", Dst: 1, Args: 1, Imm: FlowImmClass},
	FlowAtAnchor:      {Name: "ATANCHOR", Dst: 1, Imm: FlowImmAnchor},
	FlowMatchLit:      {Name: "MATCHLIT", Dst: 2, Imm: FlowImmLit},
	FlowIsolate:       {Name: "ISOLATE", Dst: 1},
	FlowUnisolate:     {Name: "UNISOLATE", Dst: 2, Args: 1},
	FlowMergeExpected: {Name: "MERGEEXPECTED", Args: 2},
	FlowNewTerminal:   {Name: "NEWTERMINAL", Dst: 1, Args: 2},
	FlowNewSeq:        {Name: "NEWSEQ", Dst: 1, Args: 2},
	FlowNewList:       {Name: "NEWLIST", Dst: 1, Args: 2},
	FlowNewVals:       {Name: "NEWVALS", Dst: 1, Imm: FlowImmCount},
	FlowAppend:        {Name: "APPEND", Dst: 1, Args: 2},
	FlowSpread:        {Name: "SPREAD", Dst: 1, Args: 2},
	FlowText:          {Name: "TEXT", Dst: 1, Args: 1},
	FlowLiteralText:   {Name: "LITERALTEXT", Dst: 1, Args: 1, Imm: FlowImmStr},
	FlowStr:           {Name: "STR", Dst: 1, Imm: FlowImmStr},
	FlowKidsLen:       {Name: "KIDSLEN", Dst: 1},
	FlowKids:          {Name: "KIDS", Dst: 1, Args: 1},
	FlowDropKids:      {Name: "DROPKIDS", Args: 1},
	FlowPushKid:       {Name: "PUSHKID", Args: 1},
	FlowShapeSingle:   {Name: "SHAPESINGLE", Dst: 2, Args: 2, Imm: FlowImmShape},
	FlowScopeValue:    {Name: "SCOPEVALUE", Dst: 1, Args: 2, Imm: FlowImmShape},
	FlowSetCapture:    {Name: "SETCAPTURE", Args: 1, Imm: FlowImmSlot},
	FlowMarkPos:       {Name: "MARKPOS", Dst: 1, Args: 1},
	FlowFrame:         {Name: "FRAME", Dst: 1},
	FlowNewFrame:      {Name: "NEWFRAME", Dst: 1, Imm: FlowImmScope},
	FlowSetFrame:      {Name: "SETFRAME", Args: 1},
	FlowClearVals:     {Name: "CLEARVALS", Args: 1},
	FlowFrameVal:      {Name: "FRAMEVAL", Dst: 1, Args: 1, Imm: FlowImmSlot},
	FlowAttachFrame:   {Name: "ATTACHFRAME", Dst: 1, Args: 3, Imm: FlowImmScope},
	FlowShapeCapture:  {Name: "SHAPECAPTURE", Dst: 1, Args: 2, Imm: FlowImmShape},
	FlowRunHot:        {Name: "RUNHOT", Dst: 1, Imm: FlowImmRunSite},
	FlowRunBegin:      {Name: "RUNBEGIN", Imm: FlowImmRunWalk},
	FlowRunWalked:     {Name: "RUNWALKED", Args: 1, Imm: FlowImmRunSite},
	FlowSkipVisit:     {Name: "SKIPVISIT"},
	FlowSkipHit:       {Name: "SKIPHIT", Dst: 1},
	FlowSkipValue:     {Name: "SKIPVALUE", Args: 1},
	FlowSkipEnd:       {Name: "SKIPEND"},
	FlowSkipKids:      {Name: "SKIPKIDS", Dst: 1},
	FlowScanReach:     {Name: "SCANREACH", Dst: 1, Args: 2, Imm: FlowImmScanSite},
	FlowScanEnd:       {Name: "SCANEND", Args: 2, Imm: FlowImmScanSite},
	FlowScanKids:      {Name: "SCANKIDS", Dst: 1, Args: 3, Imm: FlowImmScanSite},
	FlowPred:          {Name: "PRED", Dst: 1, Imm: FlowImmAction},
	FlowBind:          {Name: "BIND", Dst: 1, Imm: FlowImmBind},
	FlowCut:           {Name: "CUT", Dst: 1},
	FlowSetCut:        {Name: "SETCUT", Args: 1},
	FlowKeep:          {Name: "KEEP", Dst: 1, Args: 1},
	FlowMakeError:     {Name: "MAKEERROR", Dst: 1, Args: 2},
	FlowAppendError:   {Name: "APPENDERROR", Args: 1},
	FlowNewErrorNode:  {Name: "NEWERRORNODE", Dst: 1, Args: 2},
	FlowRecLen:        {Name: "RECLEN", Dst: 1},
	FlowRestore:       {Name: "RESTORE", Args: 2},
	FlowGuardLit:      {Name: "GUARDLIT", Dst: 1, Imm: FlowImmLitGuard},
	FlowJumpTrue:      {Name: "JUMPTRUE", Args: 1, Target: true},
	FlowMatchClass:    {Name: "MATCHCLASS", Dst: 3, Target: true, Imm: FlowImmClassTest},
	FlowGuardJump:     {Name: "GUARDJUMP", Args: 2, Target: true, Imm: FlowImmGuardTest},
	FlowGuardLitJump:  {Name: "GUARDLITJUMP", Target: true, Imm: FlowImmLitGuardTest},
	FlowAssertJump:    {Name: "ASSERTJUMP", Target: true, Imm: FlowImmAnchorTest},
	FlowMatchLitJump:  {Name: "MATCHLITJUMP", Dst: 2, Target: true, Imm: FlowImmLit},
	FlowCallJump:      {Name: "CALLJUMP", Dst: 2, Target: true, Imm: FlowImmCall},
	FlowStreamOn:      {Name: "STREAMON", Dst: 1, Imm: FlowImmRule},
	FlowStreamStart:   {Name: "STREAMSTART"},
	FlowEmitStream:    {Name: "EMITSTREAM", Args: 1},
	FlowResumeOn:      {Name: "RESUMEON", Dst: 1},
	FlowResumeStart:   {Name: "RESUMESTART", Dst: 1, Imm: FlowImmResume},
	FlowResumeMore:    {Name: "RESUMEMORE", Dst: 1, Args: 1},
	FlowResumeNext:    {Name: "RESUMENEXT", Dst: 2, Args: 1},
	FlowResumeBegin:   {Name: "RESUMEBEGIN", Args: 1},
	FlowResumeEnd:     {Name: "RESUMEEND", Args: 3},
	FlowResumeFinish:  {Name: "RESUMEFINISH", Args: 1},
	FlowMergeRenamed:  {Name: "MERGERENAMED", Args: 2, Imm: FlowImmExpect},
	FlowScanReachAt:   {Name: "SCANREACHAT", Dst: 1, Args: 2, Imm: FlowImmScanSite},
	FlowScanEndAt:     {Name: "SCANENDAT", Args: 2, Imm: FlowImmScanSite},
}

FlowShapes holds the shape of each operation, by FlowOp. A module file writes an instruction's registers, target and immediate as its shape says, and loading a module checks each instruction against it.

Functions

func IsCompiled source

func IsCompiled(data []byte) bool

IsCompiled reports whether the data begins with FileMagic, that is, whether it is in the compiled grammar format rather than a grammar source.

func ValidateModule source

func ValidateModule(m *Module, fieldType func(text string) error) error

ValidateModule checks that every table reference of the module and every reference of its Flow bodies is in range, that every rule and Operators Rule line has a body, that the Flow bodies are well formed (validateFlows) and that every rule on a cycle of calls is marked Rec. fieldType checks the text of a field type, which the type system above the module defines.

Types

type Action source

type Action struct {
	// Regs holds the sizes of the register files: values, integers and booleans, element
	// sequences and strings.
	Regs [4]int
	Code []ActionInstr
	// Consts holds the constants of the action expression.
	Consts []ActionConst
	// Names holds string table indices: built-in names, the names of Operators Rule locals and of
	// struct fields.
	Names []int
	// Members holds the member reads.
	Members []ActionMember
	// NodeTypes holds the named node types that conversions make (string table indices).
	NodeTypes []int
	// Casts holds the types of casts.
	Casts []ActionCast
	// Structs holds the struct types that literals make.
	Structs []ActionStruct
}

Action is the action code of an action, a predicate or the value of a binding: the instructions of the function that the lowering of action expressions makes of the action expression, over registers of four files, with the pools they index. The instruction set and its checks belong to the engine, which compiles and runs action code (internal/runtime/state); the module stores it and checks that the pools refer to strings of the string table. See "Action Code" in spec/bytecode/README.md.

type ActionCast source

type ActionCast struct {
	Type      int
	Line, Col int
}

ActionCast is the type of a cast: its resolved type text (string table index, as the type table writes an underlying node type) and its grammar position, 0 and 0 without source information.

type ActionConst source

type ActionConst struct {
	Kind int
	Int  int64
}

ActionConst is a constant: nil, an implementation int, a string (string table index in Int) or a bool (0 or 1 in Int).

type ActionInstr source

type ActionInstr struct {
	Op      uint8
	Dst     int32
	A, B, C int32
}

ActionInstr is an instruction of action code: an opcode, a destination register or jump target, and up to three operands.

type ActionMember source

type ActionMember struct {
	Kind        int
	Name, Field int
}

ActionMember is a member read: the kind of member (a field, Start, End, Value or a capture), the name the action expression reads and the field or capture it names (string table indices).

type ActionStruct source

type ActionStruct struct {
	Type   int
	Fields int
}

ActionStruct is a struct literal: its struct type (string table index) and number of fields.

type Class source

type Class struct {
	Ranges      []rune
	Negated     bool
	Expectation int // the expectation that a failure to match records (string table index)
}

Class is a character class. Ranges is a list of lower and upper bound pairs.

func (*Class) Has source

func (cl *Class) Has(ch rune) bool

Has reports whether the character ch is in the class.

type Decoder source

type Decoder struct {
	Data []byte   // the compiled grammar
	Pos  int      // the read position in Data
	Strs []string // the string table
	Err  error    // the first error
}

Decoder reads a compiled grammar. On invalid data it does not panic but records the first error.

func (*Decoder) Bool source

func (r *Decoder) Bool() bool

Bool reads a boolean byte.

func (*Decoder) Byte source

func (r *Decoder) Byte() byte

Byte reads a byte.

func (*Decoder) Count source

func (r *Decoder) Count() int

Count reads an element count. A count larger than the remaining data is invalid (this prevents huge allocations).

func (*Decoder) Fail source

func (r *Decoder) Fail(format string, args ...any)

Fail records the error at the read position unless one is recorded.

func (*Decoder) Int source

func (r *Decoder) Int() int

Int reads a signed integer.

func (*Decoder) ReadModule source

func (r *Decoder) ReadModule() *Module

ReadModule reads a module.

func (*Decoder) Str source

func (r *Decoder) Str() string

Str reads a string by its index in the string table.

func (*Decoder) Strings source

func (r *Decoder) Strings()

Strings reads the string table and appends it to Strs.

func (*Decoder) Uint source

func (r *Decoder) Uint() int

Uint reads a non-negative integer.

type Encoder source

type Encoder struct {
	Body bytes.Buffer   // the encoded values
	Strs map[string]int // index of each string in List
	List []string       // the string table
}

Encoder writes the payload of a compiled grammar: variable-length integers and strings by their index in a string table, which Finish writes before the body.

func (*Encoder) Bool source

func (w *Encoder) Bool(b bool)

Bool writes b as one byte.

func (*Encoder) Byte source

func (w *Encoder) Byte(b byte)

Byte writes b.

func (*Encoder) Finish source

func (w *Encoder) Finish(out *bytes.Buffer)

Finish writes the string table and the body to out and appends the checksum.

func (*Encoder) Int source

func (w *Encoder) Int(n int)

Int writes the signed integer n, zigzag-encoded.

func (*Encoder) Str source

func (w *Encoder) Str(s string)

Str writes the index of s in the string table, adding s to it.

func (*Encoder) Uint source

func (w *Encoder) Uint(n int)

Uint writes the non-negative integer n.

func (*Encoder) WriteModule source

func (w *Encoder) WriteModule(m *Module)

WriteModule writes m.

type Flow source

type Flow struct {
	// Regs is the number of registers of the frame.
	Regs int
	Code []FlowInstr
}

Flow is the body of a rule that the VM runs from Flow IR (Module.Flows): a flat list of instructions over the registers of one frame, in which the operations of the Flow dialect (internal/ir/code/flow) are the instructions that touch the parser. The rule's entry and its value are those of the rule's bytecode form: a body returns its value and true, or nil and false.

The immediates of the instructions are Go values of the types below; FlowShapes says which one each operation has and how many registers it names.

func (*Flow) RestoreAcrossCommit source

func (f *Flow) RestoreAcrossCommit() (pc int, ok bool)

RestoreAcrossCommit returns the first instruction of f, in instruction order, that may restore the parse to a state saved before an EMITSTREAM that ran since: a RESET, SETPOS or RESTORE that reads, on some path, a register that held a mark, a position or a number of recovered errors (MARK, POS or RECLEN, or a MOVE, MARKPOS, ADD or SUB of one) when an EMITSTREAM ran. ok is false if there is none. EMITSTREAM commits the input and the recovered errors before the position, so nothing after it may go back before it; a loaded module has no such instruction (ValidateModule), on any path, whether a choice, a loop or a recovery takes it.

An instruction continues at the next one unless it is JUMP or RETURN, and at its target if its operation has one (FlowShape.Target). The operations, registers and targets must be in range.

func (*Flow) Unassigned source

func (f *Flow) Unassigned() []int

Unassigned returns, in increasing order, the registers of f that an instruction may read before any instruction writes them: on some path from the first instruction to a reachable instruction that reads the register as an argument, no instruction names it as a destination. A body without such a register reads only values that its own run wrote, so a frame of registers need not be cleared before the body runs. A loaded module has none (ValidateModule); the bytecode lowering writes the zero value of a local at the start of the body only where some path reads it unwritten.

An instruction continues at the next one unless it is JUMP or RETURN, and at its target if its operation has one (FlowShape.Target). The operations, registers and targets must be in range.

type FlowAnchorTestOf source

type FlowAnchorTestOf struct {
	Anchor int
	Expect FlowExpectOf
}

FlowAnchorTestOf is the immediate of FlowAssertJump: the anchor, as the immediate of FlowAtAnchor, and the expectation recorded when the position is not at it.

type FlowBindOf source

type FlowBindOf struct {
	Var, Action int
}

FlowBindOf is the immediate of FlowBind: the variable, an index of Module.Vars, and its action expression, an index of Module.Actions.

type FlowCallOf source

type FlowCallOf struct{ Rule, Level int }

FlowCallOf is the immediate of FlowCall and FlowCallJump: the rule's index and the binding level it is called at.

type FlowClassTestOf source

type FlowClassTestOf struct {
	Class  int
	Expect FlowExpectOf
}

FlowClassTestOf is the immediate of FlowMatchClass: the class, an index of Module.Classes, and the expectation recorded when the character is not in it.

type FlowExpectOf source

type FlowExpectOf struct {
	Exp     int
	Message bool
}

FlowExpectOf is the immediate of FlowExpect: the expectation's ID in the parser's table (below FixedExps one of the fixed expectations, otherwise FixedExps plus an index of Module.Strings), and whether it is the message of a failed @error.

type FlowGuardOf source

type FlowGuardOf struct {
	Class int
	Depth int
}

FlowGuardOf is the immediate of FlowGuard: the class Class (an index of Module.Classes) and the depth the guard adds to the parser's depth.

type FlowGuardTestOf source

type FlowGuardTestOf struct {
	Guard  FlowGuardOf
	Expect FlowExpectOf
}

FlowGuardTestOf is the immediate of FlowGuardJump: the guard and the expectation recorded when it fails.

type FlowImm source

type FlowImm uint8

FlowImm is the kind of immediate of an operation.

const (
	// FlowImmNone is no immediate: Imm is nil.
	FlowImmNone FlowImm = iota
	// FlowImmConst is a constant: nil, a bool or an int.
	FlowImmConst
	// FlowImmGuard is a FlowGuardOf.
	FlowImmGuard
	// FlowImmExpect is a FlowExpectOf.
	FlowImmExpect
	// FlowImmCall is a FlowCallOf.
	FlowImmCall
	// FlowImmClass is a class: an int index of Module.Classes.
	FlowImmClass
	// FlowImmAnchor is an anchor: an int from 0 to 3.
	FlowImmAnchor
	// FlowImmLit is a FlowLitOf.
	FlowImmLit
	// FlowImmSilence is a FlowSilenceOf.
	FlowImmSilence
	// FlowImmCount is a count: an int that is not negative.
	FlowImmCount
	// FlowImmStr is a string: an int index of Module.Strings.
	FlowImmStr
	// FlowImmShape is a FlowShapeOf.
	FlowImmShape
	// FlowImmSlot is a capture slot: an int that is not negative, below the number of slots of the
	// largest frame the body may run with (its rule's scope or a scope of its FlowNewFrame).
	FlowImmSlot
	// FlowImmScope is a scope: an int index of Module.Scopes.
	FlowImmScope
	// FlowImmRunSite is a skip-run or run site, numbered as the compiler numbers the sites of the
	// grammar: an int from 1 to the number of instructions of the module's Flow bodies.
	FlowImmRunSite
	// FlowImmRunWalk is a FlowRunWalkOf.
	FlowImmRunWalk
	// FlowImmScanSite is a scan site, numbered as the compiler numbers the sites of the grammar:
	// an int from 1 to the largest scan site that a module of its Flow instructions can name.
	FlowImmScanSite
	// FlowImmAction is an action expression: an int index of Module.Actions.
	FlowImmAction
	// FlowImmBind is a FlowBindOf.
	FlowImmBind
	// FlowImmLitGuard is a FlowLitGuardOf.
	FlowImmLitGuard
	// FlowImmClassTest is a FlowClassTestOf.
	FlowImmClassTest
	// FlowImmGuardTest is a FlowGuardTestOf.
	FlowImmGuardTest
	// FlowImmLitGuardTest is a FlowLitGuardTestOf.
	FlowImmLitGuardTest
	// FlowImmAnchorTest is a FlowAnchorTestOf.
	FlowImmAnchorTest
	// FlowImmRule is a rule: an int index of Module.Rules.
	FlowImmRule
	// FlowImmResume is a FlowResumeOf.
	FlowImmResume
)

type FlowInstr source

type FlowInstr struct {
	Op     FlowOp
	Dst    []int
	Args   []int
	Target int
	Imm    any
}

FlowInstr is an instruction of a Flow body. Dst, Args and Target are used as the operation documents and its shape (FlowShapes) counts; a destination of -1 discards the value. Imm is its immediate.

type FlowLitGuardOf source

type FlowLitGuardOf struct {
	Rune  rune
	Fold  bool
	Depth int
}

FlowLitGuardOf is the immediate of FlowGuardLit: the first character of the literal (Rune, folded for a case-insensitive literal, Fold) and the depth the guard adds to the parser's depth.

type FlowLitGuardTestOf source

type FlowLitGuardTestOf struct {
	Guard  FlowLitGuardOf
	Expect FlowExpectOf
}

FlowLitGuardTestOf is the immediate of FlowGuardLitJump: the guard of a literal and the expectation recorded when it fails.

type FlowLitOf source

type FlowLitOf struct {
	Str  int
	Exp  int
	Fold bool
}

FlowLitOf is the immediate of FlowMatchLit and FlowMatchLitJump: the literal (an index of Module.Strings; for a case-insensitive literal, Fold, its text with each ASCII letter lowercased) and the expectation it records on failure, as FlowExpectOf.Exp counts it.

type FlowOp source

type FlowOp uint8

FlowOp is the operation of a FlowInstr.

const (
	// FlowConst sets Dst[0] to the immediate (a bool, an int, or nil for the null value).
	FlowConst FlowOp = iota
	// FlowMove sets Dst[0] to Args[0].
	FlowMove
	// FlowNot sets Dst[0] to the negation of the bool Args[0].
	FlowNot
	// FlowPos sets Dst[0] to the position.
	FlowPos
	// FlowSetPos sets the position to the int Args[0].
	FlowSetPos
	// FlowJump continues at Target.
	FlowJump
	// FlowJumpFalse continues at Target when the bool Args[0] is false.
	FlowJumpFalse
	// FlowReturn returns the value Args[0] and the bool Args[1].
	FlowReturn
	// FlowMark sets Dst[0] to the mark of the whole parser state (flow.markState).
	FlowMark
	// FlowReset restores the parser state to the mark Args[0] (flow.reset).
	FlowReset
	// FlowPeek sets Dst[0] to the character, Dst[1] to its width and Dst[2] to whether there is one
	// (flow.peek); a Dst of -1 is discarded.
	FlowPeek
	// FlowGuard sets Dst[0] to whether the guard of a class of the immediate FlowGuardOf fails for
	// the character Args[0] that was read when Args[1] (the peek's ok) holds (flow.guardFails).
	FlowGuard
	// FlowExpect records the expectation of the immediate FlowExpectOf at the position Args[0]
	// (flow.expect).
	FlowExpect
	// FlowUnfresh marks the node Args[0], if it is one, as no longer owned by its maker
	// (flow.unfresh).
	FlowUnfresh
	// FlowCall calls the rule of the immediate FlowCallOf and sets Dst[0] to its value and Dst[1] to
	// its success (flow.callRule).
	FlowCall
	// FlowAdd sets Dst[0] to the sum of the ints Args[0] and Args[1].
	FlowAdd
	// FlowSub sets Dst[0] to the int Args[0] minus the int Args[1].
	FlowSub
	// FlowEq sets Dst[0] to whether Args[0] and Args[1] are equal.
	FlowEq
	// FlowNe sets Dst[0] to whether Args[0] and Args[1] differ.
	FlowNe
	// FlowLt sets Dst[0] to whether the integer Args[0] (an int or a character) is less than Args[1].
	FlowLt
	// FlowLe sets Dst[0] to whether the integer Args[0] is at most Args[1].
	FlowLe
	// FlowStep charges one step of budget (flow.step).
	FlowStep
	// FlowSilent enters or leaves the lookahead of the immediate FlowSilenceOf (flow.silent).
	FlowSilent
	// FlowAdvance moves the position forward by the int Args[0] (flow.advance).
	FlowAdvance
	// FlowInClass sets Dst[0] to whether the character Args[0] is in the class of the immediate, an
	// index of Module.Classes (flow.inClass).
	FlowInClass
	// FlowAtAnchor sets Dst[0] to whether the position is at the anchor of the immediate, an int: 0
	// the beginning of the input, 1 its end, 2 the beginning of a line, 3 the end of a line
	// (flow.atAnchor).
	FlowAtAnchor
	// FlowMatchLit matches the literal of the immediate FlowLitOf and sets Dst[0] to the null value
	// and Dst[1] to whether it matched (flow.matchLit).
	FlowMatchLit
	// FlowIsolate begins an expectation scope at the position and sets Dst[0] to its mark
	// (flow.isolate).
	FlowIsolate
	// FlowUnisolate ends the expectation scope Args[0] and sets Dst[0] to the farthest failure inside
	// it and Dst[1] to its expectations (flow.unisolate).
	FlowUnisolate
	// FlowMergeExpected merges the expectations Args[1], recorded at the int Args[0], into the
	// enclosing scope (flow.mergeExpected).
	FlowMergeExpected
	// FlowNewTerminal sets Dst[0] to a terminal node from the int Args[0] to the position with the
	// text Args[1] (flow.newTerminalNode).
	FlowNewTerminal
	// FlowNewSeq sets Dst[0] to a sequence node from the int Args[0] to the position with the children
	// Args[1] (flow.newBranchNode).
	FlowNewSeq
	// FlowNewList is FlowNewSeq for a list node (flow.newBranchNode of a list).
	FlowNewList
	// FlowNewVals sets Dst[0] to an empty list of values with room for the immediate, an int
	// (flow.newVals).
	FlowNewVals
	// FlowAppend sets Dst[0] to the list Args[0] with the value Args[1] appended (flow.append).
	FlowAppend
	// FlowSpread sets Dst[0] to the list Args[0] with the children of the node Args[1] appended
	// (flow.append of a spread value).
	FlowSpread
	// FlowText sets Dst[0] to the text of the input from the int Args[0] to the position (flow.text).
	FlowText
	// FlowLiteralText sets Dst[0] to the text of the literal of the immediate, a string index, matched
	// from the int Args[0] (flow.literalText).
	FlowLiteralText
	// FlowStr sets Dst[0] to the string of the immediate, a string index (flow.str).
	FlowStr
	// FlowKidsLen sets Dst[0] to the height of the stack of gathered values (flow.kidsLen).
	FlowKidsLen
	// FlowKids sets Dst[0] to the values gathered above the int Args[0], which it takes off the stack
	// (flow.kids).
	FlowKids
	// FlowDropKids truncates the stack of gathered values to the int Args[0] (flow.dropKids).
	FlowDropKids
	// FlowPushKid pushes the value Args[0] on the stack of gathered values (flow.pushKid).
	FlowPushKid
	// FlowShapeSingle sets Dst[0] to the value of Args[0], matched from the int Args[1] by an
	// expression of the shape of the immediate FlowShapeOf, and Dst[1] to whether it has one
	// (flow.shapeSingle).
	FlowShapeSingle
	// FlowScopeValue sets Dst[0] to the value of a scope whose body, of the shape of the immediate
	// FlowShapeOf, matched Args[0] from the int Args[1] (flow.scopeValue).
	FlowScopeValue
	// FlowSetCapture sets the capture slot of the immediate, a slot, of the current frame to the
	// value Args[0], recording the old value on the capture trail (flow.setCapture).
	FlowSetCapture
	// FlowMarkPos sets Dst[0] to the position of the mark Args[0] (flow.markPos).
	FlowMarkPos
	// FlowFrame sets Dst[0] to the current capture frame (flow.frame).
	FlowFrame
	// FlowNewFrame sets Dst[0] to a new capture frame with a slot for each name of the scope of the
	// immediate (flow.newFrame).
	FlowNewFrame
	// FlowSetFrame makes the frame Args[0] the current capture frame (flow.setFrame).
	FlowSetFrame
	// FlowClearVals sets the values of the frame Args[0] to nil (flow.clearVals).
	FlowClearVals
	// FlowFrameVal sets Dst[0] to the value of the slot of the immediate of the frame Args[0]
	// (flow.frameVal).
	FlowFrameVal
	// FlowAttachFrame sets Dst[0] to the value Args[0], matched from the int Args[1], with the values
	// of the frame Args[2] attached as the fields named by the scope of the immediate
	// (flow.attachFrame).
	FlowAttachFrame
	// FlowShapeCapture sets Dst[0] to the value of Args[0], matched from the int Args[1] by a
	// captured expression of the shape of the immediate FlowShapeOf: as FlowShapeSingle, or the unit
	// node of the range matched where the expression has no value (flow.shapeCapture).
	FlowShapeCapture
	// FlowRunHot sets Dst[0] to whether the walks of the run site of the immediate keep records
	// (flow.runHot).
	FlowRunHot
	// FlowRunBegin begins a walk of the hot run site of the immediate FlowRunWalkOf at the position
	// (flow.runBegin).
	FlowRunBegin
	// FlowRunWalked ends the walk of the run site of the immediate that was not hot and began at the
	// int Args[0] (flow.runWalked).
	FlowRunWalked
	// FlowSkipVisit ends the iteration of the walk before, if any, and begins one at the position
	// (flow.skipVisit).
	FlowSkipVisit
	// FlowSkipHit sets Dst[0] to whether the walk takes the rest from a record of the position, which
	// it then does (flow.skipHit).
	FlowSkipHit
	// FlowSkipValue adds the value Args[0] of the iteration that matched to the walk (flow.skipValue).
	FlowSkipValue
	// FlowSkipEnd ends the walk, recording the rest of it, and drops the values it holds
	// (flow.skipEnd).
	FlowSkipEnd
	// FlowSkipKids ends a walk that builds a value as FlowSkipEnd does and sets Dst[0] to the list of
	// the values of its iterations (flow.skipKids).
	FlowSkipKids
	// FlowScanReach sets Dst[0] to the count of a walk of the scan site of the immediate that began at
	// the int Args[0] and has matched the int Args[1] characters, taking the rest from a record of its
	// start where the site is hot and has one (flow.scanReach).
	FlowScanReach
	// FlowScanEnd ends a value-free walk of the scan site of the immediate that began at the int
	// Args[0] and matched the int Args[1] characters (flow.scanEnd).
	FlowScanEnd
	// FlowScanKids sets Dst[0] to the elements of a walk of the scan site of the immediate that began
	// at the int Args[0], gathered on the stack from the int Args[1], which matched the int Args[2]
	// characters (flow.scanKids).
	FlowScanKids
	// FlowPred sets Dst[0] to whether the predicate whose action expression is the immediate holds in the current
	// frame (flow.pred).
	FlowPred
	// FlowBind binds the variable of the immediate FlowBindOf to the value of its action expression, evaluated in
	// the current frame, and sets Dst[0] to whether the
	// evaluation succeeded (flow.bind).
	FlowBind
	// FlowCut sets Dst[0] to the cut flag of the rule call the body runs in (the register prim.Cut).
	FlowCut
	// FlowSetCut sets the cut flag of the rule call the body runs in to the bool Args[0].
	FlowSetCut
	// FlowKeep sets Dst[0] to the expectations Args[0], kept beyond the scope that recorded them, as
	// those of a failed recovery body are (flow.keep).
	FlowKeep
	// FlowMakeError sets Dst[0] to the syntax error at the int Args[0] with the expectations Args[1]
	// (flow.makeError).
	FlowMakeError
	// FlowAppendError appends the syntax error Args[0] to the recovered errors (flow.appendError).
	FlowAppendError
	// FlowNewErrorNode sets Dst[0] to the error node of the syntax error Args[1] from the int Args[0]
	// to the position (flow.newErrorNode).
	FlowNewErrorNode
	// FlowRecLen sets Dst[0] to the number of recovered errors (flow.recLen).
	FlowRecLen
	// FlowRestore sets the position to the int Args[0] and drops the recovered errors after the first
	// int Args[1] (flow.restore).
	FlowRestore
	// FlowGuardLit sets Dst[0] to whether the guard of a literal of the immediate FlowLitGuardOf
	// fails: the input at the position does not begin with the literal's first character, compared
	// as the literal compares it (state.Parser.MatchChar), and the nesting limit allows the guard's
	// depth (flow.guardLitFails).
	FlowGuardLit

	// FlowJumpTrue continues at Target when the bool Args[0] is true.
	FlowJumpTrue
	// FlowMatchClass reads the character at the position as FlowPeek does, into Dst[0], Dst[1] and
	// Dst[2]; if there is none or it is not in the class of the immediate FlowClassTestOf, it records
	// the immediate's expectation at the position, as FlowExpect does, and continues at Target.
	FlowMatchClass
	// FlowGuardJump tests the guard of the immediate FlowGuardTestOf as FlowGuard does, for the
	// character Args[0] and the peek's ok Args[1]; if it fails, it records the immediate's expectation
	// at the position and continues at Target.
	FlowGuardJump
	// FlowGuardLitJump tests the guard of a literal of the immediate FlowLitGuardTestOf as
	// FlowGuardLit does; if it fails, it records the immediate's expectation at the position and
	// continues at Target.
	FlowGuardLitJump
	// FlowAssertJump tests the anchor of the immediate FlowAnchorTestOf as FlowAtAnchor does; if the
	// position is not at it, it records the immediate's expectation at the position and continues at
	// Target.
	FlowAssertJump
	// FlowMatchLitJump matches the literal of the immediate FlowLitOf as FlowMatchLit does, with its
	// results in Dst[0] and Dst[1], and continues at Target if it does not match.
	FlowMatchLitJump
	// FlowCallJump calls the rule of the immediate FlowCallOf as FlowCall does, with its results in
	// Dst[0] and Dst[1], and continues at Target if the call fails.
	FlowCallJump

	// FlowStreamOn sets Dst[0] to whether the call of the rule of the immediate delivers the elements
	// of its #stream repetition: the parse is a stream parse with a callback, the rule is its start
	// rule, and the call is its top call if the rule is recursive (flow.streamOn).
	FlowStreamOn
	// FlowStreamStart begins the stream: the repetition that delivers its elements is about to run, and
	// the nodes of the elements are allocated apart from those of the prefix (flow.streamStart).
	FlowStreamStart
	// FlowEmitStream delivers the element Args[0], a node or nil for a value-free repetition, to the
	// callback of the stream parse, which has consumed the input up to the position, and commits it
	// (flow.emitStream).
	FlowEmitStream

	// FlowResumeOn sets Dst[0] to whether the repetitions resume their runs: the parse is a Document's,
	// outside lookaheads and the evaluation of a skip expression (flow.resumeOn).
	FlowResumeOn
	// FlowResumeStart starts the run of the repetition of the immediate FlowResumeOf at the position,
	// with the record of the run that it takes over, and sets Dst[0] to the run (flow.resumeStart).
	FlowResumeStart
	// FlowResumeMore sets Dst[0] to whether the run Args[0] may take another element: no reused
	// element ended it, and it has fewer than its maximum (flow.resumeMore).
	FlowResumeMore
	// FlowResumeNext reuses the next element of the run Args[0] that its record holds, replaying what
	// evaluating it did, and sets Dst[0] to its value and Dst[1] to true; Dst[1] is false when the
	// repetition evaluates its element instead. A reuse polls the budget controls without charging
	// (flow.resumeNext).
	FlowResumeNext
	// FlowResumeBegin starts evaluating an element of the run Args[0], after the charge of its
	// iteration (flow.resumeBegin).
	FlowResumeBegin
	// FlowResumeEnd ends evaluating the element of the run Args[0] with the success of the bool
	// Args[1] and the value Args[2], and records the element if it matched (flow.resumeEnd).
	FlowResumeEnd
	// FlowResumeFinish records the run Args[0] of a repetition that matched (flow.resumeFinish).
	FlowResumeFinish
	// FlowMergeRenamed merges the expectations Args[1], recorded at the int Args[0], into the
	// enclosing scope, each that is no message renamed to the expectation of the immediate
	// FlowExpectOf (flow.mergeRenamed, for @diagnostic).
	FlowMergeRenamed
	// FlowScanReachAt sets Dst[0] to the count of a walk of the scan site of the immediate that began
	// at the int Args[0] and has matched the int Args[1] characters: when that count is the scan hot
	// span of the parse, as FlowScanReach does, and otherwise to Args[1]. It is the lowering's test
	// `if count == flow.ScanHotSpan { count = flow.scanReach(...) }`, with the span of the parse, which
	// is flow.ScanHotSpan but in tests that lower it.
	FlowScanReachAt
	// FlowScanEndAt ends a value-free walk of the scan site of the immediate that began at the int
	// Args[0] and matched the int Args[1] characters as FlowScanEnd does when that count is at least the
	// scan hot span of the parse, and does nothing otherwise: the lowering's test
	// `if flow.ScanHotSpan-1 < count { flow.scanEnd(...) }`, with the span of the parse.
	FlowScanEndAt
)

func (FlowOp) Shape source

func (o FlowOp) Shape() (s FlowShape, ok bool)

Shape returns the shape of o; ok is false for an operation that does not exist.

func (FlowOp) String source

func (o FlowOp) String() string

type FlowResumeOf source

type FlowResumeOf struct {
	Site, Min, Max int
	Shiftable      bool
}

FlowResumeOf is the immediate of FlowResumeStart: the repetition, numbered from 1 among those of the grammar that a Document parse resumes, the least and the most elements of the repetition (Max -1 for no bound), and whether the values of its elements hold no position, so that a run may reuse old elements that an edit moved.

type FlowRunWalkOf source

type FlowRunWalkOf struct {
	Site          int
	Build, KeyEnv bool
}

FlowRunWalkOf is the immediate of FlowRunBegin: the run site, whether the iterations of the walk have values (FlowSkipValue), which FlowSkipKids gathers in a list, and whether its records are keyed by the values of the variables of the site's run key (Module.RunKeys) besides the position.

type FlowShape source

type FlowShape struct {
	Name      string
	Dst, Args int
	Target    bool
	Imm       FlowImm
}

FlowShape is what the instructions of an operation hold: the number of destination and argument registers, whether the instruction has a jump target, and its kind of immediate.

type FlowShapeOf source

type FlowShapeOf struct {
	Elems, Shared, Tuple bool
	K, Mode              int
}

FlowShapeOf is the immediate of FlowShapeSingle, FlowShapeCapture and FlowScopeValue, the shape of an expression's value: Elems if the value is the tuple node of its elements, Shared if that node is shared through a splice rule's memo entry, K the number of valued elements, Mode how the value where it takes one element is made (0 no value, 1 the element itself, 2 the tuple node of the elements), and Tuple if the value of a scope with this body is the tuple node.

type FlowSilenceOf source

type FlowSilenceOf struct {
	Delta int
	Site  int
	Ok    bool
}

FlowSilenceOf is the immediate of FlowSilent: Delta 1 enters a lookahead and -1 leaves it, whose expression matched if Ok. Site is the lookahead's site in the parser's observation of lookaheads, or -1 for a lookahead that is no observation.

type LineInfo source

type LineInfo struct {
	Scope  []int
	Action int // action expression of the action (index of Actions, -1 if none)
	// Flow is the Flow body of the line. It returns the value
	// of the line's expression, which Shape makes the value of the line as SCOPEVALUE and
	// SHAPESINGLE do: the scope value of an operand line, and the single of an operator line.
	Flow  *Flow
	Shape FlowShapeOf
}

LineInfo is an Operators Rule operand line or operator line.

type Module source

type Module struct {
	Strings []string   // string table
	Classes []Class    // character class table
	Types   []TypeInfo // type table (struct fields and named node types)
	Scopes  [][]int    // scopes of repetition elements (lists of capture names)
	Vars    []VarInfo  // variable table
	Rules   []RuleInfo // rule table
	// Starts holds the rule table indices of the start rules, the rules where parsing may begin,
	// in declaration order, and DefaultStart that of the default one.
	Starts         []int
	DefaultStart   int
	OperatorsRules []OperatorsRuleInfo
	// Actions holds the action code of the actions, predicates and bindings.
	Actions []Action
	// SourceInfo reports that the values record source information (compiler.WithSourceInfo).
	SourceInfo bool
	// DefaultValue reports that value actions were dropped for default values
	// (compiler.WithDefaultValue).
	DefaultValue bool
	// RunKeys holds the keys of the run sites that are not free of the environment, by increasing
	// site.
	RunKeys []RunKeyInfo
	// Flows holds the bodies of the rules, selected from Flow IR, by rule index. Every rule has one
	// but a rule whose body is the expression of its Operators Rule (RuleInfo.OperatorsRule).
	Flows map[int]*Flow
	// Streams holds the stream bodies of the rules whose body has a #stream repetition, by rule
	// index: the body that a stream parse runs at the top call of its start rule
	// (FlowStreamOn), whose repetition delivers its elements to the callback instead of collecting
	// them.
	Streams map[int]*Flow
}

Module is a grammar compiled to bytecode. It is the only thing the VM runs. The instruction set and its semantics are specified in spec/bytecode/README.md.

func (*Module) Disassemble source

func (m *Module) Disassemble() string

Disassemble renders the module as human-readable text (for debugging).

func (*Module) InstructionSetVersion source

func (m *Module) InstructionSetVersion() int

InstructionSetVersion is the instruction set of every module: the value model's.

func (*Module) RuleStreams source

func (m *Module) RuleStreams(i int) bool

RuleStreams reports whether rule i can start a stream parse: whether it has a stream body (Module.Streams).

type OperatorInfo source

type OperatorInfo struct {
	ID    int
	Kind  int // OperatorPrefix, OperatorPostfix or OperatorInfix
	Assoc int // AssocLeft, AssocRight or AssocNone
	Level int
	Line  LineInfo
}

OperatorInfo is an Operators Rule operator.

type OperatorsRuleInfo source

type OperatorsRuleInfo struct {
	// SkipFlow is the Flow body of the skip, or nil for an Operators Rule without one.
	SkipFlow     *Flow
	Operands     []LineInfo
	Prefix       []OperatorInfo
	InfixPostfix []OperatorInfo // infix and postfix operators
}

OperatorsRuleInfo is an entry of the Operators Rule table.

type RuleInfo source

type RuleInfo struct {
	Name   int // string table index
	Scope  []int
	Action int // action expression of the action (index of Actions, -1 if none)
	// NodeType is the named node type the rule's value is converted to (string table index, -1 if
	// none).
	NodeType int
	Memo     bool
	Leader   bool
	// Member reports that the rule belongs to a left-recursion cycle and is not its leader.
	Member bool
	// Cycle numbers the left-recursion cycle of a leader or member from 1, and is 0 for any other
	// rule.
	Cycle int
	// PositionDependent reports that the results of the rule may depend on positions: an action or
	// predicate of the rule, or of a rule it calls, reads positions.
	PositionDependent bool
	OperatorsRule     int // OperatorsRule table index (-1 if none)
	// ValueFreeTwin reports that this is a value-free twin, called from places where the value is
	// discarded (its value is always nil). It has the same name as the original rule and is placed
	// after all the original rules in the rule table.
	ValueFreeTwin bool
	// Transient reports that the rule is not memoized in ordinary parses (even if Memo is set). In
	// incremental parsing, Memo applies.
	Transient bool
	// Rec reports that the rule is on a cycle of calls (or calls itself): its entries count
	// against the nesting limit, those of other rules do not.
	Rec bool
	// Where is the text that names the rule in the message of a failed action (string table index),
	// or -1 for the name of the rule. The hidden rule of an inline action names the rule that
	// holds it.
	Where int
	// Splice reports that the rule makes no node of its own: the elements of its value are the
	// elements of the sequence that calls it (a hidden rule without an action).
	Splice bool
	// Value is the shape of the value that the rule makes from the value of its body, as
	// FlowScopeValue makes it with the start of the call, before the captures are attached; nil if
	// the value of the body is the rule's value: a rule with an action, a splice, an Operators Rule
	// or a rule without a value. The Flow bodies of a rule leave this step to the call.
	Value *FlowShapeOf
	// Vars are the variables the rule or its callees read (string table indices, sorted by name).
	// The rule's memo entries are keyed by their values at the call.
	Vars []int
}

RuleInfo describes a rule.

type RunKeyInfo source

type RunKeyInfo struct {
	Site       int
	Vars, Bind []int
}

RunKeyInfo is the key of the records of a run site that is not free of the environment: the variables that its element reads, directly or in the rules it calls, and those it binds (Vars), and those it binds (Bind), as string table indices sorted by name. A walk that takes a record binds the variables of Bind to the values the walk of the record left in them.

type TypeInfo source

type TypeInfo struct {
	Name   int   // string table index
	Fields []int // struct field names (string table indices)
	// Zeros holds the zero descriptor of each field (string table indices, parallel to Fields): how
	// a field that a constructor left unset reads.
	Zeros []int
	// Node is, for a named node type, the underlying node type (string table index of its text,
	// written as the type texts of casts are), and -1 for a struct type.
	Node int
}

TypeInfo describes a type: a struct type, or a named node type.

type VarInfo source

type VarInfo struct {
	Name int // string table index
	Type int // VarInt, VarString or VarBool
	// Int is the initial value of an int variable, and 0 or 1 for a bool variable.
	Int int
	// Str is the string table index of the initial value of a string variable, and -1 for
	// the other types.
	Str int
}

VarInfo describes a variable. Every parse starts with the variable bound to its initial value.