Package sxir/pkg/runtime/instrument

import "syntaxir.dev/sxir/pkg/runtime/instrument"

Overview

Package instrument defines the observation of a parse: trace events of rule calls, their call sites, and profiles that aggregate them. Every SyntaxIR parser reports them with the same types and meaning; spec/runtime/observability.md defines the meaning.

Index

Types

type CallSite source

type CallSite struct {
	input.Location
	Level int `json:"level"`
}

CallSite is where a rule was called: the position and the binding level of the call.

type Profile source

type Profile struct {
	// contains filtered or unexported fields
}

Profile accumulates the cost of each rule over the parses it is given to. Its zero value is ready to use. It is not safe for concurrent use; parses that run one within another on the same goroutine (from a trace function) are accounted separately. It keeps no input, nodes or history of a parse after the parse returns, only the emptied buffers of its recorder for the next parse to reuse. Snapshot reads it.

func (*Profile) Hotspot source

func (pr *Profile) Hotspot() string

Hotspot returns the rule that the profile ranks first by repeated evaluations: the rule with the highest MaxEvals, the most evaluating calls at one position and level, with at least one repeated evaluation (MaxEvals above 1). A tie goes to the rule with the higher Repeats, and then to the first rule in name order. It returns the empty string if no rule evaluated twice at one position and level, and for a nil profile.

func (*Profile) Recorder source

func (pr *Profile) Recorder() *ProfileRecorder

Recorder returns a recorder of one parse into pr, or nil if pr is nil. The methods of a nil recorder do nothing. The recorder reuses the capacity of the buffers of the last recorder of pr that was closed, unless a recorder still open (an enclosing parse) took them.

func (*Profile) Snapshot source

func (pr *Profile) Snapshot() ProfileSnapshot

Snapshot returns a copy of the counts, which later parses do not change.

type ProfileRecorder source

type ProfileRecorder struct {
	// contains filtered or unexported fields
}

ProfileRecorder adds the events of one parse to a Profile. Parsers make one per parse (Profile.Recorder), pass it every event of the parse in order (Record), and close it when the parse returns (Close).

func (*ProfileRecorder) Close source

func (r *ProfileRecorder) Close()

Close ends the recording of the parse. Calls that began and did not complete stay counted in Entered only. Close empties the buffers of r and gives their capacity back to the profile, for the next recorder; r must not be used after it.

func (*ProfileRecorder) Discard source

func (r *ProfileRecorder) Discard(pos int)

Discard tells r that the parse no longer holds the input before pos (a stream discarded it), so that no rule is evaluated there again. A stream parser calls it as it discards input, which keeps the evaluation counts of a long stream bounded.

func (*ProfileRecorder) Hotspot source

func (r *ProfileRecorder) Hotspot() string

Hotspot returns the Hotspot of the profile that the recorder adds to, as of the events recorded so far. It is the empty string for a nil recorder.

func (*ProfileRecorder) Record source

func (r *ProfileRecorder) Record(e TraceEvent)

Record adds the event e. A parser calls it before the trace functions of the parse handle e, and Resume after they have, so that their time is not counted. Record reads the clock once: the time of an event is when Record is called.

func (*ProfileRecorder) Resume source

func (r *ProfileRecorder) Resume()

Resume tells r that the trace functions have handled the last event recorded: the time from that Record to Resume is not counted. A parser that ran no trace function for the event need not call it; the time of recording then counts as the cost of measuring.

type ProfileSnapshot source

type ProfileSnapshot struct {
	// Parses is the number of parses profiled.
	Parses int `json:"parses"`
	// Entered, Calls, Evals, MemoHits and Skipped total the rules' counts.
	Entered  int `json:"entered"`
	Calls    int `json:"calls"`
	Evals    int `json:"evals"`
	MemoHits int `json:"memoHits"`
	Skipped  int `json:"skipped"`
	// ReusedElements totals the repetition elements reused by completed start rule calls. The
	// rows of nested rules include this work too, so they are not added to it.
	ReusedElements int `json:"reusedElements"`
	// Examined totals, over the completed start rule calls, the input they examined.
	Examined int `json:"examined"`
	// Time totals the time of the completed start rule calls (in nanoseconds in JSON).
	Time time.Duration `json:"time"`
	// Rules holds the rules called or skipped, sorted by name.
	Rules []RuleProfile `json:"rules"`
}

ProfileSnapshot is a copy of the counts of a Profile, with Rules sorted by rule name. Entered counts calls that began; the other counts and the times cover only the calls that completed (reported their exit). Positions are in the position unit of each parse, not normalized.

func (ProfileSnapshot) Hints source

func (s ProfileSnapshot) Hints() []string

Hints returns observations about the profile that point at where a grammar does more work than it needs to, most important first. They are heuristics: each names what to look at, not what is wrong.

type RuleProfile source

type RuleProfile struct {
	Rule string `json:"rule"`
	// Entered is the number of calls that began, Calls the number that completed. Each completed
	// call either evaluated the rule body or took the result from the memo (MemoHits); it matched
	// (Matched) or failed (Failed).
	Entered  int `json:"entered"`
	Calls    int `json:"calls"`
	MemoHits int `json:"memoHits"`
	Matched  int `json:"matched"`
	Failed   int `json:"failed"`
	// Evals is the number of body evaluations: one per call not answered by the memo, and one more
	// for each step a left-recursive match grows.
	Evals int `json:"evals"`
	// ReusedElements counts repetition elements reused during the calls, nested calls included.
	// A nested call can report the same reuse as its caller, so rows are not to be summed.
	ReusedElements int `json:"reusedElements"`
	// Consumed totals the input matched by the calls that matched.
	Consumed int `json:"consumed"`
	// Skipped counts the calls the parser skipped (they are not entered), by reason in SkipReasons.
	Skipped     int               `json:"skipped"`
	SkipReasons []SkipReasonCount `json:"skipReasons"`
	// Examined totals the input the calls examined, from the position of each call. Wasted totals
	// the same for the calls that evaluated the body and failed: the work that led to nothing. Both
	// are inclusive: a call counts what the calls it made examined. LongestFail is the most input
	// a single such failed call examined, at LongestFailAt.
	Examined      int      `json:"examined"`
	Wasted        int      `json:"wasted"`
	LongestFail   int      `json:"longestFail"`
	LongestFailAt CallSite `json:"longestFailAt"`
	// Repeats is the number of calls that evaluated the body at a position and binding level where
	// an earlier call of the same parse had evaluated it already. MaxEvals is the most evaluating
	// calls at one position and level, first reached at MaxEvalsAt.
	Repeats    int      `json:"repeats"`
	MaxEvals   int      `json:"maxEvals"`
	MaxEvalsAt CallSite `json:"maxEvalsAt"`
	// Time is the time spent in the calls, nested calls included; a call within a call of the same
	// rule counts once. SelfTime excludes the time of nested calls. Neither includes the trace
	// functions of the parse; both include the cost of measuring, so they are meaningful relative to
	// each other. JSON gives them in nanoseconds.
	Time     time.Duration `json:"time"`
	SelfTime time.Duration `json:"selfTime"`
}

RuleProfile is the cost of one rule.

type SkipReason source

type SkipReason string

SkipReason tells why a parser skipped a call (TraceSkipped).

const SkipFirstCharacter SkipReason = "first-character"

SkipFirstCharacter is the reason of a call skipped because the next character cannot start the rule.

type SkipReasonCount source

type SkipReasonCount struct {
	Reason SkipReason `json:"reason"`
	Count  int        `json:"count"`
}

SkipReasonCount counts the calls of a rule skipped for one reason.

type TraceEvent source

type TraceEvent struct {
	Kind TraceKind `json:"kind"`
	// Rule is the name of the rule called.
	Rule string `json:"rule"`
	// Level is the binding level the rule was called with: for an Operators Rule rule called as
	// e:level, the operators that bind no tighter than that level are left to the caller. It is 0
	// for calls without a level.
	Level int `json:"level"`
	// Depth is the nesting depth of the call: 1 for the start rule.
	Depth int `json:"depth"`
	// Pos is the position at which the call starts.
	Pos int `json:"pos"`
	// Lookahead reports that the call is made where failures record no expectations: inside a
	// lookahead (& or !) or inside the skip expression of an Operators Rule, which is matched as
	// in a lookahead.
	Lookahead bool `json:"lookahead"`

	// Matched reports whether the rule matched.
	Matched bool `json:"matched"`
	// End is the position after the match (Pos if the rule failed).
	End int `json:"end"`
	// Memo reports that the result came from the memo, without evaluating the rule body.
	Memo bool `json:"memo"`
	// Evals is the number of times this call evaluated the rule body: 0 for a result from the
	// memo, 1 usually, and one more per step for a left-recursive rule whose match grows. Bodies of
	// nested calls are not included.
	Evals int `json:"evals"`
	// ReusedElements counts the repetition elements of a Document that this call reused, nested
	// calls included. It is 0 for a parser that does not resume repetitions.
	ReusedElements int `json:"reusedElements"`
	// Examined is the exclusive end of all input the call examined: looking at a character to
	// reject it counts, as do lookaheads. A result from the memo examined what the call that
	// computed it did.
	Examined int `json:"examined"`

	// SkipReason is set on TraceSkipped only.
	SkipReason SkipReason `json:"skipReason"`
	// contains filtered or unexported fields
}

TraceEvent describes the start or the end of a rule call, or a skipped call. Enter and exit events nest: every TraceEnter is followed, after the events of the calls the rule makes, by its TraceExit, unless the parse is aborted (by a runtime error, a limit, an error of a stream's reader or emit function, or a panic of the trace function, which propagates unchanged).

The fields can be kept. The methods LineCol, Text, Failure and Recovered read the state of the parser that reported the event: call them only from the trace function, while it handles the event. A producer that does not provide that state (Source) makes them return zero values.

func (TraceEvent) Failure source

func (e TraceEvent) Failure() *value.SyntaxError

Failure returns, for an exit event, the farthest failure recorded during the call: the position and what was expected there, as a parse that failed there would report it. It is nil when the call recorded no expectation (inside a lookahead, or when nothing failed). A call that matched can have one too: the input after its match was tried and rejected.

func (TraceEvent) LineCol source

func (e TraceEvent) LineCol(pos int) (line, col int)

LineCol returns the 1-based line and column of the position pos, or 0, 0 when the position is unknown: when the parser does not provide it, or, in a stream parse, for a position (other than 0) in input that has been discarded or not read yet.

func (TraceEvent) Recovered source

func (e TraceEvent) Recovered() []*value.SyntaxError

Recovered returns, for an exit event of a call that matched, the syntax errors that #recover recovered from during the call, in nested calls too, or taken from the memo with the call's result. The expectations of a recovered error are not part of Failure: recovering removes them from the record of the call that contains the #recover.

func (TraceEvent) Source source

func (e TraceEvent) Source() TraceSource

Source returns the parser state that WithSource attached to e, or nil.

func (TraceEvent) Text source

func (e TraceEvent) Text() string

Text returns the input the call matched: empty for an event other than the exit of a call that matched, and for input that a stream parse has discarded.

func (TraceEvent) WithSource source

func (e TraceEvent) WithSource(src TraceSource) TraceEvent

WithSource returns e with the parser state src behind its methods. Parsers use it to report events.

type TraceKind source

type TraceKind uint8

TraceKind tells what a TraceEvent reports: the start of a rule call, its end, or a call that the parser skipped.

const (
	// TraceEnter is reported when a rule is called, before the memo is consulted.
	TraceEnter TraceKind = iota
	// TraceExit is reported when the call returns.
	TraceExit
	// TraceSkipped is reported, unpaired, for a call that the parser did not make because it could
	// not match (SkipReason tells why). Producers that do not identify such calls never report it.
	TraceSkipped
)

func (TraceKind) MarshalText source

func (k TraceKind) MarshalText() ([]byte, error)

MarshalText returns the String form of k.

func (TraceKind) String source

func (k TraceKind) String() string

String returns "enter", "exit" or "skipped".

func (*TraceKind) UnmarshalText source

func (k *TraceKind) UnmarshalText(b []byte) error

UnmarshalText sets k from its String form.

type TraceSelection source

type TraceSelection struct {
	// Rules, if not empty, selects the calls of these rules and the calls nested in them.
	Rules []string
	// MaxDepth, if positive, selects the calls at most that deep; the start rule has depth 1.
	MaxDepth int
}

TraceSelection selects the events of a parse that the trace functions receive. Neither field changes the parse or what the profiles record. Options replace a TraceSelection instead of changing it: a Document's defaults share theirs with each parse.

type TraceSource source

type TraceSource interface {
	// LineCol returns the 1-based line and column of pos, or 0, 0 if the position is not
	// available.
	LineCol(pos int) (line, col int)
	// Text returns the input [start, end), or "" if it is not available.
	Text(start, end int) string
	// Failure returns the farthest failure recorded during the call of the current exit event.
	Failure() *value.SyntaxError
	// Recovered returns the errors recovered during the call of the current exit event.
	Recovered() []*value.SyntaxError
}

TraceSource is the parser state behind the events of a parse. It is implemented by the parsers that report events (see WithSource), not by their users. The methods are valid only while the trace function handles an event of the parse; Failure and Recovered describe that event.