Lesson 8 of 10
Shipping a Parser
A grammar becomes useful when other code can call it. There are three ways to ship one: compile the grammar in your program at start-up, which is what the previous lessons did; generate Go source, so that the program needs neither the engine nor the grammar; or save a compiled grammar and load it. This lesson shows the last two and the choice of a backend.
What You Will Build
A config Go package generated from the configuration grammar, and a compiled grammar file that the command line
accepts in place of the source.
Before You Start
Complete Building Typed Trees: you need config.sxir and a Go module (go mod init example.com/hello).
Generating a Standalone Go Parser
sxir gen turns a grammar into Go source, so your program does not need the SyntaxIR engine or the grammar at run time.
The generated package (parser.go, api.go and a copy of the parser library in internal/libgo) uses only the standard
library and SyntaxIR's small pkg/runtime packages, so your module still requires syntaxir.dev/sxir. -module is the import
path the package will have, so it starts with your module path. Generate it into a config package:
sxir gen -g config.sxir -module example.com/hello/config -o config
and call it:
package main
import (
"fmt"
"log"
"example.com/hello/config" // your module path
)
func main() {
node, err := config.Parse("port = 8080\n")
if err != nil {
log.Fatal(err)
}
fmt.Println(node)
}
Config{Entries: [Entry{Key: Key("port"), Value: Number("8080")}]}
The generated Node type mirrors value.Node, and the generated parser gives the same trees and errors as the
engine. config.ParseRule(name, input) starts at another rule. The generated entry points accept functional
options: config.Parse(input, config.WithUnit(config.Bytes), config.WithLimits(config.Limits{Depth: 600_000})).
sxir gen -max-depth 600000 sets the generated default; omitted or zero keeps 100,000.
Backends and Compiled Grammars
A grammar can run on several backends with identical results: the default closure-compiled engine, a bytecode VM, and a bytecode VM that uses an explicit stack instead of the Go stack for deeply nested input. Choose one on the command line or in Go:
sxir parse -g config.sxir -f sexpr -backend bytecode < example.conf
sxir parse -g config.sxir -recognize -backend bytecode-iterative < example.conf # recognize only, prints ok
n, err := p.Parse("abc=12") // also runtime.WithUnit(input.Bytes); p.Recognize("abc=12") checks only; p.(*vm.Parser).Iterative() selects the iterative VM
sxir compile -g config.sxir -o config.sxirc saves a compiled grammar that sxir parse accepts in place of the
.sxir file (sxir parse -g config.sxirc ...), and vm.LoadParser loads it without recompiling. The file serves
the parses of one mode: -mode parse (the default) for parsing, -mode recognize for -recognize, and -mode document
for documents and traces.
sxir convert -to json config.sxir converts a grammar to JSON, which is handy for tools that generate grammars.
The Running Parsers guide describes the backends and options in detail, and the
Compiled Grammars guide the .sxirc format.
Next Steps
Handling Big and Live Input covers input that does not fit in memory and text that is edited while parsed. The guides Generating Parsers, Generated TypeScript Parsers, Compiled Grammars and Using the Ready-Made Parsers go further, and the cookbook recipe Ship a Parser and Keep It Up to Date shows how to keep generated code in step with the grammar.