Latticework

Command Palette

Search for a command to run...

Compilers Fundamentals

Abstract Syntax Trees

22 min

Explanation

Once you have tokens, the parser groups them into a tree that reflects the expression's actual structure — the Abstract Syntax Tree (AST). This is where operator precedence gets encoded: 1 + 2 * 3 must parse so that 2 * 3 is evaluated as a single unit before being added to 1, which means the AST has to nest the multiplication INSIDE the addition, not the other way around.

def parse(tokens):
    pos = [0]
    def peek():
        return tokens[pos[0]] if pos[0] < len(tokens) else None
    def advance():
        tok = tokens[pos[0]]
        pos[0] += 1
        return tok
    def parse_factor():
        tok = advance()
        if tok == "(":
            node = parse_expr()
            advance()  # consume ")"
            return node
        return tok
    def parse_term():
        node = parse_factor()
        while peek() in ("*", "/"):
            op = advance()
            node = (op, node, parse_factor())
        return node
    def parse_expr():
        node = parse_term()
        while peek() in ("+", "-"):
            op = advance()
            node = (op, node, parse_term())
        return node
    return parse_expr()
Try it

parse_term() (handling * and /) is called FROM WITHIN parse_expr() (handling + and -) rather than the other way around -- that call-nesting order is precisely what makes higher-precedence operators end up deeper in the tree.

Loading editor…
Explanation

Evaluating an AST is a direct recursive walk: a leaf (a plain number) is already its own value; an internal node is "evaluate both children, then apply the operator." This same recursive-tree-walk shape reappears constantly — it's exactly how Data Structures' trees-bsts module walks a binary search tree, just applied to compute a result instead of searching for a key.

Exercise

Using the provided `parse(tokens)` (already correct — builds an AST as nested tuples `(op, left, right)`, respecting `*`/`/` before `+`/`-`), write `eval_ast(node)`: evaluate the AST and return the numeric result.

Quiz

Why does the AST for `1 + 2 * 3` look like `('+', 1, ('*', 2, 3))` and not `('*', ('+', 1, 2), 3)`?

Checkpoint

You understand how operator precedence gets encoded directly into an AST's shape, and can write a recursive evaluator that walks that tree.