AST & Custom Generators

FlyQL’s parser produces an Abstract Syntax Tree (AST) — a structured representation of the query that you can traverse to generate output for any target. The built-in SQL generators (ClickHouse, PostgreSQL, StarRocks) are all implemented by walking this same AST. You can build your own generator for Elasticsearch queries, MongoDB filters, Prometheus selectors, log grep commands, or any other target.

AST Node Types

The AST consists of three types: Node (tree structure), Expression (leaf conditions), and Key (field references).

Node

A Node is either a branch (combining two subtrees with a boolean operator) or a leaf (holding a single expression). It is the fundamental building block of the AST.

Field	Type	Description
boolOperator	string	"and" or "or" — how left and right are combined. Empty on leaf nodes.
expression	Expression or null	The condition, if this is a leaf node. null on branch nodes.
left	Node or null	Left child subtree. null on leaf nodes.
right	Node or null	Right child subtree. null on leaf nodes.
negated	boolean	true if this node is wrapped in not.

A node is either a leaf (expression is set, left/right are null) or a branch (left/right are set, expression is null). It is never both.

Expression

An Expression represents a single condition — a key, an operator, and a value.

Field	Type	Description
key	Key	The field being tested (e.g. status, user.metadata.role).
operator	string	One of: =, !=, ~, !~, >, <, >=, <=, truthy, in, not in, has, not has.
value	any	The comparison value. A string or number for scalar operators. Empty for truthy, in, and not in.
values	array or null	List of values for in and not in operators. null for other operators.
valuesType	string or null	Type of list values: "string" or "number". null when values is not used.

Key

A Key represents the field path being queried.

Field	Type	Description
segments	string[]	Path segments. ["status"] for a simple key, ["user", "metadata", "role"] for nested.
raw	string	The original key string as written in the query.
isSegmented	boolean	true if the key has more than one segment (i.e. uses dot notation).

Operators

All operators are available as constants you can import:

// JavaScript
import { Operator, BoolOperator } from 'flyql'

Operator.EQUALS       // "="
Operator.NOT_EQUALS   // "!="
Operator.REGEX        // "~"
Operator.NOT_REGEX    // "!~"
Operator.GREATER_THAN // ">"
Operator.LOWER_THAN   // "<"
Operator.GREATER_OR_EQUALS_THAN // ">="
Operator.LOWER_OR_EQUALS_THAN   // "<="
Operator.TRUTHY       // "truthy"
Operator.IN           // "in"
Operator.NOT_IN       // "not in"
Operator.HAS          // "has"
Operator.NOT_HAS      // "not has"

BoolOperator.AND      // "and"
BoolOperator.OR       // "or"

// Go
import flyql "github.com/iamtelescope/flyql/golang"

flyql.OpEquals          // "="
flyql.OpNotEquals       // "!="
flyql.OpRegex           // "~"
flyql.OpNotRegex        // "!~"
flyql.OpGreater         // ">"
flyql.OpLess            // "<"
flyql.OpGreaterOrEquals // ">="
flyql.OpLessOrEquals    // "<="
flyql.OpTruthy          // "truthy"
flyql.OpIn              // "in"
flyql.OpNotIn           // "not in"
flyql.OpHas             // "has"
flyql.OpNotHas          // "not has"

flyql.BoolOpAnd      // "and"
flyql.BoolOpOr       // "or"

# Python
from flyql.core.constants import Operator, BoolOperator

Operator.EQUALS       # "="
Operator.NOT_EQUALS   # "!="
Operator.REGEX        # "~"
Operator.NOT_REGEX    # "!~"
Operator.GREATER_THAN # ">"
Operator.LOWER_THAN   # "<"
Operator.GREATER_OR_EQUALS_THAN # ">="
Operator.LOWER_OR_EQUALS_THAN   # "<="
Operator.TRUTHY       # "truthy"
Operator.IN           # "in"
Operator.NOT_IN       # "not in"
Operator.HAS          # "has"
Operator.NOT_HAS      # "not has"

BoolOperator.AND      # "and"
BoolOperator.OR       # "or"

AST Structure Examples

Simple condition

Query: status=200

Node (leaf)
  expression:
    key: { segments: ["status"], raw: "status" }
    operator: "="
    value: 200

Boolean combination

Query: status=200 and method="GET"

Node (branch, boolOperator: "and")
  left: Node (leaf)
    expression:
      key: { segments: ["status"], raw: "status" }
      operator: "="
      value: 200
  right: Node (leaf)
    expression:
      key: { segments: ["method"], raw: "method" }
      operator: "="
      value: "GET"

Negation

Query: not status=500

Node (leaf, negated: true)
  expression:
    key: { segments: ["status"], raw: "status" }
    operator: "="
    value: 500

Nested boolean with parentheses

Query: env=prod and (status=500 or level=error)

Node (branch, boolOperator: "and")
  left: Node (leaf)
    expression:
      key: { segments: ["env"], raw: "env" }
      operator: "="
      value: "prod"
  right: Node (branch, boolOperator: "or")
    left: Node (leaf)
      expression:
        key: { segments: ["status"], raw: "status" }
        operator: "="
        value: 500
    right: Node (leaf)
      expression:
        key: { segments: ["level"], raw: "level" }
        operator: "="
        value: "error"

List membership

Query: status in [200, 201, 204]

Node (leaf)
  expression:
    key: { segments: ["status"], raw: "status" }
    operator: "in"
    value: ""
    values: [200, 201, 204]
    valuesType: "number"

Truthy check

Query: active

Node (leaf)
  expression:
    key: { segments: ["active"], raw: "active" }
    operator: "truthy"
    value: ""

Nested key

Query: user.metadata.role="admin"

Node (leaf)
  expression:
    key: { segments: ["user", "metadata", "role"], raw: "user.metadata.role", isSegmented: true }
    operator: "="
    value: "admin"

Traversing the AST

The AST is a binary tree. To process it, use recursive traversal:

1. If the node has an expression (leaf node) — handle the condition.
2. If the node has left/right (branch node) — recurse into children and combine with boolOperator.
3. If negated is true — wrap the result in negation.

Building a Custom Generator

Here is a complete example of a custom generator that converts a FlyQL AST into an Elasticsearch Query DSL object.

JavaScript

import { parse } from 'flyql'
import { Operator } from 'flyql'

function generateElasticsearch(node) {
  if (!node) return { match_all: {} }

  let result

  if (node.expression) {
    // Leaf node — convert the expression
    result = expressionToES(node.expression)
  } else {
    // Branch node — recurse and combine
    const left = generateElasticsearch(node.left)
    const right = generateElasticsearch(node.right)

    if (node.boolOperator === 'and') {
      result = { bool: { must: [left, right] } }
    } else {
      result = { bool: { should: [left, right], minimum_should_match: 1 } }
    }
  }

  // Handle negation
  if (node.negated) {
    result = { bool: { must_not: [result] } }
  }

  return result
}

function expressionToES(expr) {
  const field = expr.key.raw

  switch (expr.operator) {
    case Operator.EQUALS:
      return { term: { [field]: expr.value } }

    case Operator.NOT_EQUALS:
      return { bool: { must_not: [{ term: { [field]: expr.value } }] } }

    case Operator.REGEX:
      return { regexp: { [field]: String(expr.value) } }

    case Operator.NOT_REGEX:
      return { bool: { must_not: [{ regexp: { [field]: String(expr.value) } }] } }

    case Operator.GREATER_THAN:
      return { range: { [field]: { gt: expr.value } } }

    case Operator.GREATER_OR_EQUALS_THAN:
      return { range: { [field]: { gte: expr.value } } }

    case Operator.LOWER_THAN:
      return { range: { [field]: { lt: expr.value } } }

    case Operator.LOWER_OR_EQUALS_THAN:
      return { range: { [field]: { lte: expr.value } } }

    case Operator.IN:
      return { terms: { [field]: expr.values || [] } }

    case Operator.NOT_IN:
      return { bool: { must_not: [{ terms: { [field]: expr.values || [] } }] } }

    case Operator.TRUTHY:
      return { exists: { field } }

    case Operator.HAS:
      return { wildcard: { [field]: `*${expr.value}*` } }

    case Operator.NOT_HAS:
      return { bool: { must_not: [{ wildcard: { [field]: `*${expr.value}*` } }] } }

    default:
      throw new Error(`unsupported operator: ${expr.operator}`)
  }
}

// Usage
const result = parse('status=200 and message~"error.*" and env in ["prod", "staging"]')
const esQuery = generateElasticsearch(result.root)
console.log(JSON.stringify(esQuery, null, 2))

Go

package main

import (
  "encoding/json"
  "fmt"
  flyql "github.com/iamtelescope/flyql/golang"
)

func generateElasticsearch(node *flyql.Node) map[string]any {
  if node == nil {
    return map[string]any{"match_all": map[string]any{}}
  }

  var result map[string]any

  if node.Expression != nil {
    // Leaf node — convert the expression
    result = expressionToES(node.Expression)
  } else {
    // Branch node — recurse and combine
    left := generateElasticsearch(node.Left)
    right := generateElasticsearch(node.Right)

    if node.BoolOperator == flyql.BoolOpAnd {
      result = map[string]any{
        "bool": map[string]any{"must": []any{left, right}},
      }
    } else {
      result = map[string]any{
        "bool": map[string]any{"should": []any{left, right}, "minimum_should_match": 1},
      }
    }
  }

  // Handle negation
  if node.Negated {
    result = map[string]any{
      "bool": map[string]any{"must_not": []any{result}},
    }
  }

  return result
}

func expressionToES(expr *flyql.Expression) map[string]any {
  field := expr.Key.Raw

  switch expr.Operator {
  case flyql.OpEquals:
    return map[string]any{"term": map[string]any{field: expr.Value}}
  case flyql.OpNotEquals:
    return map[string]any{"bool": map[string]any{"must_not": []any{map[string]any{"term": map[string]any{field: expr.Value}}}}}
  case flyql.OpGreater:
    return map[string]any{"range": map[string]any{field: map[string]any{"gt": expr.Value}}}
  case flyql.OpLess:
    return map[string]any{"range": map[string]any{field: map[string]any{"lt": expr.Value}}}
  case flyql.OpIn:
    return map[string]any{"terms": map[string]any{field: expr.Values}}
  case flyql.OpNotIn:
    return map[string]any{"bool": map[string]any{"must_not": []any{map[string]any{"terms": map[string]any{field: expr.Values}}}}}
  case flyql.OpTruthy:
    return map[string]any{"exists": map[string]any{"field": field}}
  case flyql.OpRegex:
    return map[string]any{"regexp": map[string]any{field: fmt.Sprintf("%v", expr.Value)}}
  default:
    panic(fmt.Sprintf("unsupported operator: %s", expr.Operator))
  }
}

func main() {
  result, err := flyql.Parse(`status=200 and env in ["prod", "staging"]`)
  if err != nil {
    panic(err)
  }

  esQuery := generateElasticsearch(result.Root)
  data, _ := json.MarshalIndent(esQuery, "", "  ")
  fmt.Println(string(data))
}

Python

from flyql.core.parser import parse
from flyql.core.constants import Operator, BoolOperator

def generate_elasticsearch(node):
    if node is None:
        return {"match_all": {}}

    if node.expression is not None:
        # Leaf node — convert the expression
        result = expression_to_es(node.expression)
    else:
        # Branch node — recurse and combine
        left = generate_elasticsearch(node.left)
        right = generate_elasticsearch(node.right)

        if node.bool_operator == BoolOperator.AND.value:
            result = {"bool": {"must": [left, right]}}
        else:
            result = {"bool": {"should": [left, right], "minimum_should_match": 1}}

    # Handle negation
    if node.negated:
        result = {"bool": {"must_not": [result]}}

    return result

def expression_to_es(expr):
    field = expr.key.raw
    op = expr.operator  # operator is stored as a string value, e.g. "=", "!=", "~"

    if op == Operator.EQUALS.value:
        return {"term": {field: expr.value}}
    elif op == Operator.NOT_EQUALS.value:
        return {"bool": {"must_not": [{"term": {field: expr.value}}]}}
    elif op == Operator.REGEX.value:
        return {"regexp": {field: str(expr.value)}}
    elif op == Operator.NOT_REGEX.value:
        return {"bool": {"must_not": [{"regexp": {field: str(expr.value)}}]}}
    elif op == Operator.GREATER_THAN.value:
        return {"range": {field: {"gt": expr.value}}}
    elif op == Operator.LOWER_THAN.value:
        return {"range": {field: {"lt": expr.value}}}
    elif op == Operator.IN.value:
        return {"terms": {field: expr.values or []}}
    elif op == Operator.NOT_IN.value:
        return {"bool": {"must_not": [{"terms": {field: expr.values or []}}]}}
    elif op == Operator.TRUTHY.value:
        return {"exists": {"field": field}}
    elif op == Operator.HAS.value:
        return {"wildcard": {field: f"*{expr.value}*"}}
    elif op == Operator.NOT_HAS.value:
        return {"bool": {"must_not": [{"wildcard": {field: f"*{expr.value}*"}}]}}
    else:
        raise ValueError(f"unsupported operator: {op}")


# Usage
result = parse('status=200 and message~"error.*" and env in ["prod", "staging"]')
es_query = generate_elasticsearch(result.root)
print(es_query)

How Built-In Generators Work

The built-in SQL generators follow the same recursive pattern shown above. Each generator:

1. Receives the root Node and a column schema map.
2. Recurses through the tree: branch nodes become (left AND right) or (left OR right), leaf nodes become SQL conditions.
3. Handles each operator by switching on expression.operator and emitting dialect-specific SQL.
4. Wraps negated nodes in NOT (...).

The key difference from a custom generator is that SQL generators also validate expressions against a column schema (checking types, allowed values, and column existence). A custom generator can skip this step or implement its own validation logic.

Exported Types Reference

JavaScript

import {
  parse,               // Parse a query string into an AST
  Parser,              // Parser class (advanced usage)
  Node,                // AST branch/leaf node
  Expression,          // Leaf condition
  Key,                 // Field path
  Operator,            // Comparison operator constants
  BoolOperator,        // Boolean operator constants ("and", "or")
  FlyqlError,          // Base error class
  ParserError,         // Parse error (extends FlyqlError, includes errno)
} from 'flyql'

Go

import flyql "github.com/iamtelescope/flyql/golang"

// flyql.Parse(query string) (*ParseResult, error)
// flyql.Node        — AST node struct
// flyql.Expression  — leaf condition struct
// flyql.Key         — field path struct
// flyql.NewNode(), flyql.NewExpressionNode(), flyql.NewBranchNode()
//
// Operator constants: flyql.OpEquals, flyql.OpIn, flyql.OpTruthy, etc.
// Bool operator constants: flyql.BoolOpAnd, flyql.BoolOpOr

Python

from flyql.core.parser import parse, Parser
from flyql.core.tree import Node
from flyql.core.expression import Expression
from flyql.core.key import Key
from flyql.core.constants import Operator, BoolOperator
from flyql.core.exceptions import FlyqlError, ParserError