shaderkit

Tools and IntelliSense for GLSL and WGSL.

Table of Contents

• Installation
• Tokenize
• Minify
• Parse
• Generate
• AST
  • Literal
  • Identifier
  • Type
  • VariableDeclaration
    • VariableDeclarator
  • StructDeclaration
  • FunctionDeclaration
  • UnaryExpression
  • BinaryExpression
  • TernaryExpression
  • CallExpression
  • MemberExpression
  • ArrayExpression
  • BlockStatement
  • IfStatement
  • ForStatement
  • WhileStatement
  • DoWhileStatement
  • SwitchStatement
    • SwitchCase
  • ReturnStatement
  • PreprocessorStatement
  • PrecisionStatement
  • ContinueStatement
  • BreakStatement
  • DiscardStatement

Installation

To install, use your preferred package manager:

npm install shaderkit
yarn add shaderkit
pnpm add shaderkit

Or, use a CDN:

<script type="module">
  import * as shaderkit from 'https://unpkg.com/shaderkit'
</script>

Tokenize

Tokenizes a string of GLSL or WGSL code, returning an array of Token objects, where each Token object represents a single syntax feature in the input code.

interface Token {
  type: 'whitespace' | 'comment' | 'symbol' | 'bool' | 'float' | 'int' | 'identifier' | 'keyword'
  value: string
}

GLSL Example

import { tokenize } from 'shaderkit'

const code = 'void main() { gl_Position = vec4(0, 0, 0, 1); }'
const tokens = tokenize(code)

console.log(tokens)

The output of the above code will be:

[
  { "type": "keyword", "value": "void" },
  { "type": "whitespace", "value": " " },
  { "type": "identifier", "value": "main" },
  { "type": "symbol", "value": "(" },
  { "type": "symbol", "value": ")" },
  { "type": "whitespace", "value": " " },
  { "type": "symbol", "value": "{" },
  { "type": "whitespace", "value": " " },
  { "type": "keyword", "value": "gl_Position" },
  { "type": "whitespace", "value": " " },
  { "type": "symbol", "value": "=" },
  { "type": "whitespace", "value": " " },
  { "type": "keyword", "value": "vec4" },
  { "type": "symbol", "value": "(" },
  { "type": "int", "value": "0" },
  { "type": "symbol", "value": "," },
  { "type": "whitespace", "value": " " },
  { "type": "int", "value": "0" },
  { "type": "symbol", "value": "," },
  { "type": "whitespace", "value": " " },
  { "type": "int", "value": "0" },
  { "type": "symbol", "value": "," },
  { "type": "whitespace", "value": " " },
  { "type": "int", "value": "1" },
  { "type": "symbol", "value": ")" },
  { "type": "symbol", "value": ";" },
  { "type": "whitespace", "value": " " },
  { "type": "symbol", "value": "}" }
]

WGSL Example

import { tokenize } from 'shaderkit'

const code = '@vertex fn main() -> @builtin(position) vec4<f32> { return vec4(0, 0, 0, 1); }'
const tokens = tokenize(code)

console.log(tokens)

The output of the above code will be:

[
  { "type": "symbol", "value": "@" },
  { "type": "keyword", "value": "vertex" },
  { "type": "whitespace", "value": " " },
  { "type": "keyword", "value": "fn" },
  { "type": "whitespace", "value": " " },
  { "type": "identifier", "value": "main" },
  { "type": "symbol", "value": "(" },
  { "type": "symbol", "value": ")" },
  { "type": "whitespace", "value": " " },
  { "type": "symbol", "value": "->" },
  { "type": "whitespace", "value": " " },
  { "type": "symbol", "value": "@" },
  { "type": "keyword", "value": "builtin" },
  { "type": "symbol", "value": "(" },
  { "type": "keyword", "value": "position" },
  { "type": "symbol", "value": ")" },
  { "type": "whitespace", "value": " " },
  { "type": "keyword", "value": "vec4" },
  { "type": "symbol", "value": "<" },
  { "type": "keyword", "value": "f32" },
  { "type": "symbol", "value": ">" },
  { "type": "whitespace", "value": " " },
  { "type": "symbol", "value": "{" },
  { "type": "whitespace", "value": " " },
  { "type": "keyword", "value": "return" },
  { "type": "whitespace", "value": " " },
  { "type": "keyword", "value": "vec4" },
  { "type": "symbol", "value": "(" },
  { "type": "int", "value": "0" },
  { "type": "symbol", "value": "," },
  { "type": "whitespace", "value": " " },
  { "type": "int", "value": "0" },
  { "type": "symbol", "value": "," },
  { "type": "whitespace", "value": " " },
  { "type": "int", "value": "0" },
  { "type": "symbol", "value": "," },
  { "type": "whitespace", "value": " " },
  { "type": "int", "value": "1" },
  { "type": "symbol", "value": ")" },
  { "type": "symbol", "value": ";" },
  { "type": "whitespace", "value": " " },
  { "type": "symbol", "value": "}" }
]

The following are the supported token types and their descriptions:

Type	Description
whitespace	A sequence of one or more whitespace characters.
comment	A single-line or multi-line comment.
symbol	A symbol, such as an operator or punctuation mark.
bool	A boolean value, either true or false.
float	A floating-point number, represented by a sequence of digits and symbols.
int	An integer number, represented by a sequence of digits.
identifier	A user-defined identifier, such as a variable name or function name.
keyword	A keyword reserved by the language, such as if, else, for, etc.

Minify

Minifies a string of GLSL or WGSL code, returning a minified version of the input code.

const minified: string = minify(code: string, {
  /** Whether to rename variables. Will call a MangleMatcher if specified. Default is `false`. */
  mangle: boolean | ((token: Token, index: number, tokens: Token[]) => boolean)
  /** A map to read and write renamed variables to when mangling. */
  mangleMap: Map<string, string>
  /** Whether to rename external variables such as uniforms or varyings. Default is `false`. */
  mangleExternals: boolean
})

To shared mangled interfaces when using mangleExternal, declare and re-use a mangleMap between shaders:

const options = { mangle: true, mangleExternals: true, mangleMap: new Map() }

// #version 300 es\nin vec2 a;out vec2 b;void main(){b=a;}
minify(`#version 300 es\nin vec2 sstt;out vec2 c;void main(){c=sstt;}`, options)

// #version 300 es\nin vec2 b;out vec4 a[gl_MaxDrawBuffers];void main(){a[0]=b.sstt;}
minify(`#version 300 es\nin vec2 c;out vec4 data[gl_MaxDrawBuffers];void main(){data[0]=c.sstt;}`, options)

Parse

Parses a string of GLSL (WGSL is WIP) code into an AST.

const ast: AST[] = parse(code: string)

Generate

Generates a string of GLSL (WGSL is WIP) code from an AST.

const code: string = generate(ast: AST[], {
  target: 'GLSL' // | 'WGSL'
})

AST

An Abstract Syntax Tree loosely based on ESTree for GLSL and WGSL grammars.

Literal

A shader literal representing a bool, float, int, or uint type.

class Literal {
  value: string
}

Identifier

A variable identifier.

class Identifier {
  value: string
}

Type

Represents a type specifier and its parameters (WGSL specific).

class Type {
  name: string
  parameters: (Type | Literal | Identifier)[] | null
}

VariableDeclaration

A variable declaration with an optional binding layout, type qualifiers, kind (WGSL only), and declarators (e.g. a comma-separated list).

class VariableDeclaration {
  layout: Record<string, string | boolean> | null
  qualifiers: string[]
  kind: 'var' | 'let' | 'const' | null
  type: Type | Identifier
  declarations: VariableDeclarator[]
}

VariableDeclarator

A single named declarator as part of a VariableDeclaration.

class VariableDeclarator {
  name: string
  value: AST | null
}

StructDeclaration

A struct declaration. Can be used as a type or constructor.

class StructDeclaration {
  name: string
  members: VariableDeclaration[]
}

FunctionDeclaration

A function declaration with an optional type qualifier and arguments.

class FunctionDeclaration {
  name: string
  type: Type | Identifier
  qualifiers: string[]
  args: VariableDeclaration[]
  body: BlockStatement | null
}

UnaryExpression

A unary expression with a left or right handed operator.

class UnaryExpression {
  operator: string
  left: AST | null
  right: AST | null
}

BinaryExpression

A binary expression with a left and right operand.

class BinaryExpression {
  operator: string
  left: AST
  right: AST
}

TernaryExpression

A ternary or conditional expression.

class TernaryExpression {
  test: AST
  consequent: AST
  alternate: AST
}

CallExpression

A call expression.

class CallExpression {
  callee: AST
  args: AST[]
}

MemberExpression

A member expression.

class MemberExpression {
  object: AST
  property: AST
}

ArrayExpression

An array expression. members can be empty if uninitialized.

class ArrayExpression {
  type: Type
  members: AST[]
}

BlockStatement

A block statement.

class BlockStatement {
  body: AST[]
}

IfStatement

An if statement.

class IfStatement {
  test: AST
  consequent: AST
  alternate: AST | null
}

ForStatement

A for statement.

class ForStatement {
  init: AST | null
  test: AST | null
  update: AST | null
  body: AST
}

WhileStatement

A while statement.

class WhileStatement {
  test: AST
  body: AST
}

DoWhileStatement

A do-while statement.

class DoWhileStatement {
  test: AST
  body: AST
}

SwitchStatement

A switch statement.

class SwitchStatement {
  discriminant: AST
  cases: SwitchCase[]
}

SwitchCase

A switch-case statement. test is null for a default case.

class SwitchCase {
  test: AST | null
  consequent: AST[]
}

ReturnStatement

A return statement with an optional argument.

class ReturnStatement {
  argument: Literal | Identifier | UnaryExpression | null
}

PreprocessorStatement

A GLSL preprocessor statement with an optional value.

class PreprocessorStatement {
  name: string
  value: AST[] | null
}

PrecisionStatement

A GLSL precision statement.

class PrecisionStatement {
  precision: 'lowp' | 'mediump' | 'highp'
  type: Type
}

ContinueStatement

A continue statement.

class ContinueStatement {}

BreakStatement

A break statement.

class BreakStatement {}

DiscardStatement

A discard statement.

class DiscardStatement {}