Skip to content

4.3. The Generator

Jump to bottom
Ioannis Tsakpinis edited this page Mar 17, 2018 · 4 revisions

This page contains information about the LWJGL Generator and Template modules. Both are written in the Kotlin programming language.

Type System

The most important feature of the new code generator in LWJGL 3 is the type system it employs. Even though all type information is lost in the generated Java code, hence type safety is still an issue for end-users, it simplifies template declarations significantly and eliminates bugs caused by illegal function definitions. New types are defined in a single location and then reused by as many functions as necessary; the mapping between native and Java types happens once and any bug fixes are localized. The types themselves look (almost) exactly like their native representation and the number of required annotations and modifiers in templates are minimized.

When Java add supports for value types (Project Valhalla), a future version of LWJGL will be able to use this type information to generate type-safe bindings.

The type system is defined in org.lwjgl.generator.Types.kt and the most basic interface is the NativeType with 2 properties: a) the native type name, which is a simple string used directly in the generated native code and b) a TypeMapping instance, which describes how that native is mapped to an appropriate Java type. More complex types have additional properties. The full type system includes the following types:

  • NativeType
    • VoidType
    • OpaqueType (cannot be used directly, must be wrapped in PointerType)
    • DataType (types that can be used as parameters or struct members)
      • ValueType (passed by-value)
        • PrimitiveType
          • IntegerType
          • CharType
        • StructType (mapped to org.lwjgl.system.Struct subclass)
      • ReferenceType (passed by-reference)
        • JObjectType (only type passed as Java object to JNI)
        • PointerType (mapped to raw pointer or buffer)
          • ArrayType (mapped to Java array)
          • CharSequenceType (mapped to String/CharSequence)
          • ObjectType (mapped to a Java class)
            • CallbackType (mapped to callback interface)
            • C++ classes (not supported yet)

Common types that are used across different bindings are defined in org.lwjgl.generator.GlobalTypes.kt. For example, int32_t is defined as:

val int32_t = IntegerType("int32_t", PrimitiveMapping.INT)

Type Mappings

The generator supports the following type mappings:

  • TypeMapping

    • VOID

  • PrimitiveMapping

    • BOOLEAN
    • BOOLEAN4 (32-bit integer type mapped to Java boolean)
    • BYTE
    • SHORT
    • INT
    • LONG
    • POINTER (integer type with enough precision to store a pointer)
    • FLOAT
    • DOUBLE

  • CharMapping

    • ASCII
    • UTF8
    • UTF16

  • PointerMapping

    • DATA_POINTER // pointer to array of pointers
    • DATA_BOOLEAN // pointer to array of booleans
    • DATA_BYTE // pointer to array of bytes
    • DATA_SHORT // pointer to array of shorts
    • DATA_INT // pointer to array of ints
    • DATA_LONG // pointer to array of longs
    • DATA_FLOAT // pointer to array of floats
    • DATA_DOUBLE // pointer to array of doubles

Primitives

Primitive types have 2 specializations:

  • IntegerType (adds property unsigned, defaults to false)
  • CharType (requires a CharMapping).

Examples:

val int = IntegerType("int", PrimitiveMapping.INT)
val float = PrimitiveType("float", PrimitiveMapping.FLOAT)
val unsigned_short = IntegerType("unsigned short", PrimitiveMapping.SHORT, unsigned = true)
val TCHAR = CharType("TCHAR", CharMapping.UTF16)
val CGLError = "CGLError".enumType // enum integer
val XID = typedef("XID", unsigned_long) // alias
val Window = typedef("Window", XID) // another alias

Pointers

Pointer types have 4 specializations:

  • ArrayType (used for Java array overloads)
  • CharSequenceType (created with <CharType>.p).
  • ObjectType (adds className which defines a Java wrapper class)
    • CallbackType (adds function)

There are also shortcuts that convert a value type to an array of that type and convert a pointer type to a pointer-to-pointer type. All pointer types can set the includesPointer property, when the native type definition is a typedef that includes the pointer indirection operator (*).

Examples:

"void".opaque.p // opaque pointer (void *)
void.p // data pointer (void *)
void.p.p // pointer-to-data-pointer (void **)
int.p // pointer to a primitive type (int *)
val LPCSTR = typedef(CHAR.const.p, "LPCSTR") // 1-byte-per-char string (LPCSTR)
val VkInstance = ObjectType("VkInstance") // opaque pointer mapped to the `VkInstance` class

Structs

A StructType is a value type specialized for structs, that is configured by a Struct instance.

Examples:

// Simple struct with auto-sized members
val GLFWgammaramp = struct(GLFW_PACKAGE, "GLFWGammaRamp", nativeName = "GLFWgammaramp") {
	unsigned_short_p.member("red", "...")
	unsigned_short_p.member("green", "...")
	unsigned_short_p.member("blue", "...")
	AutoSize("red", "green", "blue")..unsigned_int.member("size", "...")
}

// Unions and nested structs
val VkClearValue = union(VULKAN_PACKAGE, "VkClearValue") {
	VkClearColorValue.member("color", "...")
	VkClearDepthStencilValue.member("depthStencil", "...")
}

// Array members, immutability (no setters)
val VkLayerProperties = struct(VULKAN_PACKAGE, "VkLayerProperties", mutable = false) {
	charUTF8.array("layerName", "...", size = "VK_MAX_EXTENSION_NAME_SIZE")
	uint32_t.member("specVersion", "...")
	uint32_t.member("implementationVersion", "...")
	charUTF8.array("description", "...", size = "VK_MAX_DESCRIPTION_SIZE")
}

Templates

The Templates module contains a package for each API we'd like to create bindings for. An API is usually split in many different classes, each of which is defined through a NativeClass instance. These instances are discovered reflectively using the following convention:

  • Each API package contains a sub-package with the name templates.
  • Public top-level functions that take no arguments and return a NativeClass instance will be called reflectively and the returned instance will be used for code generation.
  • Any struct types, callback types or custom classes encountered will be registered and generated automatically in a second pass.

The NativeClass instances are initialized using Kotlin's builder pattern. There are properties and methods for setting the class documentation, the Access level (PUBLIC or INTERNAL, i.e. "package private") and custom Java and native imports. After that there are zero or more constant blocks and then zero or more function definitions. This order is not strictly required, but it's the most common one. Some examples:

The root packages for each API usually contain global definitions for types specific to that API. In addition there might be function provider implementations (discussed below) and configuration methods; public top-level methods, with no arguments or return values, that are called reflectively. These are usually used to register types that are not part of any function definition, but are still useful, e.g. may appear in callback functions or void * arguments.

Modifiers

The type system is quite helpful and many code transformations are applied automatically. For other kinds of transformations, the LWJGL generator uses modifiers, which may be applied to functions, function arguments and function return types. The modifiers are similar to the annotations used by the LWJGL 2 generator, except that they are standard Java objects. Many modifiers have certain constraints (e.g. the target parameter must be a pointer type) that are checked at generation time.

Below is a list of the most common modifiers and their descriptions.

Parameters & return types

  • AutoSize

    Can be used on integer primitive arguments that constrain the length of one or more other pointer arguments. The constructor requires one or more strings, which must be the names of the constrained pointer arguments. This modifier caues the size argument to become implicit; the remaining() capacity of the first constrained argument is used as the size value and any other constrained arguments are checked against that size.

    The AutoSize modifier also allows the size value to be scaled. This is useful when the .remaining() expression returns the number of elements in the buffer, but the size argument expects bytes or fixed-size groups of those elements.

  • AutoSizeResult

    Can be used on input integer or output pointer-to-integer arguments, on functions that return arrays of values (i.e. a data pointer). If the function call is successful, the buffer that will be constructed on the returned pointer will have capacity equal to the value specified by the input or stored in the output size argument. In the case of an output argument, the argument is hidden by this modifier.

  • Check

    Can be used to perform a custom check against a buffer argument's remaining capacity. The Check constructor allows 3 properties to be defined:

    • expression : A string expression that evaluates to an int. This value is required and the buffer's remaining() must be at least equal to expression.
    • debug : If true, the check will only be performed in debug mode. This is useful when evaluating the expression is expensive (e.g. uses a glGetX function). Defaults to false.

  • nullable

    Can be used on pointer arguments to allow null values.

  • nullTerminated

    Can be used on data pointer arguments to specify that the data is null-terminated. A check will be added to ensure that the last element of the buffer is equal to 0. Multiple bytes will be checked if the buffer is not a ByteBuffer.

  • MultiType

    Can be used on a void * data pointer to define more specific types. For each type, an alternative method will be generated with the argument type replaced by the corresponding NIO buffer type. The types are defined using PointerMapping constants.

  • Return

    Can be used on an output string argument to generate an alternative method that returns an automatically constructed String value.

  • ReturnParam

    Can be used on an output data pointer argument to generate an alternative method in which the output argument becomes the method return value.

  • SingleValue

    Can be used on an data pointer argument to generate an alternative method in which the buffer argument is replaced with a single primitive value.

Functions

  • DependsOn

    Can be used on a function to specify that the availability of that function depends on some other functionality to be present. This is useful for OpenGL extension functions that depend on other extensions to be available in order for them to be exposed.

  • Reuse

    Can be used on a function to specify that a native method should not be generated. Instead, an existing native method of another class is used. This is commonly used in OpenGL extensions that were introduced at the same time as a new version of the OpenGL specification, with the same functionality.

  • Code [ ADVANCED ]

    Can be used on a function to inject custom code in the generated Java and native source. The Code modifier is configured with the following properties:

    • javaInit : Java code to inject before everything else.
    • javaBeforeNative : Java code to inject before the native method call.
    • javaAfterNative : Java code to inject after the native method call.
    • javaFinally : Java code to inject in a finally block after the native method call.
    • nativeBeforeCall : Native code to inject before the native function call.
    • nativeCall : Native code that replaces the native function call.
    • nativeAfterCall : Native code to inject after the native function call.

  • macro

    Can be used on a function without arguments to mark it as a macro.

  • private or internal

    Can be used on a function to make it private or "package private" respectively. Note that all functions in a NativeClass inherit its Access level by default.

Function Providers [ ADVANCED ]

Function providers are API-specific FunctionProvider implementations that handle the initialization of function addresses, check the availability of groups of functionality (e.g. extensions) and are responsible for generating the source code for the various "Capability" classes that LWJGL uses. Currently there are implementations for EGL, OpenAL, OpenCL, OpenGL, OpenGL ES and Vulkan.

Contribute

back

Clone this wiki locally