Skip to content

Latest commit

 

History

History
477 lines (375 loc) · 18.6 KB

BUILD.adoc

File metadata and controls

477 lines (375 loc) · 18.6 KB
Raw

Vulkan® Specification Build Instructions and Notes

Table of Contents

Introduction

This README describes how to build the Vulkan API specification, reference pages, and/or other related targets.

It documents how to set up your build environment, build steps and targets, and contains some troubleshooting advice.

Building The Spec

First, clone the Khronos Github repository containing the Vulkan specification to your local Linux, Windows, or Mac PC. The repository is located at https://github.com/KhronosGroup/Vulkan-Docs/.

Next, install all the necessary build tools (see Software Dependencies below). If you are using the Khronos-Provided Docker Image, which we strongly recommend, one way to build using the image (assuming a Linux docker host) is:

$ scripts/runDocker

executed from the specification repository root.

runDocker runs the Docker image with the cloned repository mounted under /vulkan and accesses it as a specified user (set to your own user and group ID), so that it does not get filled with files owned by another user. The script leaves you inside a bash shell in the running image. Execute the commands:

$ cd /vulkan
$ ./makeSpec -spec core html

to build an HTML5 specification output for the core Vulkan 1.3 specification, with no extensions included, or

$ cd /vulkan
$ ./makeSpec -spec all all

to build the spec targets html, pdf, styleguide, registry, manhtmlpages, and allchecks, with all registered extensions included.

There are many other ways of using the image, including inside a Continuous Integration pipeline; locally with persistent Docker volume storage of the repository; and so on.

If you are not using our Docker image to build with, and you have a Non-Docker Build Environment with the entire toolchain installed, you can just invoke the same make commands from the repository root.

ℹ️
Note

You can modify the runDocker script to change the docker command-line options, but it is important to always use the Docker image specified in that script, so you have the latest version of the spec toolchain.
ℹ️
Note

  • While it is possible to invoke make directly, this is rarely appropriate or useful. Usually dozens to hundreds of build options must be set to specify the desired set of extensions to include in the specification. The makeSpec python script, which is discussed in more detail below, simplifies this process for common cases.

  • The all target takes a long time to run, and generates outputs that are irrelevant for most users. The html target just generates the HTML output, which is often all that is needed for spec bugfixes not involving extensions.

  • The default make options build a Vulkan 1.3 specification with no optional extensions.

  • The validusage target is not built as part of the all target, due to it needing to be built with all extensions enabled (-spec all). Building the validusage target will fail otherwise.

These targets generate a variety of output documents in the directory specified by the Makefile variable $(OUTDIR) (by default, out/). The checked-in file out/index.html links to all these targets, or they can individually be found as follows:

Vulkan Specification

  • html — Single-file HTML5 in $(OUTDIR)/html/vkspec.html, and KaTeX dependency in $(OUTDIR)/katex

  • chunked — Chunked HTML5 in $(OUTDIR)/html/chap?.html

  • pdf — PDF in $(OUTDIR)/pdf/vkspec.pdf

“styleguide” (Vulkan Documentation and Extensions: Procedures and Conventions)

  • styleguide — Single-file HTML5 in $(OUTDIR)/styleguide.html

XML Registry schema document

  • registry — Single-file HTML5 in $(OUTDIR)/registry.html

Diff spec

  • diff_html — Single-file HTML5 in $(OUTDIR)/html/diff.html

Reference pages

  • manhtmlpages — File-per-entry-point HTML in $(OUTDIR)/man/html/*. Must be built with all extensions enabled (using makeSpec -spec all).

Validator output

  • None at present. The allchecks target writes to standard output unless the underlying script is given additional options.

Valid usage database

  • validusage - json database of all valid usage statements in the specification. Must be built with ./makeAllExts (for now). Output in $(OUTDIR)/validation/validusage.json. A validated schema for the output of this is stored in $(CURDIR)/config/vu-to-json/vu_schema.json

Once you have the basic build working, an appropriate parallelization option, such as -j 8, should significantly speed up the reference page builds.

If you encounter problems refer to the [troubleshooting] section.

Building Specifications For Different API Versions

The Makefile defaults to building a Vulkan 1.3 specification. This is controlled by Asciidoctor attributes passed in the Makefile variable $(VERSIONS) To instead build a Vulkan 1.1 specification, pass

VERSIONS="VK_VERSION_1_0 VK_VERSION_1_1"

on the makeSpec command line.

Building With Extensions Included

Extensions are defined in the same source as the core Specification, but are only conditionally included in the output. Asciidoctor attributes of the same name as the extension are used to define whether the extension is included or not — defining such an attribute will cause the output to include the text for that extension.

When building the specification, the extensions included are those specified as a space-separated list of extension names (e.g. VK_KHR_surface) in the Makefile variable $(EXTENSIONS), usually set on the make command line. When changing the list of extensions, it is critical to remove all generated files using the clean_generated Makefile target, as the contents of generated files depends on $(EXTENSIONS).

The makeSpec wrapper script can clean generated files and then build one or more specification targets for a set of explicitly specified extensions, including all implicit extension dependencies of that set. It accepts these options:

  • -clean - remove generated targets before building

  • -v - print actions as well as executing them

  • -n - print actions without executing them

  • -genpath path - specify path to generated files (default gen)

  • -spec type - build with specified sets of extensions. type may be

    • core - no extensions added (default if not specified)

    • khr - all KHR extensions added

    • ratified - all ratified extensions (KHR and some EXT) added

    • all - all registered extensions added

  • -extension extname - build with specified extension included, as well as the set specified by -spec. Can be given multiple times.

  • All remaining targets are arbitrary make options or targets in the Makefile.

The target(s) passed to these scripts are arbitrary make options, and can be used to set Makefile variables and options discussed above, as well as specify actual build targets. For example, to build the HTML specification with all KHR extensions included as well as a single vendor extension:

$ ./makeSpec -clean -spec khr -extension VK_EXT_debug_report html

The scripts makeAllExts, makeKHR, and makeExt set appropriate options and invoke makeSpec, for backwards compatibility, but are no longer used by Khronos.

The Makefile variable $(APITITLE) defines an additional string which is appended to the specification title. When building with extensions enabled, this should be set to something like (with extension VK_extension_name). The makeSpec script already does this.

The reference pages (the manhtmlpages target) must be built using the -spec all option; there are markup and scripting issues which will probably cause any more restricted set of refpages to fail to build.

Building A Highlighted Extension Diff

The diff_html target in the Makefile can be used to generate a version of the specification which highlights changes made to the specification by the inclusion of a particular set of extensions.

Extensions in the Makefile variable $(EXTENSIONS) define the base extensions to be enabled by the specification, and these will not be highlighted in the output. Extensions in the Makefile variable $(DIFFEXTENSIONS) define the set of extensions whose changes to the text will be highlighted when they are enabled. Any extensions in both variables will be treated as if they were only included in $(DIFFEXTENSIONS). $(DIFFEXTENSIONS) can be set when using the makeSpec script described above.

In the resulting HTML document, content that has been added by one of the extensions will be highlighted with a lime background, and content that was removed will be highlighted with a pink background. Each section has an anchor of #differenceN, with an arrow (⇒) at the end of each section which links to the next difference section. The first diff section is #difference1.

ℹ️
Note

This output is not without errors. It may instead result in visible [.added]##content## and [.removed]##content##, and so also highlights not being rendered. But such visible markup still correctly encapsulates the modified content.

Alternate and Test Builds

If you are just testing Asciidoctor formatting, macros, stylesheets, etc., you may want to edit vkspec.adoc to just include your test code. The asciidoctor HTML build is very fast, even for the whole Specification, but PDF builds take several minutes.

Images Used In The Specification

All images used in the specification are in the images/ directory in the SVG format, and were created with Inkscape. We recommend using Inkscape to modify or create new images, due to problems using SVG files created by some other tools; especially in the PDF builds.

Validation Scripts

The allchecks Makefile target runs a Python script that looks for markup errors, missing interfaces, macro misuse, and inconsistencies in the specification text. This script is necessarily heuristic, since it is dealing with lots of hand-written material, but it identifies many problems and can suggest solutions. This script is also run as part of the CI tests in the internal Khronos gitlab repository.

Our Asciidoctor Macros

We use many custom Ruby macros in the reference pages and API spec Asciidoctor sources. The validator scripts rely on these macros as part of their checks. and you should use the macros whenever referring to an API command, struct, token, or enum name, so the documents are semantically tagged and more easily verifiable.

The supported macros are defined in the config/spec-macros/extension.rb asciidoctor extension script.

The tags used are described in the style guide (generated from styleguide.adoc).

We (may) eventually tool up the spec and reference pages to the point that anywhere there is a type or token referred to, clicking on (or perhaps hovering over) it in the HTML view will take reader to the definition of that type/token. That will take some more plumbing work to tag the stuff in the autogenerated include files, and do something sensible in the spec (e.g. resolve links to internal references).

Most of these macros deeply need more intuitive names.

Reference Pages

The reference pages are extracted from the API Specification source, which has been tagged to help identify boundaries of language talking about different commands, structures, enumerants, and other types. A set of Python scripts extract and lightly massage the relevant tagged language into corresponding reference page sources.

To regenerate the reference page sources from scratch yourself, execute:

./makeSpec -spec all refpages

The genRef.py script will generate many warnings, but most are just reminders that some pages are automatically generated. If everything is working correctly, all the $(GENERATED)/refpage/*.adoc files will be regenerated, but their contents will not change.

If you add new API features to the Specification in a branch, make sure that the commands have the required tagging and that reference pages are generated for them, and build properly.

When executing the manhtmlpages target in the Makefile, after building HTML versions of all reference pages extracted from the spec, symbolic links from aliases to the reference page for the API they alias will also be created.

Our Stylesheets

We use an HTML stylesheet config/khronos.css derived from the Asciidoctor stylesheet factory “colony” theme, with the default Arial font family replaced by the sans-serif Noto font family.

Vulkan Style Guide

If you are writing new spec language or modifying existing language, see the “style guide” (formally titled “Vulkan Documentation and Extensions: Procedures and Conventions”) document for details of our asciidoctor macros, extensions, mathematical equation markup, writing style, etc.

Software Dependencies

This section describes the software components used by the Vulkan spec toolchain.

In the past, we previously specified package versions and instructions for installing the toolchain in multiple desktop environments including Linux, MacOS X, and Microsoft Windows. The underlying components evolve rapidly, and we have not kept those instructions up to date.

Khronos-Provided Docker Image

Khronos has published a Docker image containing a Debian Linux distribution with the entire toolchain preinstalled.

We will occasionally update this image if needed, and we recommend people needing to build from this repository use the Docker image.

Docker installation is beyond the scope of this document. Refer to the Docker website for information about installing Docker on Linux, Windows, and MacOS X.

The build image is named khronosgroup/docker-images:asciidoctor-spec. However, due to local and CI caching problems when this image is updated on dockerhub, we use the SHA256 of the latest image update, rather than the image name. The SHA256 can be determined from

$ git grep -h sha256: .gitlab-ci.yml

which will print a line like

image: khronosgroup/docker-images@sha256:42123ba13792c4e809d037b69152c2230ad97fbf43b677338075ab9c928ab6ed

Everything following image: ` is the <imagename> to use. The first time you try to run Docker with this <imagename>, as is done by the `runDocker script described above under Building the Spec, the image will be pulled from Dockerhub and cached locally on your machine.

This image is used to build Specification output documents or other Makefile targets.

ℹ️
Note

When we update the image on Dockerhub, it is to add new components or update versions of components used in the specification toolchain. To save space, you may want to periodically purge old images using docker images and docker rmi -f.

Non-Docker Build Environments

We do not actively support building outside of our Docker image, but it is straightforward to reproduce our toolchain in a Debian (or similar APT-based Linux) distribution by executing the same steps as the Dockerfile used to build our Docker image.

It should be possible to apply the same steps in a Windows Subsystem for Linux (WSL2) environment on Windows 10, as well.

For other native environments, such as MacOS X and older Unix-like environments for Windows such as MinGW and Cygwin, we provided instructions in older versions of this document. While those instructions are out of date and have been removed from current versions of this document, you may be able to make use of the version of BUILD.adoc in the v1.2.135 repository tag

ℹ️
Note

While you do not have to use our Docker image, we cannot support every possible build environment. The Docker image is a straightforward way to build the specification in most modern desktop environments, without needing to install and update the spec toolchain yourself.

Revision History

  • 2022-10-11 - Update descriptions of using the Docker image to use the runDocker script and the same SHA256 of the latest image as used by CI.

  • 2021-03-12 - Use the new Docker image.

  • 2020-07-15 - Update to use makeSpec instead of makeAllExts.

  • 2020-03-23 - Document Khronos' published Docker image for building the spec, and remove all platform-specific instructions.

  • 2018-12-04 - Update Rbenv and ruby gem installation instructions and package dependencies for Linux and Ubuntu/Windows 10.

  • 2018-10-25 - Update Troubleshooting, and Windows and Linux build. Plus random editing.

  • 2018-03-13 - Rename to BUILD.adoc and update for new directory structure.

  • 2018-03-05 - Update README for Vulkan 1.1 release.

  • 2017-03-20 - Add description of prawn versioning problem and how to fix it.

  • 2017-03-06 - Add description of ruby-enum versioning problem and how to fix it.

  • 2017-02-13 - Move some comments here from ../../../README.md. Tweak asciidoctor markup to more clearly delineate shell command blocks.

  • 2017-02-10 - Add more Ruby installation guidelines and reflow the document in accordance with the style guide.

  • 2017-01-31 - Add rbenv instructions and update the README elsewhere.

  • 2017-01-16 - Modified dependencies for Asciidoctor

  • 2017-01-06 - Replace MathJax with KaTeX.

  • 2016-08-25 - Update for the single-branch model.

  • 2016-07-10 - Update for current state of spec and ref page generation.

  • 2015-11-11 - Add new can: etc. macros and DBLATEXPREFIX variable.

  • 2015-09-21 - Convert document to asciidoc and rename to README.md in the hope the gitlab browser will render it in some fashion.

  • 2015-09-21 - Add descriptions of LaTeX and MathJax math support for all output formats.

  • 2015-09-02 - Added Cygwin package info.

  • 2015-09-02 - Initial version documenting macros, required toolchain components and versions, etc.