Skip to content

Proper Binary Format OUTDATED

Michal Babej edited this page Feb 22, 2018 · 1 revision

NOTE: this page is outdated

A binary format has been implemented, very similar to what is described in this document

Background/motivation

The current program binary format of pocl is simply an LLVM bitcode of the single kernel, thus there are lots of compilation steps performed on this format. This does not get the benefits of a good binary format that avoids invoking the compiler altogether.

The basic idea for a proper binary format for pocl

Extremely simple archive format that stores the cache entry for a program. Contains a sequence of [rel_pathname, data blob] pairs, nothing else.

The idea here is to allow quickly creating the whole directory structure of the kernel build to enable caching of the very final parts of the process as well as (optionally) storing the intermediate files for bug reporting purposes.

The file and pathnames follow a previously defined convention in order to find the files for each alternative WG sizes for each kernel in the program.

The format could be as simple as

'poclbin'
[uint32 format_version]
[base_pathname_zero_term]
[file1_relpathname_zero_term][uint32 size1][char[size1] binary_blob_of_the_file1]
[file2_relpathname_zero_term][uint32 size2][char[size2] binary_blob_of_the_file2]
...
[fileN_relpathname_zero_term][uint32 sizeN][char[sizeN] binary_blob_of_the_fileN]

Where 'poclbin' is the magic string identifier for this binary format (note that we should support the LLVM IR/SPIR input still, this is a new binary format alternative). format_id is an integer identifying the format version. We might want to add more features to this format later, this allows differentiating which version this is. The base_pathname_zero_term gives the relative directory (the cache hashstring directory usually). file1_replathname_zero_term is a relative path of the entry of the first file under the base dir. 'size1' gives the size of the contents of the given file, followed by the contents. This is repeated for all files under the directory.

Implementation

  • In the binary (size) query, create the archive in-memory of the cache contents.
  • In the clCreateProgramWithBinary, extract the binary to the cache directory and one is all set.
You can’t perform that action at this time.