Skip to content
Switch branches/tags
Go to file

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Build Status


This is work towards a rewrite of the script that generates our Unicode database on MoarVM.

How do I build this

Install all dependant modules, if you have zef: zef --depsonly install . Download the Unicode database files: perl6 UCD-download.p6

Then make sure ./Unicode-Grant which is a git submodule is also checked out git submodule update --init --recursive

Generate the C files with perl6 UCD-gen.p6

They will be placed in ./source

Running make will build bitfield and names and put them in ./build



Unicode Name hash for generating the name table


Stores the binary property names


Stores enum prop names and also the property codes which are just internal numbers to represent it in the C datastructure


Stores the decomposition data for decomposition


Stores PropertyValueAliases from PropertyValueAliases.txt Used to go from short property value names to full names. Is a hash where the keys are full length property names, and the level below that has keys of property value alias names and the values are full property value names.


Bidi_Class => ${:AL("Arabic_Letter"), :AN("Arabic_Number"),
    :Arabic_Letter("Arabic_Letter"), :Arabic_Number("Arabic_Number"),
    :B("Paragraph_Separator"), :BN("Boundary_Neutral") }

%PropertyNameAliases %PropertyNameAliases_to

Stores Property Aliases or Property Value Aliases to their Full Name mappings

sub make-property-switches

sub make-property-switches(
) returns Mu

Makes the C functions which go from a property code into a value get_prop_int returns an prop's raw int, get_prop_str returns a char * from an enum. get_prop_enum gets an integer value from an enum


sub slurp-snippets

sub slurp-snippets(
    Str $name, 
    Str $subname?, 
) returns Mu

Slurps files from the snippets folder and concatenates them together The first argument is the folder name inside /snippets that they are in The second argument make it only concat files which contain that string The third argument allows you to request only snippets starting with those numbers if the numbers are positive. If they are negative, it returns all snippets except those numbers. Takes a single number, or a List of numbers


sub get-points-ranges-array

sub get-points-ranges-array(
    Array $sorted-points?
) returns Mu

Returns a multi-dim Array. We push onto each index all of the contiguous codepoints which have the same bitfield row. We also fill in any gaps and add those to their own range number.


sub GetPropertyValueLookupHash

sub GetPropertyValueLookupHash(
    Str:D $filename = "UNIDATA/PropertyValueAliases.txt", 
    Bool:D :$use-short-pnames = Bool::True, 
    Bool:D :$missing = Bool::False
) returns Mu

Returns a hash whose keys are PropertyValues and whose values are list's of list's. AHex => [[N No F False] [Y Yes T True]], The first value in the array is the shortened property value, and the second one is the long form one. The ones after that point are additional aliases Note. Does not return a positional for integer properties, instead returns a string which contains the Unicode datafile reason/null value for that property


sub GetPropertyAliasesRevLookupHash

sub GetPropertyAliasesRevLookupHash(
    Str $filename = "UNIDATA/PropertyAliases.txt"
) returns Mu

Returns a hash whose keys are PropertyAliases and whose values are the short name which is usable with GetPropertyValueLookupHash to look up different value aliases

sub GetPropertyAliasesLookupHash

sub GetPropertyAliasesLookupHash(
    Str $filename = "UNIDATA/PropertyAliases.txt"
) returns Mu

Returns a hash whose keys are PropertyAliases and whose values are the full names

sub GetPropertyAliases

sub GetPropertyAliases(
    Str $filename = "UNIDATA/PropertyAliases.txt"
) returns Mu

Returns a hash whose values arrays of all the property names which are equivalent


sub compose-array2

sub compose-array2(
    Str:D $type, 
    Str:D $name, 
    Bool :$header = Bool::False, 
    Str:D :$delim = ",", 
    Bool:D :$no-split = Bool::False, 
) returns Mu

:partition-note option seperates them by line and adds comments so you can tell what element number each one of the items is :map-empty-as allows you to make undefined array items to a certain value. for example match all undefined items to -1 or 0 for example


No description, website, or topics provided.



No releases published


No packages published