Derive Data Files

Resources

• Unicode Technical Standard #46: IDNA
  • unicode-logic.js/idna_rules() — src
• Unicode Technical Standard #51: Emoji
• Unicode Standard Annex #15: Normalization Forms
• Unicode Standard Annex #24: Script Property
• Unicode Standard Annex #29: Text Segmentation
• Unicode Standard Annex #31: Identifier and Pattern Syntax
  • scripts-excluded.json — src
  • scripts-recommended.json — src
  • scripts-limited-use.json — src
• Unicode Technical Standard #39: Security Mechanisms
  • unicode-logic.js/get_augmented_script_set() — src
• RFC-3492: Punycode
• RFC-5891: IDNA: Protocol
• RFC-5892: The Unicode Code Points and IDNA
• WHATWG URL: IDNA
• Unicode data files
  • Download Latest: node download.js
  • To download older versions: node download.js 12.1.0
  • Already included: Unicode 11-15
• CLDR data files
  • Download Latest: node parse-cldr.js
  • Already included: CLDR 42
  • Warning: these aren't versioned with Unicode!

Instructions

1. edit unicode-version.js — specify which versions to use
2. edit Rules Files
3. node make.js — creates /output/ with data files

Rules

• chars-valid.js
• chars-ignored.js
• chars-mapped.js
• chars-disallow.js
• chars-fenced.js — characters that occur in the middle and can't touch
• chars-escape.js — characters that should be escaped
• emoji.js — various emoji configurations
• cm.js — combining mark sequence whitelist
• scripts.js — various script configurations
• confuse.js — confusables groups
• group-order.js — how groups should be sorted for matching efficiency (auto-generated)

Helpers

• node names.js 61..7A 200D — print Unicode names for hex codepoints
• node names.js script Latn — print Unicode names for Latin
• node names.js prop White_Space — print Unicode names with property White_Space
• node names.js find abc — find characters by name