LGR Core library
Core library and LGR data structure used to manipulate an LGR.
This version conforms to the specification RFC7940.
This toolset was implemented by Viagenie (Audric Schiltknecht, Julien Bernard, David Drouin, Guillaume Blanchet and Marc Blanchet) and Wil Tan on an ICANN contract.
Copyright (c) 2015-2020 Internet Corporation for Assigned Names and Numbers (“ICANN”). All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of the ICANN nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY ICANN AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL ICANN OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- Python 3.4
- LibXML2 [MIT License] used by the lxml Python bindings
- LXML for XML parsing/serializing [BSD License]
- language-tags [MIT License]
- pycountry [L-GPL License]
- picu [MIT/X license]
- munidata for Unicode-related properties access [BSD License]
For documentation generation:
Virtual Environment setup
If your distribution does not package the required dependencies, the easiest way to get a working environment in no-time is to use Python's virtual environments.
Create a python virtualenv:
$ virtualenv venv
Activate the environment:
$ source ./venv/bin/activate
(venv) $ pip install -r requirements.txt
tools directory contains some tools to exercise the library.
xml_dump.pyparses an XML LGR file, creates the LGR structure, and convert it back to XML.
make_idna_repertoire.pygenerates the IDNA2008 starting repertoire, by parsing the IDNA2008 table registry available on IANA website, and collecting all
lgr_cli.pyparsed an XML LGR file, and validate it against configured MSR and Unicode version.
rfcXXXX_dump.pyparses inputs in RFC XXXX format, and generate an XML LGR.
one_per_line_dump.pyparses input in the "one per line" format, and generate an XML LGR.
rfc7940_validate.pytakes an LGR and checks it for compliance with RFC 7940.
Other tools are available to manipulate LGR files and labels:
lgr_annotate.pytakes an LGR and a list of labels and output the list of labels with their respective disposition.
lgr_collision.pytakes an LGR and a list of labels and check for collision(s) between the labels/variants.
lgr_compare.pyis used to compare 2 LGR (output textual diff, merge, intersection).
lgr_diff_collision.pytakes 2 LGR and one set of labels, and test for collisions between labels and generated variants from the 2 LGR.
lgr_merge_set.pytakes some LGRs and create a merged LGR from the provided set.
lgr_check_harmonizedchecks that variants code points from a list of LGRs are symmetric and transitive in each LGR, and list missing variants from one LGR to another.
lgr_cross_script_variantstakes some LGRs and generate the list of cross-script variants.
lgr_idn_table_review.pyreviews IDN tables against reference LGRs.
Testing and coverage
Tests and coverage report can be run as follows:
(venv) $ pip install pytest pytest-cov (venv) $ ./runtests.sh
htmlcov/index.html in a web browser for the coverage report.
To generate the documentation, go to the
doc directory and run the following command:
(venv) $ make html
The generated documentation is available in