PyICU project repository
C++ Python C
Switch branches/tags
Nothing to show
Clone or download
Permalink
Failed to load latest commit information.
icu - removed unused docs.py Jan 5, 2018
samples version 1.0 Apr 16, 2015
test add ICU_VERSION check Jan 15, 2018
.gitignore Continuous integration with Travis CI Apr 25, 2017
.travis.yml Continuous integration with Travis CI Apr 25, 2017
CHANGES added wrapper for ULineBreak enum Aug 5, 2018
CREDITS refreshed credit list Apr 16, 2015
LICENSE version 0.8 Apr 16, 2015
MANIFEST.in add missing casemap.h t MANIFEST.in Aug 4, 2018
PyICU.py remove spurious quote from the deprecation warning Jan 11, 2016
README.md - added wrappers for MeasureFormat.formatMeasure and formatMeasurePe… Jan 21, 2018
_icu.cpp - added support for CaseMap (without Edits) Jul 2, 2018
bases.cpp - added wrapper for UMemory and made it UObject's base Jan 4, 2018
bases.h - added support for CaseMap (without Edits) Jul 2, 2018
calendar.cpp added another #under daylight Mar 21, 2017
calendar.h - renamed all Type-suffixed variable to Type_ Apr 29, 2015
casemap.cpp reworked CaseMap buffer preflighting Aug 1, 2018
casemap.h - added support for CaseMap (without Edits) Jul 2, 2018
char.cpp added wrapper for ULineBreak enum Aug 5, 2018
char.h first pass at uchar.h funtions and enums Apr 28, 2015
charset.cpp - turned icu into a package and moved docs.py into it Dec 19, 2016
charset.h version 0.9 Apr 16, 2015
collator.cpp - turned icu into a package and moved docs.py into it Dec 19, 2016
collator.h - renamed all Type-suffixed variable to Type_ Apr 29, 2015
common.cpp fixed build error with pypy3 Jun 29, 2018
common.h nil Aug 1, 2018
dateformat.cpp added ICU 54 conditional blocks for usage of UDateRelativeDateTimeFor… Apr 10, 2017
dateformat.h version 0.9 Apr 16, 2015
errors.cpp fix typos Jan 10, 2016
errors.h version 0.9 Apr 16, 2015
format.cpp - replaced occurrences of nullptr with NULL, -std=c++11 req'd w/only… Jan 27, 2018
format.h - renamed all Type-suffixed variable to Type_ Apr 29, 2015
idna.cpp - turned icu into a package and moved docs.py into it Dec 19, 2016
idna.h - renamed all Type-suffixed variable to Type_ Apr 29, 2015
iterators.cpp - fixed bug in StringCharacterIterator's dealloc freeing random memory Nov 4, 2016
iterators.h - renamed all Type-suffixed variable to Type_ Apr 29, 2015
layoutengine.cpp - added support for ICU 58.1 (conditionally removed layout engine wr… Nov 4, 2016
layoutengine.h version 1.3 Apr 16, 2015
locale.cpp - added wrappers for Region class and URegionType enum Jan 16, 2018
locale.h - renamed all Type-suffixed variable to Type_ Apr 29, 2015
macros.h - added support for CaseMap (without Edits) Jul 2, 2018
measureunit.cpp - replaced occurrences of nullptr with NULL, -std=c++11 req'd w/only… Jan 27, 2018
measureunit.h - added wrappers for MeasureFormat.formatMeasure and formatMeasurePe… Jan 21, 2018
normalizer.cpp - added wrappers for static Normalizer2 constructors that appeared i… Apr 4, 2017
normalizer.h version 1.0 Apr 16, 2015
numberformat.cpp - removed unused docs.py Jan 5, 2018
numberformat.h - renamed all Type-suffixed variable to Type_ Apr 29, 2015
regex.cpp - renamed all Type-suffixed variable to Type_ Apr 29, 2015
regex.h version 1.0 Apr 16, 2015
script.cpp - added support for CaseMap (without Edits) Jul 2, 2018
script.h version 1.6 Apr 16, 2015
search.cpp - renamed all Type-suffixed variable to Type_ Apr 29, 2015
search.h version 1.0 Apr 16, 2015
setup.py - fixed another build error with ICU < 62 Aug 5, 2018
shape.cpp - added support for CaseMap (without Edits) Jul 2, 2018
shape.h - added wrapper for Shape defines and shapeArabic (from ushape.h) Apr 28, 2015
spoof.cpp nil Aug 1, 2018
spoof.h added missing break statements May 9, 2015
transliterator.cpp - turned icu into a package and moved docs.py into it Dec 19, 2016
transliterator.h version 1.1 Apr 16, 2015
tzinfo.cpp - #define PYPY_VERSION when PYPY_VERSION_NUM is defined May 21, 2016
tzinfo.h version 1.4 Apr 16, 2015
unicodeset.cpp - preparing release 1.9 May 9, 2015
unicodeset.h - renamed all Type-suffixed variable to Type_ Apr 29, 2015

README.md

README file for PyICU

Welcome

Welcome to PyICU, a Python extension wrapping the ICU C++ libraries.

ICU stands for "International Components for Unicode". These are the i18n libraries of the Unicode Consortium. They implement much of the Unicode Standard, many of its companion Unicode Technical Standards, and much of Unicode CLDR.

The PyICU source code is hosted on GitHub at https://github.com/ovalhub/pyicu.

The ICU homepage is http://site.icu-project.org/

See also the CLDR homepage at http://cldr.unicode.org/

Building PyICU

Before building PyICU the ICU libraries must be built and installed. Refer to each system's instructions for more information.

PyICU is built with distutils or setuptools:

  • verify that the icu-config program is available or that the INCLUDES, LFLAGS, CFLAGS and LIBRARIES dictionaries in setup.py contain correct values for your platform. Starting with ICU 60, -std=c++11 must appear in your CFLAGS.
  • python setup.py build
  • sudo python setup.py install

Running PyICU

  • Mac OS X Make sure that DYLD_LIBRARY_PATH contains paths to the directory(ies) containing the ICU libs.

  • Linux & Solaris Make sure that LD_LIBRARY_PATH contains paths to the directory(ies) containing the ICU libs or that you added the corresponding -rpath argument to LFLAGS.

  • Windows Make sure that PATH contains paths to the directory(ies) containing the ICU DLLs.

What's available

See the CHANGES file for an up to date log of changes and additions.

API Documentation

There is no API documentation for PyICU. The API for ICU is documented at http://icu-project.org/apiref/icu4c/ and the following patterns can be used to translate from the C++ APIs to the corresponding Python APIs.

strings

The ICU string type, UnicodeString, is a type pointing at a mutable array of UChar Unicode 16-bit wide characters. The Python unicode type is an immutable string of 16-bit or 32-bit wide Unicode characters.

Because of these differences, UnicodeString and Python's unicode type are not merged into the same type when crossing the C++ boundary. ICU APIs taking UnicodeString arguments have been overloaded to also accept Python str or unicode type arguments. In the case of str objects, the utf-8 encoding is assumed when converting them to UnicodeString objects.

To convert a Python str encoded in an encoding other than utf-8 to an ICU UnicodeString use the UnicodeString(str, encodingName) constructor.

ICU's C++ APIs accept and return UnicodeString arguments in several ways: by value, by pointer or by reference. When an ICU C++ API is documented to accept a UnicodeString reference parameter, it is safe to assume that there are several corresponding PyICU python APIs making it accessible in simpler ways:

For example, the 'UnicodeString &Locale::getDisplayName(UnicodeString &)' API, documented at http://icu-project.org/apiref/icu4c/classLocale.html can be invoked from Python in several ways:

  1. The ICU way

     >>> from icu import UnicodeString, Locale
     >>> locale = Locale('pt_BR')
     >>> string = UnicodeString()
     >>> name = locale.getDisplayName(string)
     >>> name
     <UnicodeString: Portuguese (Brazil)>
     >>> name is string
     True                  <-- string arg was returned, modified in place
    
  2. The Python way

     >>> from icu import Locale
     >>> locale = Locale('pt_BR')
     >>> name = locale.getDisplayName()
     >>> name
     u'Portuguese (Brazil)'
    

    A UnicodeString object was allocated and converted to a Python unicode object.

A UnicodeString can be coerced to a Python unicode string with Python's unicode() constructor. The usual len(), str(), comparison, [] and [:] operators are all available, with the additional twists that slicing is not read-only and that += is also available since a UnicodeString is mutable. For example:

>>> name = locale.getDisplayName()
u'Portuguese (Brazil)'
>>> name = UnicodeString(name)
>>> name
<UnicodeString: Portuguese (Brazil)>
>>> unicode(name)
u'Portuguese (Brazil)'
>>> len(name)
19
>>> str(name)           <-- works when chars fit with default encoding
'Portuguese (Brazil)'
>>> name[3]
u't'
>>> name[12:18]
<UnicodeString: Brazil>
>>> name[12:18] = 'the country of Brasil'
>>> name
<UnicodeString: Portuguese (the country of Brasil)>
>>> name += ' oh joy'
>>> name
<UnicodeString: Portuguese (the country of Brasil) oh joy>

error reporting

The C++ ICU library does not use C++ exceptions to report errors. ICU C++ APIs return errors via a UErrorCode reference argument. All such APIs are wrapped by Python APIs that omit this argument and throw an ICUError Python exception instead. The same is true for ICU APIs taking both a ParseError and a UErrorCode, they are both to be omitted.

For example, the 'UnicodeString &DateFormat::format(const Formattable &, UnicodeString &, UErrorCode &)' API, documented at http://icu-project.org/apiref/icu4c/classDateFormat.html is invoked from Python with:

>>> from icu import DateFormat, Formattable
>>> df = DateFormat.createInstance()
>>> df
<SimpleDateFormat: M/d/yy h:mm a>
>>> f = Formattable(940284258.0, Formattable.kIsDate)
>>> df.format(f)
u'10/18/99 3:04 PM'

Of course, the simpler 'UnicodeString &DateFormat::format(UDate, UnicodeString &)' documented here: http://icu-project.org/apiref/icu4c/classDateFormat.html can be used too:

>>> from icu import DateFormat
>>> df = DateFormat.createInstance()
>>> df
<SimpleDateFormat: M/d/yy h:mm a>
>>> df.format(940284258.0)
u'10/18/99 3:04 PM'

dates

ICU uses a double floating point type called UDate that represents the number of milliseconds elapsed since 1970-jan-01 UTC for dates.

In Python, the value returned by the time module's time() function is the number of seconds since 1970-jan-01 UTC. Because of this difference, floating point values are multiplied by 1000 when passed to APIs taking UDate and divided by 1000 when returned as UDate.

Python's datetime objects, with or without timezone information, can also be used with APIs taking UDate arguments. The datetime objects get converted to UDate when crossing into the C++ layer.

arrays

Many ICU API take array arguments. A list of elements of the array element types is to be passed from Python.

StringEnumeration

An ICU StringEnumeration has three next methods: next() which returns a str objects, unext() which returns unicode objects and snext() which returns UnicodeString objects. Any of these methods can be used as an iterator, using the Python built-in iter function.

For example, let e be a StringEnumeration instance::

[s for s in e] is a list of 'str' objects
[s for s in iter(e.unext, None)] is a list of 'unicode' objects
[s for s in iter(e.snext, None)] is a list of 'UnicodeString' objects

timezones

The ICU TimeZone type may be wrapped with an ICUtzinfo type for usage with Python's datetime type. For example::

tz = ICUtzinfo(TimeZone.createTimeZone('US/Mountain'))
datetime.now(tz)

or, even simpler::

tz = ICUtzinfo.getInstance('Pacific/Fiji')
datetime.now(tz)

To get the default time zone use::

defaultTZ = ICUtzinfo.getDefault()

To get the time zone's id, use the tzid attribute or coerce the time zone to a string::

ICUtzinfo.getInstance('Pacific/Fiji').tzid -> 'Pacific/Fiji'
str(ICUtzinfo.getInstance('Pacific/Fiji')) -> 'Pacific/Fiji'