Add word level element "prediction" for Page XML datasets

[maxnth]

Adds the flag --pagexml_word_level to predict.py which enables the generation of Word elements when predicting on Page XML datasets.

"Words" are generated from the positions returned by the prediction.
All character sequences separated by spaces are classified as a "word".
All (non-leading/non-trailing) spaces are also classified as "words" and saved as Word elements.
As far as I know the PAGE XML schema doesn't have any clear decisions on whether whitespaces should be declared in Word elements, so this is probably a very subjective decision.

[andbue]

Suggestion: if entry[0] in whitespace with from string import whitespace – you never now what's in the codec...
One might even consider giving the user the possibility to supply a list of word breaking characters on the command line.

[maxnth]

from string import whitespace was added in 050276e when the user selects whitespace as word boundary.
The default mode for --pagexml_word_boundary was set to unicode and uses the unicode word boundaries.

[mittagessen]

* andbue :: 2020-05-19 14:21 Tue:

Suggestion: `if entry[0] in whitespace` with `from string import whitespace` – you never now what's in the codec... One might even consider giving the user the possibility to supply a list of word breaking characters on the command line.

string.whitespace is only ASCII whitespace so it won't deal with all the Unicode whitespaces out there. There is a Unicode algorithm for word breaking [0] that is implemented in the regex package. It is far from perfect but better than breaking at whitespace as some of the esoteric ones are sometimes used for typographic purposes. [0] https://unicode.org/reports/tr29/#Word_Boundary_Rules