[ 2491802 ] Detection of digits incomplete

[HolQue]

In test-fullautomation/robotframework-testsuitesmanagement#323 it is discussed to use the regex pattern "^[\p{L}][\p{L}\p{Nd}_]*$" as naming convention. Because of this pattern supports Unicode letters and decimal digits.

But inside the JsonPreprocessor code I still see a lot of other patterns that are not Unicode safe, e.g.:

[0-9\s]*[A-Za-z_]+

What is the impact?

I use the JsonPreprocessor to get a list element using the character '൯' (MALAYALAM DIGIT NINE) as index:

"testlist" : [0,1,2,3,4,5,6,7,8,9],
"element"  : ${testlist}[൯]

Result:

Error: 'The parameter '${testlist}[൯]' is not available!'!

The same in pure Robot Framework:

@{testlist}    0
...            1
...            2
...            3
...            4
...            5
...            6
...            7
...            8
...            9

Log    ========== testlist: ${testlist}
Log    ========== element: ${testlist}[൯]

Result:

========== testlist: ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']
========== element: 9

This means: Robot Framework is able to handle Unicode decimal digits as index, JsonPreprocessor is not.

It seems to me that all positions in the JsonPreprocessor code where regular expresions like "0-9", "A-Z" and "a-z" are used, need to be reworked to enable full Unicode support.

[test-fullautomation]

I agree to check all regexps and adapt accoridingly that they handle all unicodes properly.

[namsonx]

Hello Holger,

Thank you for the hint! I updated regex patterns as your suggestion.

Regarding getting a list element using the character '൯', Python does not consider the '൯' is a digit. Python shows the error message:

SyntaxError: invalid character '൯' (U+0D6F)

I doni't think we should consider the '൯' as a digit in JsonPreprocessor.

Thank you,
Son