Validation of fields in Admin does not support non A-Z characters

[jacobwod]

I'm extracting this comment into an issue of its own:

---

Just another small fix needed ( I have got hardcoded the value of these in Cyrillic and it works-I have setup my GeoServer Cyrillic aliases for the real field names):

  "searchFields": [
    "улица"
  ],

  "infobox": "",
  "aliasDict": "",
  "displayFields": [
    "улица"
  ],

  "shortDisplayFields": [
    "улица"
  ]

Otherwise, it is impossible to save Cyrillic values for the field names in Admin Panel:

This is a screenshot of the red colored inputs.

Originally posted by @bitmapbulgaria in #1181 (comment)

[jacobwod]

The issue lies here:

Hajk/new-admin/src/views/search.jsx

Line 366 in ed6c87e

valid = value.every((val) => /^\w+$/.test(val));

As per RegEx docs:

\w
matches any word character (equivalent to [a-zA-Z0-9_])

There are two solutions now:
a. either remove the check entirely or
b. go for something like [\wа-я]+.

The second option limits the check to latin and cyrillic only though and is not a good fix in my opinion. So I'll just remove the check, for now.

@bitmapbulgaria: Are you aware of a recommended way to check for word characters that would accept all non-latin alphabets too?

[bitmapbulgaria]

After some testing, I suggest this regex will do the work, needed for this job:

/^[\p{L}]+[-\p{L}\p{N}-]+(\s+[-\p{L}\p{N}_-]+)*$/ug

I have made some tests, you can check them on the screenshot:

My idea is not to allow some special cases in the field names (as on the screenshot - non colored lines), but allow different languages, numbers, dashes, underscore and whitespaces between the words.
Hope this will help!

[jacobwod]

Thanks, this is a really good solution and I'll go with that. It seems to work for most cases, the only noticable exception I found is Hebrew (I'm not sure why it would not match, as it \p{L} is supposed to match any letter from any language). I'm not sure how common it is to use Hebrew column names in database though, so let's consider it an edge case for now and deal the day it becomes a problem for someone.

[jacobwod]

On a second though: something like this would match Hebrew characters too. But I'm not sure if how much it does in this case, as we allow pretty much any character except a leading space.

^[\p{L}]+[\p{L}\p{N}\p{M}\p{P}\p{C}]+(\s+[-\p{L}\p{N}_-]+)*$

What do you guys think: should we allow pretty much anything except starting/ending whitespace and some punctation marks (such as .?+ and perhaps some more that aren't allowed as DB column names)? I'm not sure how much control is needed vs how much responsibility should be put to the system admin.

[bitmapbulgaria]

My regex proposal was with "I can fill some wrong here accidently and after that will try to figure out, what's wrong" in mind. Anyway, put ^ * & ( ) | \ symbols in PostgreSQL and Geoserver fields names is not possible (but I'm not sure about other DB/gisservers :) )
What about just adding Hebrew \u0590-\u05fe range?
For me, \p{L}\p{N} approach is working great, allow me to experiment with "I don't understand nothing, except Bulgarian" Hajk version.

[jacobwod]

I think we'll settle with
^[\p{L}\u0590-\u05fe]+[-\p{L}\p{N}\u0590-\u05fe-]+(\s+[-\p{L}\p{N}\u0590-\u05fe_-]+)*$
which I tested with a variety of alphabets (including Latin, Cyrillic, Arabic, Hebrew as well as various asian glyphs from East Asian countries). There are surely edge case but we'll extend to support them when detected.