Support question mark in idPattern in subscriptions

[danielvillalbamota]

Is your feature request related to a problem / use case? Please describe.
Question mark ? is not allowed in subscription idPattern, e.g. ^(?!avoid).*, which means deny entityId which starts with avoid.

Following request gets a 400 error.

curl --location --request POST 'http://host:1026/v2/subscriptions' \
--header 'Fiware-Service: jcyl' \
--header 'Fiware-ServicePath: /servicepath' \
--header 'X-Auth-Token: ...' \
--header 'Content-Type: application/json' \
--data-raw '{
  "description": "A subscription to get info about myEntity",
  "subject": {
    "entities": [
      {
        "idPattern": "^(?!avoid).*",
        "type": "myType"
      }
    ],
    "conditions": {
    }
  },
  "notification": {
    "http": {
      "url": "https://x.pipedream.net/"
    },
    "attrs": [
      
    ]
  }
}'

Error message: 400 Bad Request

{
    "error": "BadRequest",
    "description": "Invalid regex for entity idPattern"
}

The regex without question mark works.

Describe the solution you'd like
Allow this character to register this kind of regex.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Describe why you need this feature

• To approach a new use case.
• To improve or simplify an scenario.

Additional information
None

Do you have the intention to implement the solution

• Yes, I have the knowledge to implement this new feature.
• Yes, but I will need help.
• No, I do not have the skills.

[fgalan]

That error message is generated at parsing stage at parseEntitiesVector() function (in parseEntitiesVector.cpp):

        if (regcomp(&re, idPattern.c_str(), REG_EXTENDED) != 0)
        {
          *errorStringP = ERROR_DESC_BAD_REQUEST_INVALID_REGEX_ENTIDPATTERN;
          return false;
        }

Thus, it is due to regcomp() function, which belongs to regex library

[fgalan]

Using regerror() we could probably get more information about the error.

Moreover, we should use regerror() in everyplace we are currently using regcomp()

[fgalan]

Moreover, we should use regerror() in everyplace we are currently using regcomp()

After implementing that in PR #4100 we get the following error in CB logs:

from=127.0.0.1 | srv=<none> | subsrv=<none> | comp=Orion | op=string.cpp[1130]:regComp | msg=regcomp() failed for pattern '^(?!avoid).*': Invalid preceding regular expression

The message "Invalid preceding regular expression" is generated by regex library.

[fgalan]

From https://stackoverflow.com/questions/71941687/regcomp-error-in-apparently-valid-regular-expression (it seems this link is no longer available in the public internet, as SOF people has closed the question as duplicate), in question comments:

POSIX regex does not support lookarounds. See the top of this post for the POSIX workaround for strings that do not start with some word.

Looking in that post:

Another interesting piece of feedback in the comments:

Q: Is regex.h the only regular expression library in C/C++? Maybe there are alternative libraries supporting lookarounds?
A: Sure, for C++, the most common workaround is using Boost boost::regex

But the cost of changing from regex.h to boost::regex needs to be evaluated.

[fgalan]

But the cost of changing from regex.h to boost::regex needs to be evaluated.

Some things to take into account:

• It seems that both cannot be used at the same time, according to https://bytes.com/topic/c/answers/831565-boost-regex-hpp-regex-h
• Maybe an useful performance comparison (although pretty old): https://www.boost.org/doc/libs/1_46_1/libs/regex/doc/gcc-performance.html

[fgalan]

Expression to test, provided by @danielvillalbamota

^(([^\nW].{14}|.[^\ne].{13}|.{2}[^\na].{12}|.{3}[^\nt].{11}|.{4}[^\nh].{10}|.{5}[^\ne].{9}|.{6}[^\nr].{8}|.{7}[^\nO].{7}|.{8}[^\nb].{6}|.{9}[^\ns].{5}|.{10}[^\ne].{4}|.{11}[^\nr].{3}|.{12}[^\ntv].{2}|.{13}[^\ne].|.{14}[^\nd]).*|.{0,14})$

[fgalan]

curl using that expression:

curl --location --request POST 'http://host:1026/v2/subscriptions' \
--header 'Fiware-Service: jcyl' \
--header 'Fiware-ServicePath: /servicepath' \
--header 'X-Auth-Token: ...' \
--header 'Content-Type: application/json' \
--data-raw '{
  "description": "A subscription to get info about myEntity",
  "subject": {
    "entities": [
      {
        "idPattern": "^(([^\nW].{14}|.[^\ne].{13}|.{2}[^\na].{12}|.{3}[^\nt].{11}|.{4}[^\nh].{10}|.{5}[^\ne].{9}|.{6}[^\nr].{8}|.{7}[^\nO].{7}|.{8}[^\nb].{6}|.{9}[^\ns].{5}|.{10}[^\ne].{4}|.{11}[^\nr].{3}|.{12}[^\ntv].{2}|.{13}[^\ne].|.{14}[^\nd]).*|.{0,14})$",
        "type": "myType"
      }
    ],
    "conditions": {
    }
  },
  "notification": {
    "http": {
      "url": "https://x.pipedream.net/"
    },
    "attrs": [
      
    ]
  }
}'