Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support other than glob patterns with `Run Keyword And Expect Error` #2967

Closed
pekkaklarck opened this issue Oct 1, 2018 · 3 comments

Comments

@pekkaklarck
Copy link
Member

commented Oct 1, 2018

Currently Run Keyword And Expect Error considers the given expected message to be a glob pattern. This is typically very handy allowing usages like

Run Keyword And Expect Error    ValuError: *    My keyword    args    here

but causes problems if the expected error contains characters that are considered to be wildcards in glob patterns. Earlier this meant only * and ?, but #2471 added support for [...] making also [ and ] special characters and causing problems with usages like this:

Run Keyword And Expect Error    No match for '//input[@type="text"]'    Keyword    args

A simple workaround for the above problem is replacing problematic characters with ? that matches any single character:

Run Keyword And Expect Error    No match for '//input?@type="text"?'    Keyword    args

The main problem with this solution is that the specified message looks strange, but the wildcard could also match some wrong character making the test false positive.

A better solution would be making it possible to configure how Run Keyword And Expect Error should interpreter the provided message. We cannot add any new arguments to this keyword, but we can prefix the provided message with the message type. For example, something like this could be used to specify that the message should be interpreted as a literal value that must match the actual error as-is:

Run Keyword And Expect Error    EQUALS:No match for '//input[@type="text"]'    Keyword    args

Implementation of this functionality shouldn't be too complicated (thus the good first issue label), but we need to do some design decisions first:

  • What matching styles should we support? Robot's own acceptance tests have similar functionality, and there we support exact match, glob patterns, regexp patterns, and validating that the actual message starts with the given message. I think these would all be useful also with Run Keyword And Expect Error.

  • How to specify the matching style? The aforementioned system in Robot's acceptance tests uses style like REGEXP: <msg> and STARTS: <msg> which is pretty good. Then we have Catenate that supports SEPARATOR=<sep> and SeleniumLibrary uses locator:value nowadays but still supports legacy locator=value. The equal sign has also other usages in Robot's data, so probably the colon works better. That leaves us to decided should we use REGEXP: or regexp: and how to spell each of the options.

@aaltat

This comment has been minimized.

Copy link
Contributor

commented Oct 1, 2018

I prefer the style used in status checker, which mean uppercase and colon. Example REGEXP: is fine by me.

@pekkaklarck

This comment has been minimized.

Copy link
Member Author

commented Oct 24, 2018

I want to get this into RF 3.1 beta 1 to provide a simple workaround for backwards incompatibility problems caused by #2471. Already got the implementation done but still need to finish documentation.

Decided to go with these prefixes GLOB (also default), STARTS, REGEXP and EQUAL. Three first are same that status checker mentioned by @aaltat use. Status checker uses equal matching by default and doesn't have a separate prefix for that. I decided to use EQUAL but haven't really made up mind and may change that to EQUALS. EQUAL matches Should Be Equal keyword and, less importantly, our internal assert_equal utility, but EQUALS would be consistent with STARTS (which also doesn't match Should Start With keyword).

It's generally not a big deal do we in the end use EQUAL or EQUALS, but the current implementation uses the same error message regardless the prefix and thus it can be hard to detect the error if accidentally using the wrong one. Could easily support both EQUAL and EQUALS but perhaps it's just better to choose one and enhance error reporting.

@pekkaklarck

This comment has been minimized.

Copy link
Member Author

commented Oct 24, 2018

After thinking this a bit more, decided to go with EQUALS, not EQUAL. It's consistent with STARTS and just feels better in general. Also decided not to spend time enhancing error reporting at least now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.