Make DataError normal catchable error by default.

It can be turned into a syntax error with `syntax=True`. Makes IF and WHILE condition errors not being catchable. That fixes #4348. Also changes error if a keyword is used with wrong number of arguments to a normal error. That's consistent with keywords not being found already earlier causing catchable errors. There could be similar semantical changes already elsewhere.
robotframework · Sep 27, 2022 · ffa0d4e · ffa0d4e
1 parent 41053c9
commit ffa0d4e
Show file tree

Hide file tree

Showing 19 changed files with 223 additions and 66 deletions.
diff --git a/atest/robot/running/if/invalid_if.robot b/atest/robot/running/if/invalid_if.robot
@@ -64,6 +64,18 @@ Invalid IF inside FOR
 Multiple errors
     FAIL    NOT RUN    NOT RUN    NOT RUN    NOT RUN
 
+Invalid data causes syntax error
+    [Template]    NONE
+    Check Test Case    ${TEST NAME}
+
+Invalid condition causes normal error
+    [Template]    NONE
+    Check Test Case    ${TEST NAME}
+
+Non-existing variable in condition causes normal error
+    [Template]    NONE
+    Check Test Case    ${TEST NAME}
+
 *** Keywords ***
 Branch statuses should be
     [Arguments]    @{statuses}

diff --git a/atest/robot/running/test_template.robot b/atest/robot/running/test_template.robot
@@ -116,7 +116,7 @@ Templated test with for loop continues after keyword timeout
 Templated test ends after syntax errors
     Check Test Case    ${TESTNAME}
 
-Templated test continues after variable error
+Templated test continues after non-syntax errors
     Check Test Case    ${TESTNAME}
 
 Templates and fatal errors

diff --git a/atest/robot/running/while/invalid_while.robot b/atest/robot/running/while/invalid_while.robot
@@ -21,3 +21,12 @@ No body
 
 No END
     Check Test Case    ${TESTNAME}
+
+Invalid data causes syntax error
+    Check Test Case    ${TEST NAME}
+
+Invalid condition causes normal error
+    Check Test Case    ${TEST NAME}
+
+Non-existing variable in condition causes normal error
+    Check Test Case    ${TEST NAME}
diff --git a/atest/robot/standard_libraries/builtin/run_keyword_with_errors.robot b/atest/robot/standard_libraries/builtin/run_keyword_with_errors.robot
@@ -41,10 +41,10 @@ Ignore Error When Timeout Occurs
 Ignore Error When Timeout Occurs In UK
     Check Test Case    ${TEST NAME}
 
-Ignore Error When Syntax Error At Parsing Time
+Ignore Error Cannot Catch Syntax Errors
     Check Test Case    ${TEST NAME}
 
-Ignore Error When Syntax Error At Run Time
+Ignore Error Can Catch Non-Syntax Errors
     Check Test Case    ${TEST NAME}
 
 Ignore Error When Syntax Error In Setting Variables
@@ -119,10 +119,10 @@ Expect Error When Timeout Occurs
 Expect Error When Timeout Occurs In UK
     Check Test Case    ${TEST NAME}
 
-Expect Error When Syntax Error At Parsing Time
+Expect Error Cannot Catch Syntax Errors
     Check Test Case    ${TEST NAME}
 
-Expect Error When Syntax Error At Run Time
+Expect Error Can Catch Non-Syntax Errors
     Check Test Case    ${TEST NAME}
 
 Expect Error When Syntax Error In Setting Variables

diff --git a/atest/robot/standard_libraries/builtin/wait_until_keyword_succeeds.robot b/atest/robot/standard_libraries/builtin/wait_until_keyword_succeeds.robot
@@ -69,16 +69,19 @@ Retry count must be positive
     Check Test Case    ${TESTNAME} 1
     Check Test Case    ${TESTNAME} 2
 
-Invalid Number Of Arguments Inside Wait Until Keyword Succeeds
+No retry after syntax error
     Check Test Case    ${TESTNAME}
 
-Invalid Keyword Inside Wait Until Keyword Succeeds
+No retry if keyword name is not string
     Check Test Case    ${TESTNAME}
 
-Keyword Not Found Inside Wait Until Keyword Succeeds
+Retry if keyword is not found
     Check Test Case    ${TESTNAME}
 
-Fail With Nonexisting Variable Inside Wait Until Keyword Succeeds
+Retry if wrong number of arguments
+    Check Test Case    ${TESTNAME}
+
+Retry if variable is not found
     ${tc} =    Check Test Case    ${TESTNAME}
     Check Log Message    ${tc.kws[0].kws[0].kws[0].msgs[0]}    Variable '\${nonexisting}' not found.    FAIL
     Check Log Message    ${tc.kws[0].kws[1].kws[0].msgs[0]}    Variable '\${nonexisting}' not found.    FAIL

diff --git a/atest/testdata/running/if/invalid_if.robot b/atest/testdata/running/if/invalid_if.robot
@@ -167,3 +167,59 @@ Multiple errors
     ELSE    oops
     ELSE IF
     ELSE
+
+Invalid data causes syntax error
+    [Documentation]    FAIL    IF branch cannot be empty.
+    TRY
+        IF    True
+        END
+    EXCEPT
+        Fail    Syntax error cannot be caught
+    END
+
+Invalid condition causes normal error
+    [Documentation]    FAIL    Teardown failed:
+    ...    Several failures occurred:
+    ...
+    ...    1) Evaluating IF condition failed: Evaluating expression 'bad in teardown' failed: NameError: name 'bad' is not defined nor importable as module
+    ...
+    ...    2) Should be run in teardown
+    TRY
+        IF    bad
+            Fail    Should not be run
+        END
+    EXCEPT    Evaluating IF condition failed: Evaluating expression 'bad' failed: NameError: name 'bad' is not defined nor importable as module
+        No Operation
+    END
+    [Teardown]    Invalid condition
+
+Non-existing variable in condition causes normal error
+    [Documentation]    FAIL    Teardown failed:
+    ...    Several failures occurred:
+    ...
+    ...    1) Evaluating IF condition failed: Variable '\${bad}' not found.
+    ...
+    ...    2) Should be run in teardown
+    TRY
+        IF    ${bad}
+            Fail    Should not be run
+        END
+    EXCEPT    Evaluating IF condition failed: Variable '\${bad}' not found.
+        No Operation
+    END
+    [Teardown]    Non-existing variable in condition
+
+*** Keywords ***
+Invalid condition
+    IF    bad in teardown
+        Fail    Should not be run
+    ELSE
+        Fail    Sould not be run either
+    END
+    Fail    Should be run in teardown
+
+Non-existing variable in condition
+    IF    ${bad}
+        Fail    Should not be run
+    END
+    Fail    Should be run in teardown
diff --git a/atest/testdata/running/test_template.robot b/atest/testdata/running/test_template.robot
@@ -327,18 +327,22 @@ Templated test with for loop continues after keyword timeout
     END
 
 Templated test ends after syntax errors
-    [Documentation]    FAIL   Keyword 'BuiltIn.Should Be Equal' expected 2 to 8 arguments, got 9.
-    The    syntax    error    makes    any    test    end     again    here
-    Not compared    anymore
+    [Documentation]    FAIL   IF must have closing END.
+    [Template]    Syntax Error
+    fails here
+    not run
 
-Templated test continues after variable error
+Templated test continues after non-syntax errors
     [Documentation]    FAIL
     ...    Several failures occurred:
     ...
     ...    1) Variable '\${this does not exist}' not found.
     ...
-    ...    2) Compared and not equal != Fails
+    ...    2) Keyword 'BuiltIn.Should Be Equal' expected 2 to 8 arguments, got 1.
+    ...
+    ...    3) Compared and not equal != Fails
     ${this does not exist}    ${this does not exist either}
+    Too few args
     Compared and equal        Compared and equal
     Compared and not equal    Fails
 
@@ -393,3 +397,8 @@ Template with timeout
     [Timeout]    ${timeout}
     Sleep    ${sleep}
     Fail    Failing after ${sleep} sleep and before ${timeout} timeout.
+
+Syntax Error
+    [Arguments]    ${arg}
+    IF    ${arg}
+        Fail    Should not be run due to END missing.
diff --git a/atest/testdata/running/test_template_with_embeded_args.robot b/atest/testdata/running/test_template_with_embeded_args.robot
@@ -14,7 +14,12 @@ Argument names do not need to be same as in definition
     1 + 3    5
 
 Some arguments can be hard-coded
-    [Documentation]    FAIL Several failures occurred:\n\n1) 2 != 3\n\n2) 4 != 3
+    [Documentation]    FAIL
+    ...    Several failures occurred:
+    ...
+    ...    1) 2 != 3
+    ...
+    ...    2) 4 != 3
     [Template]    The result of ${expression} should be 3
     1 + 1
     1 + 2
@@ -31,12 +36,18 @@ Can use variables
     ${1} + ${2}    ${3}
 
 Cannot have more arguments than variables
-    [Documentation]    FAIL Keyword 'The result of ${calculation} should be ${expected}' expected 0 arguments, got 2.
+    [Documentation]    FAIL
+    ...    Keyword 'The result of \${calculation} should be \${expected}' expected 0 arguments, got 2.
     [Template]    The result of ${calc} should be 3
     1 + 2    extra
 
 Cannot have less arguments than variables
-    [Documentation]    FAIL Keyword 'The result of ${calculation} should be ${expected}' expected 0 arguments, got 1.
+    [Documentation]    FAIL
+    ...    Several failures occurred:
+    ...
+    ...    1) Keyword 'The result of \${calculation} should be \${expected}' expected 0 arguments, got 1.
+    ...
+    ...    2) Keyword 'The result of \${calculation} should be \${expected}' expected 0 arguments, got 1.
     [Template]    The result of ${calc} should be ${extra}
     1 + 2
     4 - 1

diff --git a/atest/testdata/running/while/invalid_while.robot b/atest/testdata/running/while/invalid_while.robot
@@ -12,13 +12,13 @@ Multiple conditions
     END
 
 Invalid condition
-    [Documentation]    FAIL STARTS: Evaluating WHILE loop condition failed: Evaluating expression 'ooops!' failed: SyntaxError:
+    [Documentation]    FAIL STARTS: Evaluating WHILE condition failed: Evaluating expression 'ooops!' failed: SyntaxError:
     WHILE    ooops!
         Fail    Not executed!
     END
 
 Non-existing variable in condition
-    [Documentation]    FAIL Evaluating WHILE loop condition failed: Variable '\${ooops}' not found.
+    [Documentation]    FAIL Evaluating WHILE condition failed: Variable '\${ooops}' not found.
     WHILE    ${ooops}
         Fail    Not executed!
     END
@@ -32,3 +32,30 @@ No END
     [Documentation]    FAIL WHILE loop must have closing END.
     WHILE    True
         Fail    Not executed!
+
+Invalid data causes syntax error
+    [Documentation]    FAIL WHILE loop cannot be empty.
+    TRY
+        WHILE    False
+        END
+    EXCEPT
+        Fail    Syntax error cannot be caught
+    END
+
+Invalid condition causes normal error
+    TRY
+        WHILE    bad
+            Fail    Should not be run
+        END
+    EXCEPT    Evaluating WHILE condition failed: Evaluating expression 'bad' failed: NameError: name 'bad' is not defined nor importable as module
+        No Operation
+    END
+
+Non-existing variable in condition causes normal error
+    TRY
+        WHILE    ${bad}
+            Fail    Should not be run
+        END
+    EXCEPT    Evaluating WHILE condition failed: Variable '\${bad}' not found.
+        No Operation
+    END
diff --git a/atest/testdata/standard_libraries/builtin/run_keyword_and_continue_on_failure.robot b/atest/testdata/standard_libraries/builtin/run_keyword_and_continue_on_failure.robot
@@ -57,8 +57,8 @@ Run Keyword And Continue On Failure with failure in keyoword teardown
     Fail    The End
 
 Run Keyword And Continue On Failure With Syntax Error
-    [Documentation]    FAIL    Keyword 'BuiltIn.No Operation' expected 0 arguments, got 1.
-    Run keyword And Continue On Failure    No Operation    illegal argument
+    [Documentation]    FAIL    Assign mark '=' can be used only with the last variable.
+    Run keyword And Continue On Failure    Syntax Error
     Fail    This Should Not Be Executed!
 
 Run Keyword And Continue On Failure With Timeout
@@ -107,3 +107,6 @@ RKACOF in UK 2
 Keyword With Failing Teardown
     No Operation
     [Teardown]    Fail    Expected error
+
+Syntax Error
+    ${x} =    ${y} =    Create List    x    y
diff --git a/atest/testdata/standard_libraries/builtin/run_keyword_and_warn_on_failure.robot b/atest/testdata/standard_libraries/builtin/run_keyword_and_warn_on_failure.robot
@@ -18,8 +18,8 @@ Run User keyword And Warn On Failure
     Log    This should be executed
 
 Run Keyword And Warn On Failure With Syntax Error
-    [Documentation]    FAIL    Keyword 'BuiltIn.No Operation' expected 0 arguments, got 1.
-    Run keyword And Warn On Failure    No Operation    illegal argument
+    [Documentation]    FAIL    Assign mark '=' can be used only with the last variable.
+    Run keyword And Continue On Failure    Syntax Error
     Fail    This Should Not Be Executed!
 
 Run Keyword And Warn On Failure With Failure On Test Teardown
@@ -40,3 +40,6 @@ Failing Keyword Teardown
 
 Exception In User Defined Keyword
     Fail    Expected Warn In User Keyword
+
+Syntax Error
+    ${x} =    ${y} =    Create List    x    y
diff --git a/atest/testdata/standard_libraries/builtin/run_keyword_with_errors.robot b/atest/testdata/standard_libraries/builtin/run_keyword_with_errors.robot
@@ -51,12 +51,11 @@ Ignore Error When Timeout Occurs In UK
     [Documentation]    FAIL Keyword timeout 100 milliseconds exceeded.
     Run Keyword And Ignore Error    Timeouting UK
 
-Ignore Error When Syntax Error At Parsing Time
+Ignore Error Cannot Catch Syntax Errors
     [Documentation]    FAIL Keyword name cannot be empty.
     Run Keyword And Ignore Error    Broken User Keyword
 
-Ignore Error When Syntax Error At Run Time
-    [Documentation]    FAIL Keyword 'BuiltIn.No Operation' expected 0 arguments, got 4.
+Ignore Error Can Catch Non-Syntax Errors
     Run Keyword And Ignore Error    No Operation    wrong    number    of    arguments
 
 Ignore Error When Syntax Error In Setting Variables
@@ -155,11 +154,11 @@ Expect Error When Timeout Occurs In UK
     [Documentation]    FAIL Keyword timeout 100 milliseconds exceeded.
     Run Keyword And Expect Error    *    Timeouting UK
 
-Expect Error When Syntax Error At Parsing Time
+Expect Error Cannot Catch Syntax Errors
     [Documentation]    FAIL Keyword name cannot be empty.
     Run Keyword And Expect Error    *    Broken User Keyword
 
-Expect Error When Syntax Error At Run Time
+Expect Error Can Catch Non-Syntax Errors
     Run Keyword And Expect Error
     ...    No keyword with name 'Non existing keyword' found.
     ...    Non existing keyword

diff --git a/atest/testdata/standard_libraries/builtin/wait_until_keyword_succeeds.robot b/atest/testdata/standard_libraries/builtin/wait_until_keyword_succeeds.robot
@@ -106,22 +106,28 @@ Retry count must be positive 2
     [Documentation]    FAIL ValueError: Retry count -8 is not positive.
     Wait Until Keyword Succeeds    -8x    1s    No Operation
 
-Invalid Number Of Arguments Inside Wait Until Keyword Succeeds
-    [Documentation]    FAIL Keyword 'BuiltIn.No Operation' expected 0 arguments, got 3.
-    Wait Until Keyword Succeeds    1 second    0.1s    No Operation    No    args    accepted
+No retry after syntax error
+    [Documentation]    FAIL FOR loop cannot be empty.
+    Wait Until Keyword Succeeds    10 second    1s    Syntax Error
 
-Invalid Keyword Inside Wait Until Keyword Succeeds
+No retry if keyword name is not string
     [Documentation]    FAIL Keyword name must be a string.
     ${list} =    Create List    1    2
     Wait Until Keyword Succeeds    1 second    0.1s    ${list}
 
-Keyword Not Found Inside Wait Until Keyword Succeeds
+Retry if keyword is not found
     [Documentation]    FAIL
     ...    Keyword 'Non Existing KW' failed after retrying for 300 milliseconds. \
     ...    The last error was: No keyword with name 'Non Existing KW' found.
     Wait Until Keyword Succeeds    0.3s    0.1s    Non Existing KW
 
-Fail With Nonexisting Variable Inside Wait Until Keyword Succeeds
+Retry if wrong number of arguments
+    [Documentation]    FAIL
+    ...    Keyword 'No Operation' failed after retrying for 50 milliseconds. \
+    ...    The last error was: Keyword 'BuiltIn.No Operation' expected 0 arguments, got 3.
+    Wait Until Keyword Succeeds    0.05 second    0.01s    No Operation    No    args    accepted
+
+Retry if variable is not found
     [Documentation]    FAIL
     ...    Keyword 'Access Nonexisting Variable' failed after retrying 3 times. \
     ...    The last error was: Variable '\${nonexisting}' not found.
@@ -166,3 +172,7 @@ Access Nonexisting Variable
 Access Initially Nonexisting Variable
     Log    ${created after accessing first time}
     [Teardown]    Set Test Variable    ${created after accessing first time}    created in keyword teardown
+
+Syntax Error
+    FOR    ${x}    IN    cannot    have    empty    body
+    END