Skip to content

Commit

Permalink
WHILE loop fixes.
Browse files Browse the repository at this point in the history 
- Fix limit in teardown. Fixes #4744.
- Fix continuable failures with `on_limit=pass`. #4562
- Nicer formatting of limit max time in error.
  • Loading branch information
@pekkaklarck
pekkaklarck committed Apr 25, 2023
1 parent 168ade4 commit c72f777
Show file tree
Hide file tree
Showing 5 changed files with 99 additions and 29 deletions.
2 changes: 1 addition & 1 deletion atest/robot/running/while/on_limit.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ Nested while on limit message
On limit message before limit
Check Test Case ${TESTNAME}

On limit messge with invalid variable
On limit message with invalid variable
Check Test Case ${TESTNAME}

Wrong WHILE arguments
Expand Down
9 changes: 9 additions & 0 deletions atest/robot/running/while/while_limit.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,15 @@ Limit can be disabled
No Condition With Limit
Check Test Case ${TESTNAME}

Limit exceeds in teardown
Check Test Case ${TESTNAME}

Limit exceeds after failures in teardown
Check Test Case ${TESTNAME}

Continue after limit in teardown
Check Test Case ${TESTNAME}

Invalid limit invalid suffix
Check Test Case ${TESTNAME}

Expand Down
39 changes: 26 additions & 13 deletions atest/testdata/running/while/on_limit.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ ${limit} 11
${number} ${0.2}
${pass} Pass
${errorMsg} Error Message
${USE LIMIT} Use the 'limit' argument to increase or remove the limit if needed.

*** Test Cases ***
On limit pass with time limit defined
Expand All @@ -18,7 +19,7 @@ On limit pass with iteration limit defined
END

On limit fail
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 5 iterations. Use the 'limit' argument to increase or remove the limit if needed.
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 5 iterations. ${USE LIMIT}
WHILE True limit=5 on_limit=FaIl
No Operation
END
Expand All @@ -30,21 +31,34 @@ On limit pass with failures in loop
END

On limit pass with continuable failure
[Documentation] FAIL Third failure, this time a hard one.
[Documentation] FAIL Several failures occurred:
...
... 1) Continuable failure!
...
... 2) Continuable failure!
...
... 3) One more failure!
[Tags] robot:continue-on-failure
WHILE limit=2 on_limit=pass
Run Keyword And Continue On Failure Fail Continuable failure!
Fail Continuable failure!
END
Fail Third failure, this time a hard one.
Fail One more failure!

On limit fail with continuable failure
[Documentation] FAIL Several failures occurred:\n\n
... 1) Continuable failure!\n\n
... 2) Continuable failure!\n\n
... 3) WHILE loop was aborted because it did not finish within the limit of 2 iterations. Use the 'limit' argument to increase or remove the limit if needed.
[Documentation] FAIL Several failures occurred:
...
... 1) Continuable failure!
...
... 2) Continuable failure!
...
... 3) WHILE loop was aborted because it did not finish within the limit of 2 iterations. ${USE LIMIT}
...
... 4) One more failure!
[Tags] robot:continue-on-failure
WHILE limit=2 on_limit=fail
Run Keyword And Continue On Failure Fail Continuable failure!
Fail Continuable failure!
END
Fail Should not be executed!
Fail One more failure!

Invalid on_limit
[Documentation] FAIL Invalid WHILE loop 'on_limit' value 'inValid': Value must be 'PASS' or 'FAIL'.
Expand All @@ -59,7 +73,7 @@ On limit without limit defined
END

On limit with invalid variable
[Documentation] FAIL Invalid WHILE loop 'on_limit' value '${does not exist}': Variable '${does not exist}' not found.
[Documentation] FAIL Invalid WHILE loop 'on_limit' value '\${does not exist}': Variable '\${does not exist}' not found.
WHILE True limit=5 on_limit=${does not exist}
Fail Oh no!
END
Expand Down Expand Up @@ -114,13 +128,12 @@ On limit message before limit
Log ${variable}
END

On limit messge with invalid variable
On limit message with invalid variable
[Documentation] FAIL Invalid WHILE loop 'on_limit_message': 'Variable '${nonExisting}' not found.
WHILE $variable < 2 on_limit_message=${nonExisting} limit=5
Log ${variable}
END


Wrong WHILE arguments
[Documentation] FAIL WHILE cannot have more than one condition, got '$variable < 2', 'limite=5' and 'limit_exceed_messag=Custom error message'.
WHILE $variable < 2 limite=5 limit_exceed_messag=Custom error message
Expand Down
62 changes: 54 additions & 8 deletions atest/testdata/running/while/while_limit.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,46 +2,47 @@
${variable} ${1}
${limit} 11
${number} ${0.2}
${USE LIMIT} Use the 'limit' argument to increase or remove the limit if needed.

*** Test Cases ***
Default limit is 10000 iterations
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 10000 iterations. Use the 'limit' argument to increase or remove the limit if needed.
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 10000 iterations. ${USE LIMIT}
WHILE $variable < 2
Log ${variable}
END

Limit with iteration count
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 5 iterations. Use the 'limit' argument to increase or remove the limit if needed.
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 5 iterations. ${USE LIMIT}
WHILE $variable < 2 limit=5
Log ${variable}
END

Limit with iteration count with spaces
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 30 iterations. Use the 'limit' argument to increase or remove the limit if needed.
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 30 iterations. ${USE LIMIT}
WHILE $variable < 2 limit=3 0
Log ${variable}
END

Limit with iteration count with underscore
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 10 iterations. Use the 'limit' argument to increase or remove the limit if needed.
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 10 iterations. ${USE LIMIT}
WHILE $variable < 2 limit=1_0
Log ${variable}
END

Limit as timestr
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 0.1 seconds. Use the 'limit' argument to increase or remove the limit if needed.
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 100 milliseconds. ${USE LIMIT}
WHILE $variable < 2 limit=0.1s
Log ${variable}
END

Limit from variable
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 11 iterations. Use the 'limit' argument to increase or remove the limit if needed.
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 11 iterations. ${USE LIMIT}
WHILE $variable < 2 limit=${limit}
Log ${variable}
END

Part of limit from variable
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 0.2 seconds. Use the 'limit' argument to increase or remove the limit if needed.
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 200 milliseconds. ${USE LIMIT}
WHILE $variable < 2 limit=${number} s
Log ${variable}
END
Expand All @@ -53,11 +54,39 @@ Limit can be disabled
END

No condition with limit
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 2 iterations. Use the 'limit' argument to increase or remove the limit if needed.
[Documentation] FAIL WHILE loop was aborted because it did not finish within the limit of 2 iterations. ${USE LIMIT}
WHILE limit=2
Log Hello
END

Limit exceeds in teardown
[Documentation] FAIL
... Teardown failed:
... Several failures occurred:
...
... 1) WHILE loop was aborted because it did not finish within the limit of 42 milliseconds. ${USE LIMIT}
...
... 2) Failing after WHILE
No Operation
[Teardown] Limit exceeds

Limit exceeds after failures in teardown
[Documentation] FAIL
... Teardown failed:
... Several failures occurred:
...
... 1) Hello!
...
... 2) Hello!
...
... 3) WHILE loop was aborted because it did not finish within the limit of 2 iterations. ${USE LIMIT}
No Operation
[Teardown] Limit exceeds after failures

Continue after limit in teardown
No Operation
[Teardown] Continue after limit

Invalid limit invalid suffix
[Documentation] FAIL Invalid WHILE loop limit: Invalid time string '1 times'.
WHILE $variable < 2 limit=1 times
Expand All @@ -81,3 +110,20 @@ Invalid values after limit
WHILE $variable < 2 limit=-1x invalid values
Log ${variable}
END

*** Keywords ***
Limit exceeds
WHILE limit=0.042s
Log Hello!
END
Fail Failing after WHILE

Limit exceeds after failures
WHILE limit=2
Fail Hello!
END

Continue after limit
WHILE limit=0.042s on_limit=pass
Log Hello!
END
16 changes: 9 additions & 7 deletions src/robot/running/bodyrunner.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@
TryBranch as TryBranchResult)
from robot.output import librarylogger as logger
from robot.utils import (cut_assign_value, frange, get_error_message, get_timestamp,
is_list_like, is_number, plural_or_not as s, seq2str,
split_from_equals, type_name, Matcher, timestr_to_secs)
is_list_like, is_number, plural_or_not as s, secs_to_timestr,
seq2str, split_from_equals, type_name, Matcher, timestr_to_secs)
from robot.variables import is_dict_variable, evaluate_expression

from .statusreporter import StatusReporter
Expand Down Expand Up @@ -410,11 +410,13 @@ def run(self, data):
except ExecutionPassed as passed:
passed.set_earlier_failures(errors)
raise passed
except LimitExceeded as exceeded:
if exceeded.on_limit_pass:
self._context.info(exceeded.message)
else:
errors.append(exceeded)
break
except ExecutionFailed as failed:
if isinstance(failed, LimitExceeded):
if failed.on_limit_pass:
self._context.info(failed.message)
return
errors.extend(failed.get_errors())
if not failed.can_continue(ctx, self._templated):
break
Expand Down Expand Up @@ -715,7 +717,7 @@ def __enter__(self):
self.limit_exceeded()

def __str__(self):
return f'{self.max_time} seconds'
return secs_to_timestr(self.max_time)


class IterationCountLimit(WhileLimit):
Expand Down

0 comments on commit c72f777

Please sign in to comment.