Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JENKINS-64516] Add support for Robot Framework 4.0 #40

Merged
merged 55 commits into from
Mar 12, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
86467ff
Add methods to get "skip" status
asimell Nov 9, 2020
076c467
Remove redundant list type references
asimell Nov 9, 2020
f1372c6
Add skip to RobotCaseResult
asimell Nov 9, 2020
e1513d4
Remove unnecessary variable
asimell Nov 9, 2020
4edd589
Remove redundant toString() call
asimell Nov 12, 2020
51a0e05
Merge branch 'master' into new-robot-parser
asimell Feb 17, 2021
e83f3aa
Apply better Java syntax
asimell Feb 17, 2021
3d19021
Apply better Java syntax
asimell Feb 17, 2021
9542f95
Merge branch 'new-robot-parser' of https://github.com/jenkinsci/robot…
asimell Feb 17, 2021
c65a2ca
Parse RF4.0 styled tags
asimell Feb 17, 2021
a0d5383
Add unit tests for skip status
asimell Feb 18, 2021
df90910
Parse "SKIP" status correctly
asimell Feb 18, 2021
4178af4
Ignore skipped tests when counting pass percentage
asimell Feb 18, 2021
88ce9bd
Add "Skipped" trend to graph and add a column to list view
asimell Feb 18, 2021
f7921ba
Add "skip" support for graphs, blue ocean, and list view
asimell Feb 18, 2021
a77980c
Add skip info to results overview
asimell Feb 19, 2021
4c830d5
Replace output.xml with more complex
asimell Feb 19, 2021
04b3b39
Change colour to orange
asimell Feb 19, 2021
b459fba
Add test to parse tags from RF 4.x output properly
asimell Feb 19, 2021
212ce5e
Add documentation to some test output.xml files
asimell Feb 19, 2021
4d38ca4
Proper syntax for skipped cases in table
asimell Feb 19, 2021
ce40a0c
Parse tags properly
asimell Feb 19, 2021
1e820bf
Add more skip status indications
asimell Feb 19, 2021
2578ca2
Small refactor for parsing arguments and add "tag" to list of caught
asimell Feb 22, 2021
fed320e
Ignore skipped test cases when getting all failed cases
asimell Feb 22, 2021
9ed6aa1
Replace StringBuffers with StringBuilders
asimell Feb 22, 2021
ef5235b
Add unit test to test stacktrace parsing
asimell Feb 22, 2021
59f8f81
Properly parse arguments and args
asimell Feb 23, 2021
b65dc5d
Add instructions for RF 4 configuration
asimell Feb 23, 2021
726f3a3
Update documentation
asimell Feb 23, 2021
e012be4
Fix getFailedSince()
asimell Feb 23, 2021
3e0eb42
Merge branch 'master' into new-robot-parser
asimell Mar 9, 2021
10a19f1
Move keyword parsing to a separate method
asimell Mar 9, 2021
a09e9a2
Refactor kw parsing and add for-loop parsing
asimell Mar 10, 2021
21f1607
Add unit test for RF4.0 nested for loop parsing
asimell Mar 10, 2021
c131f91
Refactor BlueOcean tests
asimell Mar 10, 2021
4b4013f
Fix stacktrace parsing
asimell Mar 10, 2021
d15ef8e
Delete commented lines
asimell Mar 10, 2021
cf9715f
Add test to parse if branch
asimell Mar 10, 2021
09995bb
Add if-branch parsing
asimell Mar 10, 2021
280a959
Fix for loops and ifs with no contents and prettify indentations
asimell Mar 11, 2021
092e4ca
Add test to check empty for loop
asimell Mar 11, 2021
d695079
Add tests to parse stacktrace
asimell Mar 11, 2021
78e703b
Fix test expected values
asimell Mar 11, 2021
4c89cb3
Fix stacktrace newlines
asimell Mar 11, 2021
15dcea5
Fix keyword and arg parsing
asimell Mar 11, 2021
d1f551a
Add default to switch case
asimell Mar 12, 2021
3b6085a
Add comment for clarity
asimell Mar 12, 2021
f7b69d2
Update doc/README.md
asimell Mar 12, 2021
3e76cf8
Update doc/README.md
asimell Mar 12, 2021
249cea7
Update src/main/java/hudson/plugins/robot/RobotParser.java
asimell Mar 12, 2021
ebab509
Unify structures
asimell Mar 12, 2021
788f2b4
Add explicit typing for clarity
asimell Mar 12, 2021
f81650f
Add braces for clarity
asimell Mar 12, 2021
4d9622d
Add braces to if
asimell Mar 12, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
35 changes: 23 additions & 12 deletions doc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,10 +107,10 @@ build. Conversely, the project view side panel link won't go away until
after configuration change/resave in project. We're working on a
solution to this.

![](images/loghtml_sidepanel.png)
![sidepanel link](images/sidepanel.png)
*side panel link*

![](images/robo_loglink_projectpage.png)
![summary link](images/robot_4_table.png)
*Summary link*

#### Configuring Robot overall pass/fail to show in the project list
Expand All @@ -128,7 +128,7 @@ solution to this.
[http://YOURJENKINSHOST/configure](http://yourjenkinshost/configure)
)and select your newly created view to be the default one.

![](images/robot_view_column.png)
![List view](images/robot_view_column.png)
*List view column in action*

#### Using token macros in e-mail report
Expand Down Expand Up @@ -178,25 +178,36 @@ and [here](https://content-security-policy.com/) how to change you CSP settings
but be aware that **Changing CSP settings will potentially expose you Jenkins instance for
security vulnerabilities**.

### Overall Screenshots
### Robot Framework 4.x compatibility

##### Config
The plugin supports both Robot Framework 3.x and 4.x output files. However, in order to support both, the plugin
shows some extra information for both. [In Robot Framework 4.0 test criticality was removed and "SKIP" status was added](https://github.com/robotframework/robotframework/blob/master/doc/releasenotes/rf-4.0.rst). So for 3.x the
results overview will show a `Skipped` column, which will always be 0 and for Robot Frameowork 4.x output files
the `Critical tests` row will always be 0.

![](images/config1.png)
Skipped tests aren't taken into account when calculating pass percentage, but they are calculated to the total
amount of tests.

##### Project view
Because criticality was removed in Robot Framework 4.0, having the `Use thresholds for critical tests only` checkbox
checked will always result in a passing step (because pass percentage is always considered to be 100% when there are
0 tests). In order to have set build status correctly, you **must** uncheck the checkbox or use `onlyCritical: false`
in your pipeline when you call `robot`.

asimell marked this conversation as resolved.
Show resolved Hide resolved
![](images/project_page.png)
We are planning of dropping support for Robot Framework 3.x and earlier some time in distant future.

### Overall Screenshots


##### Project view

##### Build view
![Build page overview](images/build_page.png)

![](images/build_page.png)

##### Detailed build view

![](images/detailed.png)
![Detailed results view](images/detailed.png)

## Bugs

[List of open bugs on
robot-plugin](https://issues.jenkins-ci.org/issues/?jql=component%20%3D%20robot-plugin%20AND%20resolution%20is%20EMPTY%20ORDER%20BY%20updated%20DESC)
robot-plugin](https://issues.jenkins-ci.org/issues/?jql=component%20%3D%20robot-plugin%20AND%20resolution%20is%20EMPTY%20ORDER%20BY%20updated%20DESC)
Binary file modified doc/images/build_page.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified doc/images/detailed.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed doc/images/loghtml_sidepanel.png
Binary file not shown.
Binary file removed doc/images/robo_loglink_projectpage.png
Binary file not shown.
Binary file added doc/images/robot_4_table.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified doc/images/robot_view_column.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/images/sidepanel.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
15 changes: 13 additions & 2 deletions src/main/java/hudson/plugins/robot/AggregatedRobotAction.java
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ public static class AggregatedRobotResult extends RobotResult {

private static final long serialVersionUID = 1L;
private final transient AggregatedRobotAction parent;
private int passed, failed, criticalPassed, criticalFailed;
private int passed, failed, skipped, criticalPassed, criticalFailed;

public AggregatedRobotResult(AggregatedRobotAction parent) {
this.parent = parent;
Expand All @@ -120,6 +120,7 @@ public void addResult(RobotResult result) {
criticalPassed += result.getCriticalPassed();
failed += result.getOverallFailed();
passed += result.getOverallPassed();
skipped += result.getOverallSkipped();
}

@Override
Expand Down Expand Up @@ -147,9 +148,14 @@ public long getOverallFailed() {
return failed;
}

@Override
public long getOverallSkipped() {
return skipped;
}

@Override
public long getOverallTotal() {
return failed + passed;
return failed + passed + skipped;
}

@Override
Expand All @@ -162,6 +168,11 @@ public int getPassed() {
return (int) getOverallPassed();
}

@Override
public int getSkipped() {
return (int) getOverallSkipped();
}

@Override
public AbstractBuild getOwner() {
return parent.getOwner();
Expand Down