Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cleanup acceptance tests #2091

Closed
8 tasks done
jussimalinen opened this issue Aug 6, 2015 · 2 comments
Closed
8 tasks done

Cleanup acceptance tests #2091

jussimalinen opened this issue Aug 6, 2015 · 2 comments
Assignees
@jussimalinen
Labels
enhancement priority: medium
Milestone
2.9.2

Comments

@jussimalinen
Copy link
Member

The Robot Framework's own acceptance need some cleanup. Different platforms and requirements are handled sometimes by tags, sometimes by setting test non critical. The names for these tags are inconsistent as well. Also the keyword names are not written in consistent style. There is probably much more... It would be great to clean this stuff up along with the Python 3.0 support.

  • Change pybot, jybot, and x-fails-on-ipy tags to no-jython, only-jython, no-ipy etc
  • Get rid of regression tag if possible
  • If possible, change the non critical tests to be critical and just don't run them on a platform where they don't work
  • Add tags require-lxml, require-docutils, require-yaml, and screenshot and make all of these tests to be always critical.
  • Add no-ci tag or similar, which would skip manual and telnet tests (all other could then be critical on ci?)
  • Keyword names should be written in consistent style
  • Cleanup and streamline the test running system and atest/run_atest.py in general. Perhaps rename it to atest/run.py.
  • Update atets/README.rst.
@jussimalinen jussimalinen added this to the 3.0 milestone Aug 6, 2015
pekkaklarck added a commit that referenced this issue Sep 11, 2015
@pekkaklarck
Tag cleanup. One part of #2091.
65efe0c 
Includes:
- Remove pybot/jybot tags. Use only-jython, no-jython, etc. instead.
- Do not set IronPython, Python 2.6, Windows, etc. tests
  non-critical when they cannot pass. Use no-ipy, no-python26,
  no-windows, etc. tags instead.

TODO related to tags:
- Still some more tests that are set non-critical can be excluded.
- Fix --TagStatXXX options in run_atests.py.
@pekkaklarck pekkaklarck modified the milestones: 2.9.2, 3.0 Sep 14, 2015
jussimalinen added a commit that referenced this issue Sep 14, 2015
@jussimalinen
Instead of making tests non critical, prefer not running them. #2091
e23e718
@jussimalinen jussimalinen self-assigned this Sep 15, 2015
jussimalinen added a commit that referenced this issue Sep 15, 2015
@jussimalinen
Add tags require-lxml, require-docutils, require-yaml, and screenshot…
b95d4f3 
… and make all of these tests to be always critical. #2091
jussimalinen added a commit that referenced this issue Sep 16, 2015
@jussimalinen
Change requires-lxml to require-lxml. #2091
dc2f27a
jussimalinen added a commit that referenced this issue Sep 16, 2015
@jussimalinen
Cleanup atest tags to use require-something when ever possible. #2091
318facb 
The run_atest will exclude the require-something tags based on the platform.
jussimalinen added a commit that referenced this issue Sep 16, 2015
@jussimalinen
Removed regression tag. #2091
09a6205
jussimalinen added a commit that referenced this issue Sep 21, 2015
@jussimalinen
Cleanup interpreter checks in atests. #2091
d6024f4
jussimalinen added a commit that referenced this issue Sep 21, 2015
@jussimalinen
added requirements file for atest dependencies. #2091
e68e559
pekkaklarck added a commit that referenced this issue Sep 21, 2015
@pekkaklarck
cleaned up run_atests.py. lot's to do still. #2091
b655d9c
pekkaklarck added a commit that referenced this issue Sep 22, 2015
@pekkaklarck
atest execution cleanup (#2091)
b003a24 
- Separate class to inspect and store interpreter information.
- Use that class to pass info to executed tests via a variable file.
jussimalinen added a commit that referenced this issue Sep 23, 2015
@jussimalinen
Use semicolon as argument separator in atest running to support bette…
29a0a98 
…r paths in windows. #2091
jussimalinen added a commit that referenced this issue Sep 25, 2015
@jussimalinen
Cleanup acceptance tests runnign. #2091
962b786
@pekkaklarck pekkaklarck mentioned this issue Sep 28, 2015
Closed
pekkaklarck added a commit that referenced this issue Sep 28, 2015
@pekkaklarck
Fixed atest issues on Windows and added kws to split/join cli args.
334886c 
- Added new 'Command Line To List' and 'List To Command Line' keywords to the
  Process library to ease splitting/joining cli args. Test and code review as
  well as documentation still missing. #2160

- Replaced simple 'shlex.split' usage in atests with the aforementioned
  Command Line To List keyword in Robot data and with enhanced shlex usage
  in Python libraries.

- Added output files to generic Robot/Rebot execution keywords to avoid output
  streams getting full and execution hang on Windows.

- Added 5 minute timeouts to Robot/Rebot execution keywords to avoid hanging
  in general.

Above fixes related to the overall atest execution cleanup (#2091).
jussimalinen added a commit that referenced this issue Sep 29, 2015
@jussimalinen
run_atests and run_utests are now just run. #2091
5667e46
pekkaklarck added a commit that referenced this issue Sep 29, 2015
@pekkaklarck
Cleaning running atests. Not yet ready but getting there... #2091
ee232b9
jussimalinen added a commit that referenced this issue Sep 30, 2015
@jussimalinen
dont write to run directory during atest run. #2091
2ff9166
pekkaklarck added a commit that referenced this issue Sep 30, 2015
@pekkaklarck
atests: Pass sources to robot/rebot as string, not list. #2091
ad3c0d7
pekkaklarck added a commit that referenced this issue Sep 30, 2015
@pekkaklarck
atests: Re-introduced helper to run robot/rebot. Return result, not rc.
61d2e6e 
#2091
pekkaklarck added a commit that referenced this issue Sep 30, 2015
@pekkaklarck
atests: Cleaning up execution: Avoiding Run R?ebot Directly. #2091
772270f
pekkaklarck added a commit that referenced this issue Sep 30, 2015
@pekkaklarck
Cleaning up atest execution #2091.
521ab92 
- Replace `Run Robot/Rebot Directly` with `Run Tests/Rebot Without Defaults`
  that utilizes `Run Tests/Rebot` internally.
- Cleanup to related tests.
pekkaklarck added a commit that referenced this issue Oct 1, 2015
@pekkaklarck
atests: Refactoring "Run Tests/Rebot (Without ...)". #2091
bddef28
pekkaklarck added a commit that referenced this issue Oct 1, 2015
@pekkaklarck
atests: Refactoring "Run Tests/Rebot (Without ...)". #2091
4b5bf13
pekkaklarck added a commit that referenced this issue Oct 2, 2015
@pekkaklarck
atest_resource: TODOs related to #2091. Also minor cleanup.
dd3fc16
@pekkaklarck
Copy link
Member

This task turned out to be much bigger than we expected but we are also happy with the end results. Remaining tasks:

  1. Resolve TODOs in atest_resource.robot added in dd3fc16.
    • Mainly keyword cleanup.
    • Some may not be good idea (generally contain ? at the end)
  2. Update atest/README.rst
    • Explain using no-ci.
    • Update section about required modules.
      • Add a link to requirements.txt.
      • Explain how to avoid running tests if modules not available

jussimalinen added a commit that referenced this issue Oct 2, 2015
@jussimalinen
atest: cleanup for keyword names. #2091
03053e0
jussimalinen added a commit that referenced this issue Oct 2, 2015
@jussimalinen
atest: moved Check Log Message to python. #2091
58c1905
jussimalinen added a commit that referenced this issue Oct 2, 2015
@jussimalinen
atest: move Check Test Case to library. (lost 10mb on atest output.xml)
ac9c63e 
#2091
jussimalinen added a commit that referenced this issue Oct 2, 2015
@jussimalinen
atest: kw cleanup. #2091
2a09145
jussimalinen added a commit that referenced this issue Oct 7, 2015
@jussimalinen
atest: document new tags and installing requirements. #2091
d104828
jussimalinen added a commit that referenced this issue Oct 7, 2015
@jussimalinen
fix some typos. #2091
47f9bbd
pekkaklarck added a commit that referenced this issue Oct 8, 2015
@pekkaklarck
atest/README: Enhancing docs about tags and require modules #2091
6658b5e
@jussimalinen
Copy link
Member Author

Thats it for now I think.

pekkaklarck added a commit that referenced this issue Oct 9, 2015
@pekkaklarck
atests: Doc modules required by screenshot tests, consistent tag name #…
a9888c7 
…2091
@pekkaklarck pekkaklarck mentioned this issue Mar 26, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jussimalinen jussimalinen

Labels
enhancement priority: medium
Projects
None yet
Milestone
2.9.2
Development

No branches or pull requests
2 participants
@pekkaklarck @jussimalinen