Automated testing w/ Travis or Github Actions

[jwlodek]

Is your feature request related to a problem? Please describe.
Unit testing

Describe the solution you'd like
Add automated CI unit testing with Travis CI / Github Actions

Additional context
At the moment there don't seem to be too many unit tests either. I think they should be expanded, and CI testing should be enabled. I could look at this if you would like.

[GeorgeCiesinski]

Hi @jwlodek , thanks for opening this issue. We absolutely need to expand on testing in this repository. I looked into Travis CI and I think this is perfect for this repository. I would be happy to have you look into this.

I just wanted to note that we have been writing some testing in Unit Test, but it is regrettably very behind. There have been a few issues for testing open which I haven't gotten around to yet. Please let me know if testing will need to be done a different way to be compatible with Travis CI or if anything else is required from my end to accommodate this.

[jwlodek]

Travis can be configured to run any shell commands, so as long as you can run the tests from the terminal, Travis can handle it. I usually use pytest for this, since you only need to run the pytest command to run all the tests automatically, so it makes this configuration very simple, but the standard unit testing library should work fine. Travis will run the specified shell command, and return a failure with a non-zero exit code, or a success with a 0 exit code.

You can take a look here:
https://github.com/jwlodek/py_cui/blob/master/.travis.yml

at a travis configuration I made for one of my repos, and here:
https://travis-ci.org/github/jwlodek/py_cui/builds/660696169?utm_medium=notification&utm_source=github_status

to see how the automated testing interface works for that configuration.

[GeorgeCiesinski]

This looks great. I have looked into Pytest as well, and would be happy to use this instead of Unit Test. I haven't built any significant test scripts in Unit Test so it will not be difficult to change over to Pytest.

[jwlodek]

I added this changes in #55. You will still need to enable travis on your end. Running pytest in the root directory of the project will run the tests (at least it does ok for me on linux).

Additional tests can be added by making a test_$MODULE.py file in the tests directory, and making a function

def test_FUNCTION_NAME():

function

[GeorgeCiesinski]

Awesome, thanks for adding this and explaining how it works! It works great on my end. This was a major milestone I had planned but haven't had a chance to look into. I just have a few more questions:

• Does this work across all branches and pull requests as well?
• What is the purpose of requirements_dev.txt btw?
• Also, you mentioned a change to the documentation. Is this something outside of your pull request, or did you already include the change?
• Do you want me to link your github profile under contributors?

[jwlodek]

I'll do my best to answer these:

• It currently does work on all branches and PRs. If you don't want that, you can edit the .travis.yml file and add a job. There you can specify an action to perform given certain conditions, like only running tests on master, or on tag creating new release versions. For now it is just the super basic "run the tests on all branches". When there is a PR, it will run on the merging branch, and will tell you if all the unit tests pass for it. Check out this link for how to configure jobs on travis.
• For projects that end up on PyPi, the requirements.txt file is used to know which dependencies should also be installed with pip install. Usually, end users don't care about things like unit testing, and debug logging, so packages specific to that are placed in a separate file (that also contains the same packages as requirements.txt) for developers and CI. It isn't super important, just a conveniance.
• For the docs, I wasn't sure if you had anything about writing unit tests, I just added that tag in case it had to be changed from unittest to pytest
• Sure, thanks!

[GeorgeCiesinski]

Thanks for answering the questions, that makes sense. I will look further into Travis CI and pytest!