Skip to content

Commit

Permalink
remove tox in favor of gh actions/act (#1644)
Browse files Browse the repository at this point in the history
* remove tox in favor of gh actions/act

* update testing docs with local testing info
  • Loading branch information
bewing committed Mar 15, 2023
1 parent 22ae327 commit 8b670fe
Show file tree
Hide file tree
Showing 3 changed files with 44 additions and 51 deletions.
55 changes: 44 additions & 11 deletions docs/development/testing_framework.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Testing Framework
-----------------

As napalm consists of multiple drivers and all of them have to provide similar functionality, we have developed a testing framework to provide a consistent test suite for all the drivers.
As NAPALM consists of multiple drivers and all of them have to provide similar functionality, we have developed a testing framework to provide a consistent test suite for all the drivers.

Features
________
Expand Down Expand Up @@ -42,7 +42,7 @@ By default, the tests are going to be run against mocked data but you can change
* ``NAPALM_USERNAME``
* ``NAPALM_PASSWORD``
* ``NAPALM_OPTIONAL_ARGS``

Mocking the ``open`` method
^^^^^^^^^^^^^^^^^^^^^^^^^^^

Expand All @@ -56,29 +56,29 @@ Multiple test cases::
(napalm) ➜ napalm-eos git:(test_framework) ✗ ls test/unit/mocked_data/test_get_bgp_neighbors
lots_of_peers no_peers normal
(napalm) ➜ napalm-eos git:(test_framework) ✗ py.test test/unit/test_getters.py::TestGetter::test_get_bgp_neighbors
...
...
test/unit/test_getters.py::TestGetter::test_get_bgp_neighbors[lots_of_peers] <- ../napalm/napalm.base/test/getters.py PASSED
test/unit/test_getters.py::TestGetter::test_get_bgp_neighbors[no_peers] <- ../napalm/napalm.base/test/getters.py PASSED
test/unit/test_getters.py::TestGetter::test_get_bgp_neighbors[normal] <- ../napalm/napalm.base/test/getters.py PASSED

Missing test cases::

(napalm) ➜ napalm-eos git:(test_framework) ✗ ls test/unit/mocked_data/test_get_bgp_neighbors
ls: test/unit/mocked_data/test_get_bgp_neighbors: No such file or directory
(napalm) ➜ napalm-eos git:(test_framework) ✗ py.test test/unit/test_getters.py::TestGetter::test_get_bgp_neighbors
...
...
test/unit/test_getters.py::TestGetter::test_get_bgp_neighbors[no_test_case_found] <- ../napalm/napalm.base/test/getters.py FAILED

========================================================= FAILURES ==========================================================
___________________________________ TestGetter.test_get_bgp_neighbors[no_test_case_found] ___________________________________

cls = <test_getters.TestGetter instance at 0x10ed5eb90>, test_case = 'no_test_case_found'

@functools.wraps(func)
def wrapper(cls, test_case):
cls.device.device.current_test = func.__name__
cls.device.device.current_test_case = test_case

try:
# This is an ugly, ugly, ugly hack because some python objects don't load
# as expected. For example, dicts where integers are strings
Expand All @@ -87,7 +87,7 @@ Missing test cases::
if test_case == "no_test_case_found":
> pytest.fail("No test case for '{}' found".format(func.__name__))
E Failed: No test case for 'test_get_bgp_neighbors' found

../napalm/napalm.base/test/getters.py:64: Failed
================================================= 1 failed in 0.12 seconds ==================================================

Expand All @@ -96,8 +96,41 @@ Method not implemented::
(napalm) ➜ napalm-eos git:(test_framework) ✗ py.test test/unit/test_getters.py::TestGetter::test_get_probes_config
...
test/unit/test_getters.py::TestGetter::test_get_probes_config[no_test_case_found] <- ../napalm/napalm.base/test/getters.py SKIPPED

================================================= 1 skipped in 0.09 seconds =================================================

Testing Matrix
--------------

NAPALM leverages [Github Actions](https://docs.github.com/en/actions) to test and lint code on commits and pull requests.
If you want to test prior to opening a pull request, you can use [nektos/act](https://github.com/nektos/act) and Docker to locally run the tests

.. code-block:: console
$ act -j std_tests
[build/std_tests-4] 🚀 Start image=catthehacker/ubuntu:act-latest
[build/std_tests-3] 🚀 Start image=catthehacker/ubuntu:act-latest
[build/std_tests-1] 🚀 Start image=catthehacker/ubuntu:act-latest
[build/std_tests-2] 🚀 Start image=catthehacker/ubuntu:act-latest
[build/std_tests-5] 🚀 Start image=catthehacker/ubuntu:act-latest
[build/std_tests-4] 🐳 docker pull image=catthehacker/ubuntu:act-latest platform= username= forcePull=true
[build/std_tests-1] 🐳 docker pull image=catthehacker/ubuntu:act-latest platform= username= forcePull=true
[build/std_tests-3] 🐳 docker pull image=catthehacker/ubuntu:act-latest platform= username= forcePull=true
[build/std_tests-5] 🐳 docker pull image=catthehacker/ubuntu:act-latest platform= username= forcePull=true
[build/std_tests-2] 🐳 docker pull image=catthehacker/ubuntu:act-latest platform= username= forcePull=true
...
| ---------------------------------------------------------------------
| TOTAL 9258 1836 80%
|
| ================= 619 passed, 80 skipped, 3 warnings in 19.97s =================
[build/std_tests-5] ✅ Success - Main Run Tests
[build/std_tests-5] ⭐ Run Post Setup Python 3.11
[build/std_tests-5] 🐳 docker exec cmd=[node /var/run/act/actions/actions-setup-python@v2/dist/cache-save/index.js] user= workdir=
[build/std_tests-5] ✅ Success - Post Setup Python 3.11
[build/std_tests-5] 🏁 Job succeeded
.. _`test_getters.py`: https://github.com/napalm-automation/napalm-eos/blob/a2fc2cf6a98b0851efe4cba907086191b8f1df02/test/unit/test_getters.py
.. _`conftest.py`: https://github.com/napalm-automation/napalm-eos/blob/a2fc2cf6a98b0851efe4cba907086191b8f1df02/test/unit/conftest.py
1 change: 0 additions & 1 deletion requirements-dev.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ pytest-json==0.4.0
pyflakes==2.5.0
pylama==8.4.1
mock==4.0.3
tox==3.25.1
mypy==0.982
types-requests==2.28.0
types-six==1.16.17
Expand Down
39 changes: 0 additions & 39 deletions tox.ini

This file was deleted.

0 comments on commit 8b670fe

Please sign in to comment.