Welkin is a test automation framework shell written in Python and using the pytest test runner.
Welkin is designed to support functional testing for one or more apps in an ecosystem, as well as end-to-end testing across these apps.
Welkin provides a basic starting point for building a custom test framework; you add the test data models, custom application wrappers, and the tests. Welkin is intended as a teaching tool for beginning test automators; welkin is NOT a general test framework or test tool.
Welkin is not a test framework, because
- it doesn't have a body of tests beyond simple demonstration examples;
- it doesn't have any application models or wrappers.
Welkin is a scaffolding on which a custom test automation framework can be built.
And Welkin is not a turn-key tool; you will need to modify Welkin to make it useful for your test needs.
Ok, here's the context behind welkin:
- A robust and useful test automation framework requires a test runner to collect, execute, and report on tests. Welkin is built around the pytest test runner, and relies on pytest for test collection, test execution, and the reporting of test results.
- A test framework needs to manage how and where the test results are reported, along with managing the logging of test activity. Welkin creates date and timestamped logs and html test results into a testrun-specific output folder.
- A test framework needs a body of tests in a place where the test runner can find them. Welkin is just a scaffolding, so it provides some example tests to show how this all works together.
- A test framework probably should provide support for abstracting out some test logic so that tests can be written at higher level.
- A test framework should provide a mechanism for abstracting out test data from the tests.
- A test framework should provide tools and utilities to support test activities, and to abstract out repeated supporting actions from the tests.
A test framework should be customized to the specific business domain you are working in, and to the applications under test.
Welkin is intended to be a starting point for your automation code framework, so while there is a range of ways the code could be packaged and distributed cleanly, just fork this project and set it up locally.
What I do is:
- think up a cool name for the test automation framework I'm going to build
- clone the code to a local repo
- set up a virtualenv
- run requirements.txt
- add mappings to the local welkin project in my bash profile
- create a new repo on github for the new framework (Welkin has done it's job)
You will need to create the following directory in your home folder: ~/dev
Create a virtualenv in the directory ~/dev/venv. Don't activate that yet.
On a Mac, you'll probably have to add mappings in your bash profile to the project:
export PATH=$PATH:~/dev/venv
export PATH=$PATH:~/dev/welkin
export PYTHONPATH=$PYTHONPATH:~/dev/welkin
Activate the virtualenv ~/dev/venv.
Welkin uses Selenium Manager for managing browser drivers. In order to allow the automatic provision of chrome drivers, do not have a chromedriver executable in your PATH. What happens is roughly this:
- there must not be a driver in the PATH
- Selenium Manager will perform driver discovery for ChromeforTesting driver (https://googlechromelabs.github.io/chrome-for-testing/)
- default behavior calls the stable version
- if not cached in ~/.cache/selenium/chrome, then it's downloaded and cached
For more information on Selenium manager, see What's new in Selenium Manager with Selenium 4.11.0.
Welkin is typically run from the command line. Pytest handles test collection and the reporting of errors.
The base command to collect and run tests with pytest is the following:
$ cd <path_to_welkin>
# you should be in the top-level welkin folder
$ pytest welkin/tests
Pytest supports several ways to control test collection.
- string matches in the name path for the test methods
# collect and run every test with the string "example" in filename, classname, method name
$ pytest welkin/tests -k example
- marker matches for test classes or methods
# collect and run every test class or test method marked with the pytest.marker "example"
$ pytest welkin/tests -m example
- combined marker matches AND string matches
# collect and run every test marked with the pytest.marker "example" AND containing the string "simple"
$ pytest welkin/tests -m example -k simple
Sometimes you'll need to collect but not run tests:
# collect every test marked with the pytest.marker "example" AND containing the string "simple"
$ pytest welkin/tests -m example -k simple --collect-only
collected 7 items
<Module 'examples/test_examples.py'>
<Class 'ExampleTests'>
<Instance '()'>
<Function 'test_simple_pass'>
<Function 'test_simple_fail'>
To stop a test run, hit CTRL + C.
Optional command line arguments you can pass to welkin:
- env is the environment to run the tests against; the choices are 'local', 'qa', 'staging'; defaults to 'qa'. These don't work out of the box; they are placeholders that need to be configured with the actual name and URLs.
Welkin is intended to be verbose in its logging; however, that's up to you to implement as you build out your own framework.
By default, Welkin creates an output folder at welkin/output, and then for each test run Welkin creates a folder in output named with the testrun's timestamp; this folder gets the HTML test results page, plus the text log of test run activity. This output is not automatically cleaned up. You'll have to define a workflow for this, if you want.
From the Flake8 description:
"Flake8 is a command-line utility for enforcing style consistency across Python projects. By default, it includes lint checks provided by the PyFlakes project, PEP-0008 inspired style checks provided by the PyCodeStyle project, and McCaWhat's new in Selenium Manager with Selenium 4.11.0be complexity checking provided by the McCabe project. It will also run third-party extensions if they are found and installed."
Running flake8 against the welkin codebase:
# from the project directory
$ flake8 > output/flake8.txt
Specific configuration instructions can be set in setup.cfg.
Some examples of solving particular automation challenges.