Skip to content
Branch: master
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
154 lines (109 sloc) 7.6 KB

Where to start

We welcome contributions of any type (e.g., bug fixes, new features, reporting issues, documentation, etc). If you're looking for a good place to get started you might like to peruse our current Git issues (those marked with help wanted are a good place to start).

Coding conventions

Parsl code should adhere to Python pep-8. Install flake8 and run the following code to identify non-compliant code:

$ flake8 parsl/

Note: the continuous integration environment will validate all pull requests using this command.

Naming conventions

The following convention should be followed: ClassName, ExceptionName, GLOBAL_CONSTANT_NAME, and lowercase_with_underscores for everything else.

Version increments

Parsl follows the major.minor[.maintenance[.build]] numbering scheme for versions. Once major features for a specific milestone (minor version) are met, the minor version is incremented and released via PyPI and Conda. Fixes to minor releases are made via maintenance releases. Packaging instructions are included in the packaging docs


Classes should be documented following the NumPy/SciPy style. A concise summary is available here. User and developer documentation is auto-generated and made available on ReadTheDocs.


Parsl uses pytest to run most tests. All tests should be placed the parsl/tests directory. Before running tests usage tracking should be disabled using the PARSL_TESTING environment variable:

$ export PARSL_TESTING="true"

There are two broad groups of tests: those which must run with a specific configuration, and those which should work with any reasonable configuration.

Tests which run with a specific configuration live under the parsl/tests/sites and parsl/tests/integration directory. They are launched by .travis.yaml with a pytest parameter of --config local and each test file should initialise a DFK explicitly.

Tests which should with with any configuration live under themed directories parsl/tests/test*/ and should named test*.py. They can be run with any configuration, by specifying --config CONFIGPATH where CONFIGPATH is a path to a .py file exporting a parsl configuration object named config. The parsl-specific test fixtures with ensure a suitable DFK is loaded with that configuration for each test.

There is more fine-grained enabling and disabling of tests within the above categories:

A pytest marker of cleannet (for clean network) can be used to select or deselect tests which need a very cleen network (for example, for tests making FTP transfers). Travis does not provide a sufficiently clean network and so .travis.yml runs all tests with -k "not cleannet" to disable those tests.

Some other markers are available but unused in travis testing; see pytest --markers parsl/tests/ for more details.

A specific test in a specific file can be run like this::

$ pytest test_python_apps/

A timeout can be added to test runs using a pytest parameter such as --timeout=60

Many tests are marked with @pytest.mark.skip for reasons usually specified directly in the annotation - generally because they are broken in one way or another.

Coverage testing

There is also some coverage testing available. The CI by default records coverage for most of the tests that it runs and outputs a brief report at the end of each CI run. This is purely informational and a Lack of coverage won't produce a CI failure.

It is possible to produce a more detailed coverage report on your development machine: make sure you have no .coverage file, run the test commands as shown in .travis.yml, and then run coverage report to produce the summary as seen in CI, or run coverage html to produce annotated source code in the htmlcov/ subdirectory. This will show, line by line, if each line of parsl source code was executed during the coverage test.

Development Process

If you are a contributor to Parsl at large, we recommend forking the repository and submitting pull requests from your fork. The Parsl development team has the additional privilege of creating development branches on the repository. Parsl development follows a common pull request-based workflow similar to GitHub flow. That is:

  • every development activity (except very minor changes, which can be discussed in the PR) should have a related GitHub issue
  • all development occurs in branches (named with a short descriptive name which includes the associated issue number, for example, add-globus-transfer-#1)
  • the master branch is always stable
  • development branches should include tests for added features
  • development branches should be tested after being brought up-to-date with the master (in this way, what is being tested is what is actually going into the code; otherwise unexpected issues from merging may come up)
  • branches what have been successfully tested are merged via pull requests (PRs)
  • PRs should be used for review and discussion (except hot fixes, which can be pushed to master)
  • PRs should be reviewed in a timely manner, to reduce effort keeping them synced with other changes happening on the master branch

Git commit messages should include a single summary sentence followed by a more explanatory paragraph. Note: all commit messages should reference the GitHub issue to which they relate. A nice discussion on the topic can be found here.

Implemented Globus data staging support

Added the ability to reference and automatically transfer Globus-accessible
files. References are represented using the Parsl file format
“globus://endpoint/path/file.” If Globus endpoints are known for source and
destination Parsl will use the Globus transfer service to move data to the
compute host. Fixes #-1.

Git hooks

Developers may find it useful to setup a pre-commit git hook to automatically lint and run tests. This is a script which is run before each commit. For example:

$ cat ~/parsl/.git/hooks/pre-commit

flake8 parsl
nosetests -vx parsl/tests/test_threads parsl/tests/test_data parsl/tests/test_checkpointing

Project documentation

All project documentation is written in reStructuredText. Sphinx is used to generate the HTML documentation from the rst documentation and structured docstrings in Parsl code. Project documentation is built automatically and added to the Parsl documentation.

Credit and Contributions

Parsl wants to make sure that all contributors get credit for their contributions. When you make your first contribution, it should include updating the codemeta.json file to include yourself as a contributor to the project.

Discussion and Support

The best way to discuss development activities is via Git issues.

To get involved in community discussion please join the Parsl Slack channel.

You can’t perform that action at this time.