From 052434300d228dad4001742a7d9ab37ccf4f0bad Mon Sep 17 00:00:00 2001 From: joaomcteixeira Date: Tue, 22 Feb 2022 13:31:17 +0100 Subject: [PATCH 1/7] further improves CONTRIBUTING --- CONTRIBUTING.md | 221 ++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 203 insertions(+), 18 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8ec4fbdac..afa466a18 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -10,25 +10,29 @@ fantastic project. There are several ways to contribute: * write more unittests (we dare you to do that :godmode:) Before attempting any development, please install HADDOCK3 following the -instructions in the [INSTALL](INSTALL.md) file. +instructions in the [INSTALL](INSTALL.md) file. Afterwards, follow the +instructions in this file. -## Contributing new code +## 1. Contributing new code HADDOCK3 has two main testing workflows. Here, within the repository, we -test the HADDOCK3's Python shell, code style, and building. Our +test the HADDOCK3's Python shell, code style, and package building. Our Continuous Integration (CI) pipeline is based on -[tox](https://tox.wiki/en/latest/index.html) and GitHub Actions. +[tox][tox] and GitHub Actions. We will +explain you how to use them. To contribute to the HADDOCK3's Python shell, follow these steps: -1. [Fork](https://www.earthdatascience.org/workshops/intro-version-control-git/about-forks/) the HADDOCK3 repository +1. [Fork][fork] the HADDOCK3 repository 2. Create a new branch in your fork (`git checkout -b `) 3. Develop your code and tests: 1. HADDOCK3 source is in `src/haddock`. Always implement code in - the lowest Python version supported (this case is 3.9) + the lowest Python version supported (this case is 3.9). 2. Tests sit in the `tests/` folder. Use [pytest](https://docs.pytest.org/en/6.2.x/) for testing. + 3. See more details on how to contribute with code and tests in the + subheadings below. 4. While you are developing (or when you think you have finished), you can (should) use our `tox` environments to test your code. Use the @@ -42,17 +46,20 @@ To contribute to the HADDOCK3's Python shell, follow these steps: 5. You can work on these `tox` tests until they all pass green before submitting your PR. -6. We also have an `examples` folder with `test` cases that you can run +6. We also have an `examples` folder with test cases that you can run (should) to ensure the integrity of the python shell as a whole: 1. Navigate to any of the examples folder and run the `-test.cfg` file (see [USAGE](USAGE.md)). 2. If you have a powerful computer and want to run all tests in a row, navigate to the `examples` folder and run `python run_tests.py -b`. The `-b` flag ensures the run will stop asap an error is found. + 3. if your computer is not powerful enough, you can ask us to run + those tests once you submitted the pull request. 7. Add a list of your new additions to the `CHANGELOG.md` file by adding a new sub-header as described bellow. This is mandatory for `tox --e build` to pass. +-e build` to pass. Note this applies only after we have released the +stable `3.0.0` version. ```markdown # Changelog @@ -65,17 +72,195 @@ adding a new sub-header as described bellow. This is mandatory for `tox (... the rest of the file ...) ``` -7. If you have difficulties with `tox`, let us know. These `tox` tests +8. If you have difficulties with `tox`, let us know. These `tox` tests are the same that run on the GitHub Actions once you send the PR. So, sending the PR is another way to ensure all tests pass. -8. Submit your PR if you haven't done so yet :wink: +9. Submit your PR if you haven't done so yet :wink: - 1. Now, GitHub allows submitting "Draft PRs" you can use this + 1. GitHub allows submitting "Draft PRs". You can use this option to let us know you are working on some new good stuff so we can help you from the beginning. -## Contributing with documentation +### 1.1 Contributing with code (additional details) + +**You should also know that...** HADDOCK3 folders, files, and code +follow a well defined structure with very specific patterns. Inside each +source folder you will (likely) find a `README` file describing the +structure of the folder and presenting guidelines on how to better +contribute to that folder and respective files. Summarizing here: + +* New command-line clients go in the `clis/` folder. See how the current + clients here created and use that as a template. Remember to update also the + `setup.py` file. +* Add any new functions that you foresee are general and + could serve different places in the code to the `libs/` folder. Find a + `lib*.py` files that would serve your purpuses, otherwise create a new + one. +* Any plug-in like functionality, for example *"check if input is + correct"*, should go into its own python module inside the `gear/` + folder. Any variables, functions, and classes related only to that + implementation should go all inside the same module (Python file). +* Any general hard definitions, or physical constants, should go in the + `core/` folder. +* If you want to implement a new HADDOCK3 module, navigate to the + `modules/` folder and follow the instructions in the `README` file + there. You will see that all folders and files follow a pattern. +* Talk with us before developing any CNS related part. + +### 1.2 Contributing with tests (additional details) + +Inside the `tests/` folder you will find several `test_*.py` files. +Normally, each file has the tests for each `*.py` file in the source. If +you create new `*.py` files you should create a new test file of the +same name, `test_new_file_name.py`. Aim at 100% test coverage for the +code you have created. Write tests according to [pytest][pytest]. You +can see examples in our `test_*.py` files. You can run the tests using +the `tox -e py39` commands explained above. Or, if you want to run the +tests for a singly file use `tox -e py39 -- tests/test_myfile.py`. + +### 1.3 Dependencies + +HADDOCK3 is highly interconnected with other projects. Many of them use +HADDOCK3 core functionalities. Therefore, we aim to keep HADDOCK3 with +the lowest dependency footprint possible. Avoid adding dependencies when +developing new functionalities. How? + +1. Use the Python standard library as much as possible. +1. Numpy is always allowed. +1. You need a small function from a large library. Try to reimplement it +yourself with the Python standard library. + 1. If you can't, talk first with +us by opening an issue. +1. You need a big and complex function or maybe a whole python file from +another project. If licenses are compatible, copy their code to the +HADDOCK3 repository writing all necessary headers to grant authorship +and comply with license requirements. + 1. If licenses aren't compatible, talk is us. We may have an + alternative. +1. Your new module largely depends on a library and reimplementing or +copying is not an option. Then, consider if we can use that dependency +as a runtime dependency (like `gdock` or `lightdock`) instead of an +installation dependency. +1. Nothing of the above is possible. You **really** need an install +dependency. Talk with us before. + +### 1.4 Code style + +HADDOCK3 follows nice code style rules. These are *almost* hard rules, +but there is some room for exceptions - common sense prevails. When +developing code, run `tox -e lint` to inspect if your code follows our +conventions. We use [flake8][flake] with the [following rules][fr] and +docstrings follow [numpydoc][numpydoc] style. You can hold on our code +for style. Nevertheless, here's a dummy code snippet for your reference. +Line have a soft max of 80 chars. + +```python +"""Module's docstring.""" +# any comments. You can add here licensing stuff +# first import standard library +import os +from path import Pathlib + +# import third library +import numpy as np + +# import from haddock3 +from haddock import log +# multiline import +from haddock.libs.libio import ( + lib1, + lib2, + lib3, + lib4, + ) + + +# all this are possible +GLOBAL_VARIBLE = None +_hidden_global_variable = None + +avoid_lower_case_globals = None + + +def my_nice_func(arg1, arg2, arg3): + """Docstring here.""" + # do stuf + +# if the function has many arguments, separate them by new lines. +# mind the identations +def my_nice_func_with_big_name( + var1, + var2, + var3, + named_var1="something", + named_var2="else", + **kwargs, + ): + """Docstring here.""" + # do stuff + return + + +multiline_list = [ + var1, + var2, + var3, + var4, + ] + +multiline_dict = { + "key1": 1, + "key2": 2, + "key3": 3, + "key4": 4, + "key5": 5, + } + +# separate comprehensions by sections if line don't fit 80 chars +# same for dictionaries +cool_list_comprehension = [ + dome_some_long_stuff(i) + for i in some_iterable + if validate(i) + ] + +double_quotes_allowed = "string" +single_quotes_allowed = 'string' +# don't try to homogenize quotes all around the code, respect the +# original input. + +# define first use later +generator_with_a_large_name = zip( + iter1, + iter2, + iter3, + iter4, + iter5, + ) + +for a, b, c, d, e in generator_with_a_large_name: + # do stuff + +# use noqa: E501 for slightly long lines, in case splitting into +# multiple lines reduces readability +this_is_a_long_line = that_makes_no_much_sense(to_separate, in_multiple_lines, abcd) # noqa: E501 +``` + +### 1.5 Code structure + +Write code in the form of small functions, because small functions are +easier to test. Combine small functions to compose larger functions. If +you need to use a global variable in a function, pass it as a default +value of a parameter. Avoid using complex classes, or avoid using +classes at all, unless you need to maintain a state or you really know +what you are doing. Flat is better than nested (though it's harder to +write, but it's easier to maintain and read). Use long variable names if +needed. Write comments that explain why you do stuff, and not how you do +stuff. Use the `TODO:` flag in your comments to note something for the +future. If needed, raise an issue. + +## 2. Contributing with documentation HADDOCK3 has (will have) two sources of documentation: 1) the library documentation itself that is built from code docstrings and `.rst` pages @@ -85,10 +270,10 @@ talk with us before start working. On the other hand, if you want to improve code's documentation, go for it following the same procedure as described above for *code contributions*. -## You should also know that... -HADDOCK3 folders, files, and code follow a well defined structure with -very specific patterns. Inside each source folder you will (likely) find -a `README` file describing the structure of the folder and presenting -guidelines on how to better contribute to that folder and respective -files. +[tox]: https://tox.wiki/en/latest/index.html "tox" +[fork]: https://docs.github.com/en/get-started/quickstart/fork-a-repo +[pytest]: https://docs.pytest.org/ "pytest" +[flake]: https://flake8.pycqa.org/en/latest/ "flake8" +[fr]: https://github.com/haddocking/haddock3/blob/b44304f0a1509d44d9689d7e2e4124f0ae3af0a1/tox.ini#L116-L130 +[numpydoc]: https://numpydoc.readthedocs.io/en/latest/format.html From 68e89390cd47a5e4e31fd44283b38c652dfa79e2 Mon Sep 17 00:00:00 2001 From: joaomcteixeira Date: Tue, 22 Feb 2022 17:24:10 +0100 Subject: [PATCH 2/7] added PR template + corrections --- .github/pull_request_template.md | 21 +++++++++++++++++++++ CONTRIBUTING.md | 12 +++++++++++- 2 files changed, 32 insertions(+), 1 deletion(-) create mode 100644 .github/pull_request_template.md diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 000000000..5a65bc344 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,21 @@ +# Pull Request template + +You are about to submit a new Pull Request. Before continuing make sure +you read the [contributing guidelines](CONTRIBUTING.md) and you comply +with the following criteria: + +- [ ] your code is well documented +- [ ] you wrote tests for the new code +- [ ] `tox` tests pass. Run `tox` command inside the repository folder. +- [ ] `-test.cfg` examples execute without faults. Inside `examples/` + run `python run_tests.py -b` +- [ ] PR contains only Python code +- [ ] PR follows our coding style +- [ ] you avoided structuring the code into classes as much as possible. + Use small functions instead. +- [ ] PR does not contain any additional dependencies unless permission + granted by the HADDOCK team +- [ ] PR does not break licensing +- [ ] you get extra bonus if you write tests for already existing code + :godmode: + diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index afa466a18..e129966ef 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -41,7 +41,8 @@ To contribute to the HADDOCK3's Python shell, follow these steps: 1. `tox -e py39` runs tests in Python 3.9 environment. 2. `tox -e lint` shows you errors in the code style. 3. `tox -e build` simulates building the HADDOCK3 package. - 4. If you want to submit high-quality code, you may wish to run + 4. Run the above altogether with the simple `tox` command + 5. If you want to submit high-quality code, you may wish to run `tox -e radon` to assess your code cyclomatic complexity. 5. You can work on these `tox` tests until they all pass green before @@ -50,11 +51,17 @@ submitting your PR. (should) to ensure the integrity of the python shell as a whole: 1. Navigate to any of the examples folder and run the `-test.cfg` file (see [USAGE](USAGE.md)). + 2. `*-test.cfg` runs are meant for testing purposes only. The + `examples/` folder also contain `*-full.cfg` files for production runs. + You don't need to run these for testin. 2. If you have a powerful computer and want to run all tests in a row, navigate to the `examples` folder and run `python run_tests.py -b`. The `-b` flag ensures the run will stop asap an error is found. 3. if your computer is not powerful enough, you can ask us to run those tests once you submitted the pull request. + 4. For example, the `docking-protein-protein-test.cfg` run in less + than 3 minutes on a `Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz` laptop + using 7 cores. 7. Add a list of your new additions to the `CHANGELOG.md` file by adding a new sub-header as described bellow. This is mandatory for `tox @@ -128,6 +135,8 @@ developing new functionalities. How? 1. Use the Python standard library as much as possible. 1. Numpy is always allowed. +1. If you need to implement heavy calculations, it is best you use +[Numba][numba] instead of C libraries. Talk with us before. 1. You need a small function from a large library. Try to reimplement it yourself with the Python standard library. 1. If you can't, talk first with @@ -277,3 +286,4 @@ described above for *code contributions*. [flake]: https://flake8.pycqa.org/en/latest/ "flake8" [fr]: https://github.com/haddocking/haddock3/blob/b44304f0a1509d44d9689d7e2e4124f0ae3af0a1/tox.ini#L116-L130 [numpydoc]: https://numpydoc.readthedocs.io/en/latest/format.html +[numba]: https://numba.pydata.org/ "Numba" From fa99b1807a19de7d3a6017784d5f476aa54711e5 Mon Sep 17 00:00:00 2001 From: joaomcteixeira Date: Tue, 22 Feb 2022 17:42:14 +0100 Subject: [PATCH 3/7] add elapsed total time --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e129966ef..9bb6a41a6 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -61,7 +61,7 @@ submitting your PR. those tests once you submitted the pull request. 4. For example, the `docking-protein-protein-test.cfg` run in less than 3 minutes on a `Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz` laptop - using 7 cores. + using 7 cores. All `-test.cfg` together take about 45 minutes. 7. Add a list of your new additions to the `CHANGELOG.md` file by adding a new sub-header as described bellow. This is mandatory for `tox From 7e42d57679b9e25c5067bae435adec6eeca07ef1 Mon Sep 17 00:00:00 2001 From: rodrigo Date: Tue, 22 Feb 2022 18:07:44 +0100 Subject: [PATCH 4/7] Update pull_request_template.md --- .github/pull_request_template.md | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 5a65bc344..2b5a51666 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -4,18 +4,14 @@ You are about to submit a new Pull Request. Before continuing make sure you read the [contributing guidelines](CONTRIBUTING.md) and you comply with the following criteria: -- [ ] your code is well documented -- [ ] you wrote tests for the new code +- [ ] Your code is well documented +- [ ] You wrote tests for the new code - [ ] `tox` tests pass. Run `tox` command inside the repository folder. -- [ ] `-test.cfg` examples execute without faults. Inside `examples/` - run `python run_tests.py -b` +- [ ] `-test.cfg` examples execute without errors. Inside `examples/` run `python run_tests.py -b` - [ ] PR contains only Python code - [ ] PR follows our coding style -- [ ] you avoided structuring the code into classes as much as possible. - Use small functions instead. -- [ ] PR does not contain any additional dependencies unless permission - granted by the HADDOCK team +- [ ] You avoided structuring the code into classes as much as possible, using small functions instead. +- [ ] PR does not add any dependencies unless permission was granted by the HADDOCK team - [ ] PR does not break licensing -- [ ] you get extra bonus if you write tests for already existing code - :godmode: +- [ ] You get extra bonus if you write tests for already existing code :godmode: From e12b9bcfaa3310a351fc7737ab3c9f271a256e16 Mon Sep 17 00:00:00 2001 From: rodrigo Date: Wed, 23 Feb 2022 11:35:41 +0100 Subject: [PATCH 5/7] add motivation for the PR checklist --- CONTRIBUTING.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9bb6a41a6..d19903804 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -47,7 +47,12 @@ To contribute to the HADDOCK3's Python shell, follow these steps: 5. You can work on these `tox` tests until they all pass green before submitting your PR. -6. We also have an `examples` folder with test cases that you can run + +6. Check if your contribution fulfills the requeriments proposed in the PR template, +these are based on the experience of our developer @joaomcteixeira and are in place +to ensure the long-term survival of the codebase. + +7. We also have an `examples` folder with test cases that you can run (should) to ensure the integrity of the python shell as a whole: 1. Navigate to any of the examples folder and run the `-test.cfg` file (see [USAGE](USAGE.md)). @@ -63,7 +68,7 @@ submitting your PR. than 3 minutes on a `Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz` laptop using 7 cores. All `-test.cfg` together take about 45 minutes. -7. Add a list of your new additions to the `CHANGELOG.md` file by +8. Add a list of your new additions to the `CHANGELOG.md` file by adding a new sub-header as described bellow. This is mandatory for `tox -e build` to pass. Note this applies only after we have released the stable `3.0.0` version. From 45d89f6397054157bab92ce1da33a28c7a1e5fe1 Mon Sep 17 00:00:00 2001 From: joaomcteixeira Date: Wed, 23 Feb 2022 12:47:31 +0100 Subject: [PATCH 6/7] improvements --- .github/pull_request_template.md | 20 +++++++++++++------- CONTRIBUTING.md | 12 ++++++------ 2 files changed, 19 insertions(+), 13 deletions(-) diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 2b5a51666..fcedc3209 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -4,14 +4,20 @@ You are about to submit a new Pull Request. Before continuing make sure you read the [contributing guidelines](CONTRIBUTING.md) and you comply with the following criteria: -- [ ] Your code is well documented -- [ ] You wrote tests for the new code +- [ ] Your code is well documented. Functions and classes have proper docstrings + as explained in contributing guidelines. You wrote explanatory comments for + those tricky parts. +- [ ] You wrote tests for the new code. - [ ] `tox` tests pass. Run `tox` command inside the repository folder. -- [ ] `-test.cfg` examples execute without errors. Inside `examples/` run `python run_tests.py -b` -- [ ] PR contains only Python code -- [ ] PR follows our coding style -- [ ] You avoided structuring the code into classes as much as possible, using small functions instead. -- [ ] PR does not add any dependencies unless permission was granted by the HADDOCK team +- [ ] `-test.cfg` examples execute without errors. Inside `examples/` run + `python run_tests.py -b` +- [ ] You have stick to Python. Talk with us before if you really need to add + other programming languages to HADDOCK3 +- [ ] code follows our coding style +- [ ] You avoided structuring the code into classes as much as possible, using + small functions instead. But, you can use classes if there's a purpose. +- [ ] PR does not add any *install dependencies* unless permission was granted + by the HADDOCK team - [ ] PR does not break licensing - [ ] You get extra bonus if you write tests for already existing code :godmode: diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d19903804..24719e272 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -48,9 +48,9 @@ To contribute to the HADDOCK3's Python shell, follow these steps: 5. You can work on these `tox` tests until they all pass green before submitting your PR. -6. Check if your contribution fulfills the requeriments proposed in the PR template, -these are based on the experience of our developer @joaomcteixeira and are in place -to ensure the long-term survival of the codebase. +6. Check if your contribution fulfills the requeriments proposed in the +PR template, these are based on our experience to ensure the long-term +survival of the codebase. Suggestions are always welcomed. 7. We also have an `examples` folder with test cases that you can run (should) to ensure the integrity of the python shell as a whole: @@ -154,10 +154,10 @@ and comply with license requirements. alternative. 1. Your new module largely depends on a library and reimplementing or copying is not an option. Then, consider if we can use that dependency -as a runtime dependency (like `gdock` or `lightdock`) instead of an +as a **runtime dependency** (like `gdock` or `lightdock`) instead of an installation dependency. -1. Nothing of the above is possible. You **really** need an install -dependency. Talk with us before. +1. Nothing of the above is possible. You **really** need an *install +dependency*. Talk with us before. ### 1.4 Code style From 654c0d5f8fb88e44a2fd8fd9ca69b43b6246435e Mon Sep 17 00:00:00 2001 From: Rodrigo V Honorato Date: Wed, 23 Feb 2022 15:37:46 +0100 Subject: [PATCH 7/7] Update CONTRIBUTING.md --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 24719e272..e43549ebf 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -49,7 +49,7 @@ To contribute to the HADDOCK3's Python shell, follow these steps: submitting your PR. 6. Check if your contribution fulfills the requeriments proposed in the -PR template, these are based on our experience to ensure the long-term +PR template, these are based on the experience of some developers to ensure the long-term survival of the codebase. Suggestions are always welcomed. 7. We also have an `examples` folder with test cases that you can run