Merge pull request #294 from JoostJM/docs-add-contributing

Add contributing to documentation

CONTRIBUTING.rst

@@ -0,0 +1,114 @@
+.. _radiomics-contributing-label:
+
+===========================
+Contributing to pyradiomics
+===========================
+
+There are many ways to contribute to pyradiomics, with varying levels of effort.  Do try to
+look through the documentation first if something is unclear, and let us know how we can
+do better.
+
+* Ask a question on the `pyradiomics email list <https://groups.google.com/forum/#!forum/pyradiomics)>`_
+* Submit a parameter file you used for your extraction
+* Submit a feature request or bug, or add to the discussion on the `pyradiomics issue tracker <https://github.com/Radiomics/pyradiomics/issues)>`_
+* Submit a `Pull Request <https://github.com/Radiomics/pyradiomics/pulls>`_ to improve pyradiomics or its documentation
+
+We encourage a range of Pull Requests, from patches that include passing tests and
+documentation, all the way down to half-baked ideas that launch discussions.
+
+----------------------------------------------
+The PR Process, Circle CI, and Related Gotchas
+----------------------------------------------
+
+How to submit a PR ?
+####################
+
+If you are new to pyradiomics development and you don't have push access to the pyradiomics
+repository, here are the steps:
+
+1. `Fork and clone <https://help.github.com/articles/fork-a-repo/>`_ the repository.
+2. Create a branch.
+3. `Push <https://help.github.com/articles/pushing-to-a-remote/>`_ the branch to your GitHub fork.
+4. Create a `Pull Request <https://github.com/Radiomics/pyradiomics/pulls>`_.
+
+This corresponds to the *Fork & Pull Model* mentioned in the `GitHub flow <https://guides.github.com/introduction/flow/index.html>`_
+guides.
+
+If you have push access to pyradiomics repository, you could simply push your branch
+into the main repository and create a `Pull Request <https://github.com/Radiomics/pyradiomics/pulls>`_. This corresponds to the
+*Shared Repository Model* and will facilitate other developers to checkout your
+topic without having to `configure a remote <https://help.github.com/articles/configuring-a-remote-for-a-fork/>`_.
+It will also simplify the workflow when you are *co-developing* a branch.
+
+When submitting a PR, make sure to add a ``cc: @Radiomics/developers`` comment to notify pyradiomics
+developers of your awesome contributions. Based on the
+comments posted by the reviewers, you may have to revisit your patches.
+
+How to integrate a PR ?
+#######################
+
+Getting your contributions integrated is relatively straightforward, here
+is the checklist:
+
+* Your changes include an update of the documentation if necessary
+
+  * Documentation on modules, classes and functions is contained in the respective docstrings
+  * More global documentation is contained in the ``docs`` folder.
+  * New modules need to be added to the auto-generated documentation. See
+    `here <http://pyradiomics.readthedocs.io/en/latest/developers.html#documentation>`_ for more
+    information on adding new modules to the documentation.
+
+* Your changes are added in the `changelog <https://github.com/Radiomics/pyradiomics/tree/master/CHANGES.rst>`_ in the *Next Release*  section.
+* All tests pass
+* Consensus is reached. This usually means that at least one reviewer reviewed and approved your
+  changes or added a ``LGTM`` comment, which is an acronym for *Looks Good to Me*.
+
+Next, there are two scenarios:
+
+* You do NOT have push access: A pyradiomics core developer will integrate your PR.
+* You have push access: Simply click on the "Merge pull request" button.
+
+Then, click on the "Delete branch" button that appears afterward.
+
+Automatic testing of pull requests
+##################################
+
+Every pull request is tested automatically using CircleCI, TravisCI and AppVeyor each time you push
+a commit to it. The Github UI will restrict users from merging pull requests until
+the builds have returned with a successful result indicating that all tests have
+passed and there were no problems detected by the linter. These tests include the following
+
+* flake8 to check adherence to the code style. See ``.flake8`` and ``.editorconfig`` for styles,
+  exceptions to the PEP8 style, etc.
+* If a feature class has a function ``_calculateCMatrix()``, identifying it as a C enhanced class,
+  output from the C extension is compared to the output from full python calculation. A absolute
+  difference of 1e-3 is allowed to account for machine precision errors.
+* All implemented features and feature classes have docstrings at the class level and feature
+  definition level.
+* A baseline is available for all features extracted from the 5 included test cases and
+  calculated features match this baseline to within 3% (allowing for machine precision errors)
+
+.. _radiomics-submit-parameter-file-label:
+
+---------------------------
+Submitting a parameter file
+---------------------------
+
+Different inputs into PyRadiomics require different settings. We encourage users to share their parameter file to help
+others extract features using the best settings for their use case.
+
+How to submit your parameter file?
+##################################
+
+Parameter files are stored in the repository under ``examples/exampleSettings``. If you wish to submit your parameters
+to the community, you can add your file here via a pull request (see above for details on making PRs).
+To help you along, here is a small checklist:
+
+* The filename should at least contain the modality (e.g. "MR") for which it is intended, and optionally the body part
+  (e.g. "prostate").
+* Ensure the file has the correct extension (either ".yml" or ".yaml")
+* Using comments in the parameter file, briefly explain your use case.
+
+After you've opened a PR to submit your parameter file, it will be checked for validity by the automatic testing. You
+don't have to specify your file anywhere, as parameter files are detected automatically in the ``exampleSettings``
+folder. If you want, you can also check your file manually using ``bin/testParams.py``.

README.md

@@ -131,8 +131,8 @@ This package is covered by the open source [3-clause BSD License](LICENSE.txt).
 
 We are happy to help you with any questions. Please contact us on the [pyradiomics email list](https://groups.google.com/forum/#!forum/pyradiomics).
 
-We welcome contributions to PyRadiomics. Please read the [contributing guidelines](CONTRIBUTING.md) on how to contribute
-to PyRadiomics.
+We welcome contributions to PyRadiomics. Please read the [contributing guidelines](CONTRIBUTING.rst) on how to
+contribute to PyRadiomics.
 
 **This work was supported in part by the US National Cancer Institute grant 
 5U24CA194354, QUANTITATIVE RADIOMICS SYSTEM DECODING THE TUMOR PHENOTYPE.**

bin/testParams.py

@@ -0,0 +1,27 @@
+# This is a simple script to check if your parameter file is valid. Run it with the location of your parameter
+# file as a command line argument (i.e. 'python testParams.py PATH/TO/PARAMFILE'). If successful, a message displaying
+# custom parameters specified will be printed. If validation fails, an error message specifying cause of validation
+# error will be printed.
+
+import sys
+
+import pykwalify.core
+
+from radiomics import getParameterValidationFiles
+
+def main(paramsFile):
+  schemaFile, schemaFuncs = getParameterValidationFiles()
+
+  c = pykwalify.core.Core(source_file=paramsFile, schema_files=[schemaFile], extensions=[schemaFuncs])
+  try:
+    params = c.validate()
+    print('Parameter validation successfull!\n\n'
+          '###Enabled Features###\n%s\n'
+          '###Enabled Image Types###\n%s\n'
+          '###Settings###\n%s' % (params['featureClass'], params['imageType'], params['setting']))
+  except Exception as e:
+    print('Parameter validation failed!\n%s' % e.message)
+
+
+if __name__ == '__main__' and len(sys.argv) > 1:
+    main(sys.argv[1])

docs/contributing.rst

@@ -0,0 +1 @@
+.. include:: ../CONTRIBUTING.rst

docs/customization.rst

@@ -262,7 +262,9 @@ optional argument (``--param``) when running pyradiomics from the command line.
 during initialization of the :ref:`feature extractor <radiomics-featureextractor-label>`, or using
 :py:func:`~radiomics.featureextractor.RadiomicsFeaturesExtractor.loadParams` after initialization. This removes the need
 to hard code a customized extraction in a python script through use of functions described above. Additionally, this
-also makes it more easy to share settings for customized extractions.
+also makes it more easy to share settings for customized extractions. We encourage users to share their parameter files
+in the PyRadiomics repository. See :ref:`radiomics-submit-parameter-file-label` for more information on how to submit
+your parameter file.
 
 .. note::
     Examples of the parameter file are provided in the ``pyradiomics/examples/exampleSettings`` folder.

docs/developers.rst

@@ -8,8 +8,7 @@ This section contains information on how to add or customize the feature classes
 PyRadiomics enumerates the available feature classes and input image types at initialization of the toolbox. These are
 available from the global ``radiomics`` namespace by use of the functions :py:func:`~radiomics.getFeatureClasses()` and
 :py:func:`~radiomics.getInputImageTypes()`, respectively. Individual features in a feature class are enumerated at
-initialization of the class. See also the
-`contributing guidelines <https://github.com/Radiomics/pyradiomics/blob/master/CONTRIBUTING.md>`_
+initialization of the class. See also the :ref:`contributing guidelines <radiomics-contributing-label>`.
 
 .. _radiomics-developers-featureclass:
 

docs/index.rst

@@ -41,6 +41,7 @@ Table of Contents
    customization
    radiomics
    features
+   contributing
    developers
    FAQs <faq>
    changes
@@ -148,6 +149,5 @@ Contact
 We are happy to help you with any questions. Please contact us on the `pyradiomics email list <https://groups.google.com/forum/#!forum/pyradiomics>`_.
 
 We'd welcome your contributions to PyRadiomics. Please read the
-`contributing guidelines <https://github.com/Radiomics/pyradiomics/blob/master/CONTRIBUTING.md>`_ on how to contribute
-to PyRadiomics. Information on adding / customizing feature classes and filters can be found in the
-:ref:`radiomics-developers` section.
+:ref:`contributing guidelines <radiomics-contributing-label>` on how to contribute to PyRadiomics. Information on
+adding / customizing feature classes and filters can be found in the :ref:`radiomics-developers` section.

radiomics/__init__.py

@@ -223,6 +223,19 @@ def getTestCase(testCase, repoDirectory=None):
   return imageFile, maskFile
 
 
+def getParameterValidationFiles():
+  """
+  Returns file locations for the parameter schema and custom validation functions, which are needed when validating
+  a parameter file using ``PyKwalify.core``.
+  This functions returns a tuple with the file location of the schema as first and python script with custom validation
+  functions as second element.
+  """
+  dataDir = os.path.abspath(os.path.join(os.path.dirname(__file__), 'schemas'))
+  schemaFile = os.path.join(dataDir, 'paramSchema.yaml')
+  schemaFuncs = os.path.join(dataDir, 'schemaFuncs.py')
+  return schemaFile, schemaFuncs
+
+
 class _DummyProgressReporter(object):
   """
   This class represents the dummy Progress reporter and is used for where progress reporting is implemented, but not

radiomics/featureextractor.py

@@ -11,7 +11,7 @@
 import six
 
 import radiomics
-from radiomics import generalinfo, getFeatureClasses, getImageTypes, imageoperations
+from radiomics import generalinfo, getFeatureClasses, getImageTypes, getParameterValidationFiles, imageoperations
 
 
 class RadiomicsFeaturesExtractor:
@@ -119,9 +119,7 @@ def loadParams(self, paramsFile):
     If supplied file does not match the requirements (i.e. unrecognized names or invalid values for a setting), a
     pykwalify error is raised.
     """
-    dataDir = os.path.abspath(os.path.join(os.path.dirname(__file__), 'schemas'))
-    schemaFile = os.path.join(dataDir, 'paramSchema.yaml')
-    schemaFuncs = os.path.join(dataDir, 'schemaFuncs.py')
+    schemaFile, schemaFuncs = getParameterValidationFiles()
     c = pykwalify.core.Core(source_file=paramsFile, schema_files=[schemaFile], extensions=[schemaFuncs])
     params = c.validate()
 

tests/test_exampleSettings.py

@@ -0,0 +1,33 @@
+# to run this test, from directory above:
+# setenv PYTHONPATH /path/to/pyradiomics/radiomics
+# nosetests --nocapture -v tests/test_exampleSettings.py
+
+import os
+
+from nose_parameterized import parameterized
+import pykwalify.core
+
+from radiomics import getParameterValidationFiles
+
+def exampleSettings_name_func(testcase_func, param_num, param):
+  return '%s_%s' % (testcase_func.__name__, os.path.splitext(os.path.basename(param.args[0]))[0])
+
+class TestExampleSettings:
+  def __init__(self):
+    self.schemaFile, self.schemaFuncs = getParameterValidationFiles()
+
+  def generateScenarios():
+    dataDir = os.path.join(os.path.dirname(os.path.abspath(__file__)), '..', 'examples', 'exampleSettings')
+    if os.path.isdir(dataDir):
+      settingsFiles = [fname for fname in os.listdir(dataDir) if fname.endswith('.yaml') or fname.endswith('.yml')]
+
+      for fname in settingsFiles:
+        yield os.path.join(dataDir, fname)
+
+  @parameterized.expand(generateScenarios(), testcase_func_name=exampleSettings_name_func)
+  def test_scenarios(self, settingsFile):
+
+    assert os.path.isfile(self.schemaFile)
+    assert os.path.isfile(self.schemaFuncs)
+    c = pykwalify.core.Core(source_file=settingsFile, schema_files=[self.schemaFile], extensions=[self.schemaFuncs])
+    c.validate()