Merge pull request #1 from sentinel-hub/develop

Develop

.travis.yml

@@ -1,4 +1,4 @@
-dist: xenial
+dist: bionic
 language: python
 
 notifications:
@@ -8,6 +8,7 @@ python:
 - "3.5"
 - "3.6"
 - "3.7"
+- "3.8"
 
 before_install:
 - sudo add-apt-repository -y ppa:jonathonf/backports
@@ -25,9 +26,11 @@ install:
   else
     python install_all.py;
   fi
-- pip install pandas==0.23.4  # This is temporal solution
 
 script:
+- sentinelhub.config --instance_id $INSTANCE_ID
+- sentinelhub.config --sh_client_secret $SH_CLIENT_SECRET
+- sentinelhub.config --sh_client_id $SH_CLIENT_ID
 - if [ $TRAVIS_PYTHON_VERSION == '3.6' ]; then
     pylint core/eolearn/core/*.py &&
     pylint coregistration/eolearn/coregistration/*.py &&
@@ -54,7 +57,7 @@ script:
       pytest;
     fi
   fi
-- 'if [ "$TRAVIS_PULL_REQUEST" = "false" ]; then pytest --nbval-lax examples/visualization/EOPatchVisualization.ipynb  ; fi'
+# - 'if [ "$TRAVIS_PULL_REQUEST" = "false" ]; then pytest --nbval-lax examples/visualization/EOPatchVisualization.ipynb  ; fi'
 
 after_success:
 - if [ $TRAVIS_PYTHON_VERSION == '3.6' ] && [ "$TRAVIS_PULL_REQUEST" = "false" ]; then codecov; fi

CONTRIBUTING.md

@@ -5,9 +5,9 @@ to the library is very much appreciated.
 
 Here is how you can contribute:
 
- * [Bug Reports](#bug-reports)
- * [Feature Requests](#feature-requests)
- * [Pull Requests](#pull-requests)
+* [Bug Reports](#bug-reports)
+* [Feature Requests](#feature-requests)
+* [Pull Requests](#pull-requests)
 
 All contributors agree to follow our [Code of Conduct][code-of-conduct].
 
@@ -30,41 +30,31 @@ When reporting a bug, please check [here][open-bug-list] whether
 the bug was already reported. If not, open an issue with the **bug** label and
 report the following information:
 
- * Issue description
- * How to reproduce the issue
- * Expected outcome
- * Actual outcome
- * OS and package versions
+* Issue description
+* How to reproduce the issue
+* Expected outcome
+* Actual outcome
+* OS and package versions
 
 This information helps us to reproduce, pinpoint, and fix the reported issue.
 
-If you are not sure whether the odd behaviour is a bug or a _feature_, best to open
-an issue and clarify.
+If you are not sure whether the odd behaviour is a bug or a _feature_, best to open an issue and clarify.
 
 [open-bug-list]: https://github.com/sentinel-hub/eo-learn/issues?q=state:open+type:issue+label:"bug"
 
 ## Feature Requests
 
 Existing feature requests can be found [here][existing-feature-requests].
 
-A new feature request can be created by opening a new issue with the **enhancement** label,
-and describing how the feature would benefit the **eo-learn** community.
-Providing an example use-case would help assessing the scope of the
-feature request.
+A new feature request can be created by opening a new issue with the **enhancement** label, and describing how the feature would benefit the **eo-learn** community. Providing an example use-case would help assessing the scope of the feature request.
 
 [existing-feature-requests]: https://github.com/sentinel-hub/eo-learn/issues?q=state:open+type:issue+label:"enhancement"
 
 ## Pull Requests
 
-The GitHub Pull Request (PR) mechanism is the best option to contribute code
-to the library. Users can fork the repository, make their contribution to their
-local fork and create a PR to add those changes to the codebase. GitHub provides excellent
-tutorials on how the [fork and pull][fork-and-pull] mechanism work, and on
-how to best [create a PR][create-pr].
+The GitHub Pull Request (PR) mechanism is the best option to contribute code to the library. Users can fork the repository, make their contribution to their local fork and create a PR to add those changes to the codebase. GitHub provides excellent tutorials on how the [fork and pull][fork-and-pull] mechanism work, and on how to best [create a PR][create-pr].
 
-Existing PRs can be found [here][existing-prs]. Before creating new PRs, you should check
-whether someone else has contributed a similar feature, and if so, you can add your
-input to the existing code review.
+Existing PRs can be found [here][existing-prs]. Before creating new PRs, you should check whether someone else has contributed a similar feature, and if so, you can add your input to the existing code review.
 
 The following guidelines should be observed when creating a PR.
 
@@ -74,38 +64,64 @@ The following guidelines should be observed when creating a PR.
 
 ### General guidelines
 
- * Where applicable, create your contribution in a new branch of your fork based on the
-   `develop` branch, as the `master` branch is aligned to the released package on [PyPI][pypi]. Upon
-   completion of the code review, the branch will be merged into `develop` and, at
-   the next package release, into `master`.
+* Where applicable, create your contribution in a new branch of your fork based on the `develop` branch, as the `master` branch is aligned to the released package on [PyPI][pypi]. Upon completion of the code review, the branch will be merged into `develop` and, at the next package release, into `master`.
 
- * Document your PR to help maintainers understand and review your contribution. The PR
-   should include:
+* Document your PR to help maintainers understand and review your contribution. The PR should include:
 
-   * Description of contribution;
-   * Required testing;
-   * Link to issue/feature request.
+  * Description of contribution;
+  * Required testing;
+  * Link to issue/feature request.
 
- * Your contribution should include unit tests, to test correct behaviour of the new feature
-   and to lower the maintenance effort. Bug fixes as well as new features should include unit tests.
-   When submitting the PR, check whether the Travis CI testing returns any errors, and if it does,
-   please try to fix the issues causing failure. A test `EOPatch` is made available [here][test-eo-patch]
-   with data for each `FeatureType`. Unit tests evaluating the correctness of new tasks should use data
-   available in this `EOPatch`. New fields useful for testing purposes can be added, but should
-   be consistent with the `bbox` and `timestamp` of the `EOPatch`.
+* Your contribution should include unit tests, to test correct behaviour of the new feature and to lower the maintenance effort. Bug fixes as well as new features should include unit tests. When submitting the PR, check whether the Travis CI testing returns any errors, and if it does, please try to fix the issues causing failure. A test `EOPatch` is made available [here][test-eo-patch] with data for each `FeatureType`. Unit tests evaluating the correctness of new tasks should use data available in this `EOPatch`. New fields useful for testing purposes can be added, but should be consistent with the `bbox` and `timestamp` of the `EOPatch`. To execute all unit tests locally on you machine, use the command: `pytest` from the main folder. See also the [examples/README.md](examples/README.md) for how to setup you SentinelHub account and local config for testing.
 
- * Try to keep contributions small, as this speeds up the reviewing process. In the case of large
-   contributions, e.g. a new complex `EOTask`, it's best to contact us first to review the scope
-   of the contribution.
+* Try to keep contributions small, as this speeds up the reviewing process. In the case of large contributions, e.g. a new complex `EOTask`, it's best to contact us first to review the scope of the contribution.
 
- * Keep API compatibility in mind, in particular when contributing a new `EOTask`. In general,
-   all new tasks should adhere to the modularity of **eo-learn**.
-   Check the Section below for more information on how to contribute an `EOTask`.
+* Keep API compatibility in mind, in particular when contributing a new `EOTask`. In general, all new tasks should adhere to the modularity of **eo-learn**. Check the Section below for more information on how to contribute an `EOTask`.
 
- * New features or tasks should be appropriately commented using Sphinx style docstrings. The documentation uses
-   the [PEP-8][pep-8] formatting guidelines. [Pylint][pylint] is used to check the coding standard.
-   Therefore please run `pylint` from the the main folder, which contains the `pylintrc` file, to make sure your
-   contribution is scored `10.0/10.0`.
+* New features or tasks should be appropriately commented using Sphinx style docstrings. The documentation uses the [PEP-8][pep-8] formatting guidelines. [Pylint][pylint] is used to check the coding standard. Therefore, please run `pylint */ *.py` from the the main folder, which contains the `pylintrc` file, to make sure your contribution is scored `10.0/10.0`.
+
+### Development environment
+
+* Get the latest development version by creating a fork and clone the repo:
+
+```bash
+git clone git@github.com:<username>/eo-learn.git
+```
+
+* Make sure that you have python 3 installed, i.e. version 3.5 or higher. See [Python downloads][python] for general Python versions or [Anaconda distributions][conda] for using Conda.
+
+* All **eo-learn** packages can be installed at once using `python install_all.py -e`. To install each package separately, run `pip install -e <package_folder>`. We strongly recommend initializing a virtual environment before installing the required packages. For example by using Python Package Index (PyPI) and virtualenv:
+
+```bash
+cd eo-learn
+# The following creates the virtual environment in the ".env" folder.
+virtualenv --python python3 .env
+source .env/bin/activate
+
+# The following installs all eo-learn subpackages and development packages
+# using PyPI in the activated virtualenv environment.
+python install_all.py -e
+pip install -r requirements-dev.txt -r requirements-docs.txt
+```
+
+Or alternatively by using Conda:
+
+```bash
+cd eo-learn
+# The following creates the virtual environment named "dev_eolearn" where the
+# Conda installation in located. ipykernel enables Jupyter (using
+# nb_conda_kernels) to select the environment as a kernel. ipywidgets enables
+# Jupyter to show the progress bar of EOExecutor without raising an error.
+conda create -n dev_eolearn python=3.7 ipykernel ipywidgets
+conda activate dev_eolearn
+
+# The following installs all eo-learn subpackages and development packages
+# using PyPI in the activated Conda environment.
+python install_all.py -e
+pip install -r requirements-dev.txt -r requirements-docs.txt
+```
+
+**Note:** to reduce later merge conflicts, always pull the latest version of the `develop` branch from the upstream eo-learn repository ([located here][dev-branch]) to your fork before starting the work on your PR.
 
 ### Contribute an `EOTask`
 
@@ -126,26 +142,21 @@ class FooTask(EOTask):
         # do what foo does on input eopatch and return it
         return eopatch
 ```
+
 When creating a new task, bear in mind the following:
 
- * Tasks should be as modular as possible, facilitating task re-use and sharing.
- * An `EOTask` should perform a well-defined operation on the input eopatch(es). If the operation
-   could be split into atomic sub-operations that could be used separately, then consider splitting
-   the task into multiple tasks. Similarly, if tasks share the bulk of the implementation but differ
-   in a minority of implementation, consider using Base classes and inheritance. The interpolation
-   tasks represent a good example of this.  
- * Tasks should be as generalizable as possible, therefore hard-coding of task parameters or `EOPatch`
-   feature types should be avoided. Use the `EOTask._parse_features` method to parse input features in a task,
-   and pass task parameters as arguments, either in the constructor, or at run-time.
- * If in doubt on whether a task is general enough to be of interest to the community, or you are not
-   sure to which sub-package to contribute your task to, send us an email or open a
-   [feature request](#feature-requests).
+* Tasks should be as modular as possible, facilitating task re-use and sharing.
+* An `EOTask` should perform a well-defined operation on the input eopatch(es). If the operation could be split into atomic sub-operations that could be used separately, then consider splitting the task into multiple tasks. Similarly, if tasks share the bulk of the implementation but differ in a minority of implementation, consider using Base classes and inheritance. The interpolation tasks represent a good example of this.  
+* Tasks should be as generalizable as possible, therefore hard-coding of task parameters or `EOPatch` feature types should be avoided. Use the `EOTask._parse_features` method to parse input features in a task, and pass task parameters as arguments, either in the constructor, or at run-time.
+* If in doubt on whether a task is general enough to be of interest to the community, or you are not sure to which sub-package to contribute your task to, send us an email or open a [feature request](#feature-requests).
 
 Looking forward to include your contributions into **eo-learn**.
 
-
 [pypi]: https://pypi.org/project/eo-learn/
 [pep-8]: https://www.python.org/dev/peps/pep-0008/
 [pylint]: https://www.pylint.org/
 [existing-eo-tasks]: https://eo-learn.readthedocs.io/en/latest/eotasks.html
 [test-eo-patch]: https://github.com/sentinel-hub/eo-learn/tree/master/example_data/TestEOPatch
+[python]: https://www.python.org/downloads/
+[conda]: https://www.anaconda.com/distribution/
+[dev-branch]: https://github.com/sentinel-hub/eo-learn/tree/develop/

README.md

@@ -99,6 +99,11 @@ If you would like to contribute to `eo-learn`, check out our [contribution guide
  * [Use eo-learn with AWS SageMaker](https://medium.com/@drewbo19/use-eo-learn-with-aws-sagemaker-9420856aafb5) (by Drew Bollinger)
  * [Spatio-Temporal Deep Learning: An Application to Land Cover Classification](https://www.researchgate.net/publication/333262625_Spatio-Temporal_Deep_Learning_An_Application_to_Land_Cover_Classification) (by Anze Zupanc)
  * [Tree Cover Prediction with Deep Learning](https://medium.com/dataseries/tree-cover-prediction-with-deep-learning-afeb0b663966) (by Daniel Moraite)
+ * [NoRSC19 Workshop on eo-learn](https://github.com/sentinel-hub/norsc19-eo-learn-workshop)
+ * [Tracking a rapidly changing planet](https://medium.com/@developmentseed/tracking-a-rapidly-changing-planet-bc02efe3545d) (by Development Seed)
+  * [Land Cover Monitoring System](https://medium.com/sentinel-hub/land-cover-monitoring-system-84406e3019ae) (by Jovan Visnjic and Matej Aleksandrov)
+
+
 
 ## Questions and Issues
 

core/eolearn/core/__init__.py

@@ -8,13 +8,14 @@
 from .eodata import EOPatch
 from .eotask import EOTask, CompositeTask
 from .eoworkflow import EOWorkflow, LinearWorkflow, Dependency, WorkflowResults
-from .eoexecution import EOExecutor
+from .eoexecution import EOExecutor, execute_with_mp_lock
 
-from .core_tasks import CopyTask, DeepCopyTask, SaveToDisk, LoadFromDisk, AddFeature, RemoveFeature, RenameFeature,\
+from .core_tasks import CopyTask, DeepCopyTask, SaveTask, LoadTask, AddFeature, RemoveFeature, RenameFeature,\
     DuplicateFeature, InitializeFeature, MoveFeature, MergeFeatureTask, MapFeatureTask, ZipFeatureTask,\
-    ExtractBandsTask, CreateEOPatchTask
+    ExtractBandsTask, CreateEOPatchTask, SaveToDisk, LoadFromDisk
 
+from .fs_utils import get_filesystem, load_s3_filesystem
 from .utilities import deep_eq, negate_mask, constant_pad, get_common_timestamps, bgr_to_rgb, FeatureParser
 
 
-__version__ = '0.6.0'
+__version__ = '0.7.0'

core/eolearn/core/constants.py

@@ -4,7 +4,7 @@
 Credits:
 Copyright (c) 2017-2019 Matej Aleksandrov, Matej Batič, Andrej Burja, Eva Erzin (Sinergise)
 Copyright (c) 2017-2019 Grega Milčinski, Matic Lubej, Devis Peresutti, Jernej Puc, Tomislav Slijepčević (Sinergise)
-Copyright (c) 2017-2019 Blaž Sovdat, Jovan Višnjić, Anže Zupanc, Lojze Žust (Sinergise)
+Copyright (c) 2017-2019 Blaž Sovdat, Nejc Vesel, Jovan Višnjić, Anže Zupanc, Lojze Žust (Sinergise)
 
 This source code is licensed under the MIT license found in the LICENSE
 file in the root directory of this source tree.
@@ -55,7 +55,7 @@ class FeatureType(Enum):
     @classmethod
     def has_value(cls, value):
         """True if value is in FeatureType values. False otherwise."""
-        return any(value == item.value for item in cls)
+        return value in cls._value2member_map_
 
     def is_spatial(self):
         """True if FeatureType has a spatial component. False otherwise."""