Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data source improvements #138

Merged
merged 28 commits into from
Sep 21, 2020
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
c915e84
refactored DataSource, updated code and tests
AleksMat Sep 2, 2020
e0572d5
removed a few OGC integration tests and added 1 more
AleksMat Sep 2, 2020
dfa6899
fixed a new pylint issue about exceptions
AleksMat Sep 2, 2020
5b9ddeb
In case of Python 3.6 install dataclasses package
AleksMat Sep 2, 2020
3984335
minor update
AleksMat Sep 3, 2020
9a4ba93
added minor fixes
AleksMat Sep 3, 2020
9fa31a1
improvements regarding data sources and processing API
AleksMat Sep 3, 2020
e2b542b
updated OGC examples and a DataSource.get_available_sources
AleksMat Sep 3, 2020
fead982
improvements of data source handling
AleksMat Sep 4, 2020
5e057cd
handled cases if data source name or definition has already been used
AleksMat Sep 4, 2020
653bb43
updated processing api notebook
AleksMat Sep 4, 2020
968a24e
Added notebook about data sources, updated documentation, minor impro…
AleksMat Sep 7, 2020
76ad92a
minor pylint fix
AleksMat Sep 7, 2020
c7abfd8
WFS url change according to data source
AleksMat Sep 7, 2020
2a3ba5e
a few typo fixes
AleksMat Sep 8, 2020
64a1a33
minor imporvements in data sources
AleksMat Sep 8, 2020
a56fcac
renamed data source to data collection - updated everything
AleksMat Sep 10, 2020
531e2a0
added swath mode
AleksMat Sep 10, 2020
9f357e4
added timeliness data source parameter
AleksMat Sep 10, 2020
dd67d2d
minor fix
AleksMat Sep 10, 2020
a77b317
updated notebooks regarding new data collection parameters
AleksMat Sep 10, 2020
448efbc
better handling of data_collection / data_source parameters
AleksMat Sep 17, 2020
4af4ec7
commented out a problematic geopedia test
AleksMat Sep 17, 2020
188e3ee
minor import fix in ogc examples
AleksMat Sep 17, 2020
8eb1048
moved handle_deprecated_data_source to data_collections
AleksMat Sep 17, 2020
85b08d5
InputDataDict instead of base_url parameter in SentinelHubRequest
AleksMat Sep 17, 2020
8fc0859
marked geopedia wms stats test with xfail
AleksMat Sep 18, 2020
bb72154
Merge pull request #143 from sentinel-hub/feat/sh-request-base-url-ha…
AleksMat Sep 18, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/source/data_sources.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
data_sources
============

.. automodule:: sentinelhub.data_sources
:members:
:show-inheritance:
1 change: 1 addition & 0 deletions docs/source/docs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Modules
config
constants
data_request
data_sources
download.aws_client
download.client
download.request
Expand Down
1 change: 1 addition & 0 deletions docs/source/examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Examples
:maxdepth: 4

examples/processing_api_request.ipynb
examples/data_sources.ipynb
examples/ogc_request.ipynb
examples/large_area_utilities.ipynb
examples/fis_request.ipynb
Expand Down
669 changes: 669 additions & 0 deletions examples/data_sources.ipynb

Large diffs are not rendered by default.

141 changes: 73 additions & 68 deletions examples/ogc_request.ipynb

Large diffs are not rendered by default.

124 changes: 10 additions & 114 deletions examples/processing_api_request.ipynb

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion sentinelhub/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@
from .geopedia import GeopediaFeatureIterator, GeopediaSession

from .geometry import BBox, Geometry, BBoxCollection
from .constants import DataSource, CustomUrlParam, CRS, MimeType, SHConstants, AwsConstants, ServiceType, \
from .constants import CustomUrlParam, CRS, MimeType, SHConstants, AwsConstants, ServiceType, \
HistogramType
from .data_sources import DataSource

from .config import SHConfig

Expand Down
5 changes: 3 additions & 2 deletions sentinelhub/areas.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@
from shapely.geometry import Polygon, MultiPolygon, GeometryCollection

from .config import SHConfig
from .constants import CRS, DataSource
from .constants import CRS
from .data_sources import DataSource
from .geometry import BBox, BBoxCollection, BaseGeometry, Geometry
from .geo_utils import transform_point
from .ogc import WebFeatureService
Expand Down Expand Up @@ -358,7 +359,7 @@ class TileSplitter(AreaSplitter):
as `(n, n)`.
:type split_shape: int or (int, int)
:param data_source: Source of requested satellite data. Default is Sentinel-2 L1C data.
:type data_source: sentinelhub.constants.DataSource
:type data_source: DataSource
:param config: A custom instance of config class to override parameters from the saved configuration.
:type config: SHConfig or None
:param reduce_bbox_sizes: If `True` it will reduce the sizes of bounding boxes so that they will tightly fit the
Expand Down
9 changes: 5 additions & 4 deletions sentinelhub/aws.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@
from abc import ABC, abstractmethod

from .config import SHConfig
from .constants import AwsConstants, EsaSafeType, MimeType, DataSource
from .constants import AwsConstants, EsaSafeType, MimeType
from .data_sources import DataSource
from .download import DownloadRequest
from .download.aws_client import get_aws_json
from .exceptions import AwsDownloadFailedException, SHUserWarning
Expand Down Expand Up @@ -467,7 +468,7 @@ class will try to find the index automatically. If there will be multiple choice
:type aws_index: int or None
:param data_source: Source of requested AWS data. Supported sources are Sentinel-2 L1C and Sentinel-2 L2A,
default is Sentinel-2 L1C data.
:type data_source: constants.DataSource
:type data_source: DataSource
:param parent_folder: folder where the fetched data will be saved.
:type parent_folder: str
:param bands: List of Sentinel-2 bands for request. If parameter is set to `None` all bands will be used.
Expand Down Expand Up @@ -527,8 +528,8 @@ def parse_datetime(time):
"""
try:
return parse_time(time)
except Exception:
raise ValueError('Time must be in format YYYY-MM-DD or YYYY-MM-DDTHH:MM:SS')
except Exception as exception:
raise ValueError('Time must be in format YYYY-MM-DD or YYYY-MM-DDTHH:MM:SS') from exception

def get_requests(self):
"""
Expand Down
3 changes: 2 additions & 1 deletion sentinelhub/aws_safe.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@
import warnings

from .aws import AwsProduct, AwsTile
from .constants import AwsConstants, EsaSafeType, MimeType, DataSource
from .constants import AwsConstants, EsaSafeType, MimeType
from .data_sources import DataSource
from .download.aws_client import get_aws_xml
from .exceptions import SHRuntimeWarning

Expand Down
2 changes: 1 addition & 1 deletion sentinelhub/commands.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
import click

from .config import SHConfig
from .constants import DataSource
from .data_sources import DataSource
from .data_request import get_safe_format, download_safe_format
from .download import DownloadRequest, DownloadClient

Expand Down
4 changes: 2 additions & 2 deletions sentinelhub/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -272,8 +272,8 @@ def get_sh_ogc_url(self):
:return: An URL endpoint
:rtype: str
"""
ogc_enpoint = 'v1' if self.has_eocloud_url() else 'ogc'
return '{}/{}'.format(self.sh_base_url, ogc_enpoint)
ogc_endpoint = 'v1' if self.has_eocloud_url() else 'ogc'
return '{}/{}'.format(self.sh_base_url, ogc_endpoint)

def get_sh_rate_limit_url(self):
""" Provides URL for Sentinel Hub rate limiting endpoint
Expand Down