Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
97e3ad1
Add street address search
mileswwatkins Aug 18, 2015
b7129b4
update join method for download URL
RoboDonut Sep 24, 2015
e6f6625
Merge branch 'master' of github.com:mileswwatkins/landsat-util into m…
mileswwatkins Oct 26, 2015
39422b4
Merge in production
mileswwatkins Oct 27, 2015
8c68be8
support bare json fixes #96
Oct 27, 2015
973abe2
add bare json to docs
Oct 27, 2015
a4b1dcc
Merge pull request #123 from developmentseed/json
Oct 27, 2015
c839788
Tweak precision on White House address test
mileswwatkins Oct 27, 2015
a6a32da
Merge in production
mileswwatkins Oct 27, 2015
260f74e
add test suits to the docker image
Oct 29, 2015
c8650d4
use docker for testing
Oct 29, 2015
5139fd6
add travis test note
Oct 29, 2015
c0ae5e8
remove python
Oct 29, 2015
50f1d20
Merge pull request #124 from developmentseed/faster_travis_tests
Oct 29, 2015
b1626b3
Merge branch 'miles/feature/address_search' of https://github.com/mil…
Oct 29, 2015
6a12014
add docker test command
Oct 29, 2015
6cf52c1
update todo list
Oct 29, 2015
ae22ea3
update recently added features
Oct 29, 2015
499ab1c
update link to docs [ci skip]
Oct 29, 2015
fad40e2
update commands [ci skip]
Oct 29, 2015
f4fa584
build docker image first
Oct 29, 2015
579a339
use special dockerfile for travis
Oct 29, 2015
0a936a6
always login to docker
Oct 29, 2015
2bfb4aa
push to docker after success
Oct 29, 2015
19369f3
add docker hub login info
Oct 29, 2015
e36e99d
Merge pull request #125 from developmentseed/mileswwatkins-miles/feat…
Oct 29, 2015
6bcdd08
Correct default for download
gmaclennan Nov 29, 2015
16f4d23
linting
Dec 1, 2015
fbd7458
apply ndvigrey to process and download
Dec 1, 2015
5af7dd3
don't apply colormap to ndvi fixes #127
Dec 1, 2015
d128944
add ndvigrey to download
Dec 1, 2015
59a6190
download 4, 5 if ndvigrey is used
Dec 1, 2015
6583794
adjust for WH coordinates change
Dec 1, 2015
bf1e35d
capture change in function call
Dec 1, 2015
18cbdba
push to pypi on both master and develop branches
Dec 1, 2015
1333af2
bump version to 0.10b1
Dec 1, 2015
49498a5
Merge pull request #134 from developmentseed/hotfix
Dec 1, 2015
62aaafd
Merge branch 'patch-1' of https://github.com/gmaclennan/landsat-util …
Dec 1, 2015
e1aace2
if process requested after download use 432
Dec 1, 2015
1e2fc64
test default download
Dec 1, 2015
7413e4a
update commands help file
Dec 1, 2015
d1efe8f
update overview guide
Dec 1, 2015
fb0ccc6
apply pansharpen and ndvi only if -p is used
Dec 1, 2015
2862d4c
test default bands are applied with -p flag
Dec 1, 2015
9186ec2
add geocoder
Dec 1, 2015
4cb9143
bump up version to 0.10b2
Dec 1, 2015
b44bfcf
Merge pull request #135 from developmentseed/default_download
Dec 1, 2015
9e4da58
Merge branch 'win_support' of https://github.com/RoboDonut/landsat-ut…
Dec 1, 2015
9ee59fd
a new url builder
Dec 2, 2015
15ac137
keep slashes
Dec 2, 2015
47051d0
use url_builder instead of join
Dec 2, 2015
d4dc657
install twine before travis deploy
Dec 2, 2015
d04f69b
bump to 0.10b3
Dec 2, 2015
449762a
Merge pull request #136 from developmentseed/update_join
Dec 2, 2015
91f7e70
install twine in the beginning
Dec 2, 2015
dff5a8a
use edge
Dec 2, 2015
7c289bc
install twine
Dec 2, 2015
2ef77ba
with user
Dec 2, 2015
5e90341
second attempt in fixing #114
Jan 5, 2016
18aee23
Merge pull request #139 from developmentseed/ndvi_hotfix
Jan 5, 2016
2e30fe5
update authors list
Jan 5, 2016
7035cc2
bump up to version 0.10.0
Jan 5, 2016
e0766d7
Merge pull request #140 from developmentseed/v10
Jan 5, 2016
1175020
include color maps in manifest
Jan 5, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 20 additions & 36 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,44 +1,27 @@
language: python
sudo: required

sudo: false

cache:
directories:
- ~/.cache/pip
services:
- docker

env:
global:
- PIP_WHEEL_DIR=$HOME/.cache/pip/wheels
- PIP_FIND_LINKS=file://$HOME/.cache/pip/wheels

virtualenv:
system_site_packages: true

addons:
apt:
packages:
- libgdal1h
- gdal-bin
- libgdal-dev
- libatlas-dev
- libatlas-base-dev
- gfortran
- python-numpy
- python-scipy

python:
- '2.7'

before_install:
- pip install -U pip
- pip install wheel
- secure: QsF7ignSAbH/WCyO6v9bw1exmCWDQR0DqmHkwJ5swc9N44OOOzbWGsaMSYB5y9h+d70fz4arbxQDhsk2KvX4Zd1/2YIMOrIsbgDYeegpkhVPgyQNPKmVqiX+Tb47t1C/TgkC7A07tiPpuefYcLNMZ8gzz7oKhh1UKapYftqzZ+g=
- secure: HxjeKWSROBQYy9NuNkgQeaK1ubTF8vH5FcR8nUTSAYxxw/qOzKpqkiq4BcJSRcIwTbkvaBf4MshLGVOxPjMeyJFe06UD/6LvTUGS3bwdya+m0RFjHe5/3wzS8/MxLbTlvgzmuGLLKOsJjXCi9eQQchKfHv+QuhGxhYVLQpnbU9E=
- secure: Zq0Z2UA2A7/ieXX8XoMweClJTp8hiVBxoQ1ylJYNd7qsRSk0QvZhn62db5/x48L9S1kELk0sG64q5Pf96/RPLpdjkBUAdEkS7qF+QOvRvAv2woNEHutjlMUvP6jwYGbug+AORg76btZ57OwMOi3aTkagQMMKnokfo7KGbffy0Jo=


install:
- "pip wheel -r requirements/dev.txt"
- "pip install -r requirements/dev.txt"
- pip install --user twine

before_script:
- docker build --file="travis-dockerfile" -t "developmentseed/landsat-util:travis" .

script:
- nosetests
- docker run --rm -it -v "$(pwd)":/test -w /test developmentseed/landsat-util:travis nosetests

after_success:
- docker login -e ${DOCKER_EMAIL} -u ${DOCKER_USER} -p ${DOCKER_PASSWORD}
- docker push developmentseed/landsat-util:travis

deploy:
provider: pypi
Expand All @@ -47,13 +30,14 @@ deploy:
secure: WtawFW/999XYszmZfj1Qk82l00OSyP2JBVOOGCERrW1gVO7MYtYsgP31HKRSzNTCTHJNVDpdK4WZWY6zPQqC3l2UfWYYsvRn0hCoI8AJxE5VCUEg6Ccpe6fMJuhp1pq6Zy7yrfBSZcOB9aqSHLBJsunD2o3mNlTC8WV8vNK74ck=
on:
repo: developmentseed/landsat-util
branch: master
branch:
- master
- develop

after_deploy:
if [ "$TRAVIS_BRANCH" == "master" ]; then
echo "Start Docker Hub Push"
VER=$(python -c "import landsat; print landsat.__version__")
VER=$(docker run --rm -it -v "$(pwd)":/test -w /test developmentseed/landsat-util:travis python landsat/landsat.py --version | sed s/[^0-9\.]//g)
docker build . -t developmentseed/landsat-util:$VER
docker login -e ${DOCKER_EMAIL} -u ${DOCKER_USER} -p ${DOCKER_PASSWORD}
docker push developmentseed/landsat-util:$VER
fi
2 changes: 0 additions & 2 deletions AUTHORS.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,5 @@ Scisco https://github.com/scisco
Marc Farra https://github.com/kamicut
Drew Bollinger https://github.com/drewbo
Alex Kappel https://twitter.com/alex_kappel
Eric Miller https://github.com/millerEric
Yuriy Czoli https://github.com/YKCzoli

See also https://github.com/developmentseed/landsat-util/graphs/contributors.
13 changes: 13 additions & 0 deletions CHANGES.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,19 @@
Changes
=======

0.10.0 (2016-01-05)
------------------
- add support for bare json output
- faster travis tests
- add street search
- apply ndvigrey to process and download
- fixes #127
- update commands help file
- apply pansharpen and ndvi only if -p is used
- download zip if bands are not specified
- better handle url joins to a fix error on Windows
- other small bug fixes

0.9.1 (2015-10-26)
------------------
- Add missing package (polyline) to setup.py
Expand Down
1 change: 0 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,3 @@ ADD landsat /usr/local/lib/python2.7/dist-packages/landsat
ADD bin/landsat /usr/local/bin/
ADD . /landsat
RUN cd /landsat && pip install -r requirements/docker.txt

2 changes: 1 addition & 1 deletion MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ recursive-exclude * __pycache__
recursive-exclude * *.py[co]

recursive-include docs *.rst conf.py Makefile make.bat
recursive-include landsat/tests/samples *.tar.bz2
recursive-include landsat/maps *.txt
20 changes: 16 additions & 4 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,20 +21,32 @@ Landsat-util is a command line utility that makes it easy to search, download, a
Docs
+++++

For full documentation visit: http://landsat-util.readthedocs.org/
For full documentation visit: https://pythonhosted.org/landsat-util/

To run the documentation locally::

$ pip install -r requirements/dev.txt
$ cd docs
$ make html

Travis Tests
++++++++++++

To speed up testing on travis, we use a docker image.

To test with docker image locally run:

.. code::
$ docker run --rm -it -v "$(pwd)":/test developmentseed/landsat-util:travis nosetests
Recently Added Features
+++++++++++++++++++++++

- Add longitude latitude search
- Improve console output
- Add more color options such as false color, true color, etc.
- Improved pansharpening
- Use BQA bands for cloud/snow coverage and use in color correction
- Add support for different NDVI color maps (three included)
- Add support for image clipping using the new `--clip` flag

Change Log
++++++++++
Expand Down
6 changes: 6 additions & 0 deletions docs/commands.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ Commands

--lon Longitude

--address Street address

-l LIMIT, --limit LIMIT
Search return results limit default is 10

Expand All @@ -35,6 +37,8 @@ Commands
-c CLOUD, --cloud CLOUD
Maximum cloud percentage. Default: 20 perct

--json Returns a bare JSON response

-h, --help Show this help message and exit

Download:
Expand All @@ -60,6 +64,8 @@ Commands

--ndvi Calculates NDVI and produce a RGB GTiff with seperate colorbar.

--ndvigrey Calculates NDVI and produce a greyscale GTiff.

--clip Clip the image with the bounding box provided. Values must be in WGS84 datum,
and with longitude and latitude units of decimal degrees separated by comma.
Example: --clip -346.06658935546875,49.93531194616915,-345.4595947265625,50.2682767372753
Expand Down
5 changes: 4 additions & 1 deletion docs/overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,9 @@ Search by latitude and longitude::

$: landsat search --lat 38.9004204 --lon -77.0237117

Search by latitude and longitude with pure json output::

$: landsat search --lat 38.9004204 --lon -77.0237117 --json

Download
++++++++
Expand All @@ -50,7 +53,7 @@ Download images by their custom sceneID, which you get from landsat search::

$: landsat download LC80090452014008LGN00

Download only band 4, 3 and 2 for a particular sceneID::
By default landsat-util downloads the full zip file from Google Storage unless you specify the bands or run an image processing right after download. For example to download only band 4, 3 and 2 for a particular sceneID run::

$: landsat download LC80090452014008LGN00 --bands 432

Expand Down
3 changes: 0 additions & 3 deletions docs/todo.rst
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
To Do List
++++++++++

- Add Sphinx Documentation
- Add capacity for NDVI output
- Add alternative projections (currently only option is default web-mercator; EPSG: 3857)
- Connect search to Google Address API
- Include 16-bit image variant in output
- Add support for color correct looping over multiple compressed inputs (currently just 1)
2 changes: 1 addition & 1 deletion landsat/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = '0.9.1'
__version__ = '0.10.0'
8 changes: 4 additions & 4 deletions landsat/downloader.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
from homura import download as fetch
import requests

from utils import check_create_folder
from utils import check_create_folder, url_builder
from mixins import VerbosityMixin
import settings

Expand Down Expand Up @@ -192,7 +192,7 @@ def google_storage_url(self, sat):
(String) The URL to a google storage file
"""
filename = sat['scene'] + '.tar.bz'
return join(self.google, sat['sat'], sat['path'], sat['row'], filename)
return url_builder([self.google, sat['sat'], sat['path'], sat['row'], filename])

def amazon_s3_url(self, sat, filename):
"""
Expand All @@ -210,7 +210,7 @@ def amazon_s3_url(self, sat, filename):
:returns:
(String) The URL to a S3 file
"""
return join(self.s3, sat['sat'], sat['path'], sat['row'], sat['scene'], filename)
return url_builder([self.s3, sat['sat'], sat['path'], sat['row'], sat['scene'], filename])

def remote_file_exists(self, url):
""" Checks whether the remote file exists.
Expand Down Expand Up @@ -284,4 +284,4 @@ def scene_interpreter(self, scene):

d = Downloader()

# d.download(['LC81990242015046LGN00', 'LC80030172015001LGN00'])
# d.download(['LC81990242015046LGN00', 'LC80030172015001LGN00'])
Loading