Skip to content

[WIP] update images vignette #171

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Chris-Schnaufer merged 1 commit into from
Jun 26, 2019
Merged

[WIP] update images vignette #171

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 18 additions & 72 deletions vignettes/03-get-images-python.Rmd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,26 @@ All the `terrautils` functions are now available in Python, although we will onl
In this section we retrieve the names of different sensor types that are available.
This will allow you to understand what files may be avaialable other than just those containing RBG image data.

In order to run Python functions, including those from the `terrautils` library, within this Rmarkdown, we have to install and set up `reticulate`.

```{r get_pkgs, message=FALSE}
if(!require(reticulate)){
install.packages("reticulate")
reticulate::py_install("terrautils")
}

library(reticulate)
use_virtualenv("r-reticulate")
```

We will first be using the `get_sensor_list` function to retrieve all the data on available sensors.
We will then use the `unique_sensor_names` function to extract only the sensor names from the data we just retrieved.

```{python py_get_sensor_data, eval=FALSE}
```{python py_get_functions}
from terrautils.products import get_sensor_list, unique_sensor_names
```

```{python py_get_sensor_data}
url = 'https://terraref.ncsa.illinois.edu/clowder/'
key = ''

Expand Down Expand Up @@ -61,7 +75,7 @@ If the `stream=True` parameter was omitted the file's entire contents would be i

To illustrate how this might work we are going to pre-populated an array of file names and their associated Clowder IDs.

```{python py_sample_images_data, eval=FALSE}
```{python py_sample_images_data}
files = [ {"id": "5c507cb74f0c4b0cbe6705f2",
"filename": "rgb_geotiff_L1_ua-mac_2018-06-02__14-12-05-077_right.tif"},
{"id": "5c507cb84f0cfd2aedf5a75a",
Expand All @@ -78,12 +92,12 @@ First we format the base URL for our query allowing us to reuse it for each file
Next we loop through our array and create a customized URL while making the call to fetch the data using the `requests` interface.
Finally we open the output file and use a loop to write the retrieved data.

```{python py_fetch_image_files, eval=FALSE}
```{python py_fetch_image_files}
import requests
from io import open

# We are using the same `url` and `key` variables declared in the previous example above.
filesurl = url + '/files/'
filesurl = url + 'files/'
params={ 'key': key }

for f in files:
Expand All @@ -108,71 +122,3 @@ Below are examples of images captured approximately one month apart [^1] [^2]

[^1]: May 4, 2018 - rgb_geotiff_L1_ua-mac_2018-05-04__13-07-04-077_right.tif,rgb_geotiff_L1_ua-mac_2018-05-04__13-07-04-077_left.tif
[^2]: Jun 2, 2018 - rgb_geotiff_L1_ua-mac_2018-06-02__14-12-05-077_right.tif,rgb_geotiff_L1_ua-mac_2018-06-02__14-12-05-077_left.tif


<!-- The following is the 'correct' way of retrieving file names and IDS based upon a site and sensor name



## Objective: To be able to demonstrate how to locate and retrieve RGB image files

This vignette shows how to locate and retrieve image files from your site for a specific date range using Python.
We will be searching for and retrieving [ublicly available images associated with growing Season 6 from the University of Arizona's [Maricopa Agricultural Center](http://cals-mac.arizona.edu/).
Note that the search in this vignette will not return any results, but we wull walk you through the process so that you can query your own data.
The files are stored online on the data management system Clowder, which is accessed using an API.

After completing this vignette it should be possible to search for and retrieve other files through the use of the API.

As an added bonus we've also included an exmple of how to retrieve the list of available sensor names through the API.
By using the sensor names returned, it's possible to retrieve other files containing the data the sensors have collected.

**Requirements**

* Python
* the terrautils library
+ this can be installed from pypi by running `pip install terrautils` in the terminal
* an API key to access these data

The API key is a string that gets generated upon request through your Clowder account.
Existing API keys will work with this vignette.
The following steps can help you get a new API key

1. first register with Clowder at "https://terraref.ncsa.illinois.edu/clowder/" site
a) click the `Login` button and wait for the login screen to appear
b) then select the `Sign up` button and enter an email address you have access to
2. an email is sent to the entered address with instructions for completing the registration process
3. once registration is complete
c) log into Clowder and select the `View profile` menu option from the drop-down that is near the search control
d) enter a name and click the `+ Add` button under "User API Keys" heading in the profile page to generate a new key
4. copy the key shown for use in this vignette

## Locating the images

To begin looking for files, a sensor name and site name are needed.
We will be using 'RGB GeoTIFFs Datasets' as the sensor name and 'MAC Field Scanner Season 6 Range 20 Column 6' as the site name.
Later in this vignette we show how to retrieve the list of available sensors.
When seraching for your images, you will want to use your site name.

The URL string points to the API to query.
In this case we'll be using "https://terraref.ncsa.illinois.edu/clowder/api".
We will be using an empty key for this example since we are querying public data.
You will want to use the key you created for your Clowder account when querying your data.

As mentioned in the Objectives, we are limiting the search timeframe through the use of the `since` and `until` parameters.
These parameters can be omitted to search for all available images (or all other files by changing the sensor).

```{python py_get_file_listing, eval=FALSE}
from terrautils.products import get_file_listing

url = 'https://terraref.ncsa.illinois.edu/clowder/'
key = ''
sensor = 'RGB GeoTIFFs Datasets'
sitename = 'MAC Field Scanner Season 6 Range 20 Column 6'
files = get_file_listing(None, url, key, sensor, sitename,
since='2018-05-01', until='2018-05-31')
```

The `files` variable now contains an array of all the file in the datasets that match the sensor in the plot for the month of May.
As with this example, when performing you own queries it's possible that there are no matches found and the `files` array would be empty.

-->