diff --git a/.github/ISSUE_TEMPLATE/default_template.md b/.github/ISSUE_TEMPLATE/default_template.md index 2a2c87b..b054e9b 100644 --- a/.github/ISSUE_TEMPLATE/default_template.md +++ b/.github/ISSUE_TEMPLATE/default_template.md @@ -1,4 +1,4 @@ -*Fill out this issue by completing the items below. Not every item may be relevent to you, so feel free to make changes* +*Fill out this issue by completing the items below. Not every item may be relevant to you, so feel free to make changes* #### The tutorial or vignette name Name: diff --git a/index.Rmd b/index.Rmd index e318ad0..5a55af4 100644 --- a/index.Rmd +++ b/index.Rmd @@ -12,7 +12,7 @@ documentclass: book This book is intended to quickly introduce users to TERRA REF data through a series of tutorials. TERRA REF has many types of data, and most can be accessed in multiple ways. Although this makes it more complicated to learn (and teach!), the objective is to provide users with the flexibility to access data in the most useful way. -The first section walks the user through the steps of downloading and combining three different types of data: plot level phenotypes, meteorological data, and images. Subesquent sections provide more detailed examples that show how to access a larger variety of data and meta-data. +The first section walks the user through the steps of downloading and combining three different types of data: plot level phenotypes, meteorological data, and images. Subsequent sections provide more detailed examples that show how to access a larger variety of data and meta-data. For those who are eager to see what is on deck, there are additional tutorials under development in the [tutorials repository](https://github.com/terraref/tutorials) on GitHub. These can be found in the traits, sensors, and genomics subfolders. Users are encouraged to revise and contribute new content. @@ -20,16 +20,16 @@ For those who are eager to see what is on deck, there are additional tutorials u While we assume that readers will have some familiarity with the nature of the problem - remote sensing of crop plants - for the most part, these tutorials assume that the user will bring their own scientific questions and a sense of curiosity and are eager to learn. -These tutorials are aimed at users who are familiar with or willing to learn programming languages including R (particularly for accessing plot level trait data) and Python (primarily for accessing environmental data and sensor data). In addition, there are examples of using SQL for more sophisticated database queries as well as the bash terminal. +These tutorials are aimed at users who are familiar with or willing to learn programming languages including R (particularly for accessing plot level trait data) and Python (primarily for accessing environmental data and sensor data). In addition, there are examples of using SQL for more sophisticated database queries, as well as the Bash terminal. -Some of the lessons only require a web browser; others will assume familarity with programming at the command line in (typically only one of) Python, R, and / or SQL. You should be willing to find help (see finding help, below). +Some of the lessons only require a web browser; others will assume familiarity with programming at the command line in (typically only one of) Python, R, and / or SQL. You should be willing to find help (see finding help, below). ### Technical Requirements At a minimum, you should have: * An internet connection -* Web Browser +* Web browser * Access to the data that you are using + The tutorials will state which databases you will need access to * Software: @@ -39,12 +39,12 @@ At a minimum, you should have: The first few chapters in the 'vignettes' section use publicly available sample data sets. Subsequent sections are also written to use publicly available data sets, but some of the examples require data that requires users to sign up. -To sign up, you will need to 1) fill out the TERRA REF Beta user questionaire ([terraref.org/beta](https://terraref.org/beta)) and 2) request access to specific databases. +To sign up, you will need to 1) fill out the TERRA REF Beta user questionnaire ([terraref.org/beta](https://terraref.org/beta)) and 2) request access to specific databases. - -## Ways of Acessing Data +## Ways of Accessing Data * Web Interfaces + [Clowder](https://docs.terraref.org/user-manual/how-to-access-data/using-clowder-sensor-and-genoomics-data) (sensor and genomic data) diff --git a/metadata/metadata_tutorial_addition.Rmd b/metadata/metadata_tutorial_addition.Rmd index 6063675..4c831d1 100644 --- a/metadata/metadata_tutorial_addition.Rmd +++ b/metadata/metadata_tutorial_addition.Rmd @@ -27,7 +27,7 @@ URL to a public published google sheet (you will need to provide this url in the # How to Run this Rmd -Knitting this markdown file will not upload any data to the test database. Being able to knit this file means that you will not encounter any errors when uploading your metadata. All experiments, sites, treatments, cutlivars, and citations can be uploaded successfully. +Knitting this markdown file will not upload any data to the test database. Being able to knit this file means that you will not encounter any errors when uploading your metadata. All experiments, sites, treatments, cultivars, and citations can be uploaded successfully. To actually commit additions to the test database, you will need to make an edit to this markdown file. diff --git a/metadata/test_database_tutorial.Rmd b/metadata/test_database_tutorial.Rmd index 307f81f..dbc9198 100644 --- a/metadata/test_database_tutorial.Rmd +++ b/metadata/test_database_tutorial.Rmd @@ -11,7 +11,7 @@ The purpose of this tutorial is to demonstrate how to set up a local instance of You will want to set up this test database when preparing to upload data to the master database. -You should make sure that your data can be succesfully uploaded to the test database before trying to upload to master. +You should make sure that your data can be successfully uploaded to the test database before trying to upload to master. # Requirements: @@ -41,7 +41,7 @@ After initial set up, you will only need to run step 4 to restart the database. ## Step 1: Clone bety repository -In a directory of your chioce, clone the bety repository from github and cd to the bety folder: +In a directory of your choice, clone the bety repository from github and cd to the bety folder: ```sh git clone https://github.com/PecanProject/bety.git @@ -107,7 +107,7 @@ Open a new tab in the terminal and connect to the test database using: ```sh psql -h localhost -p 5433 -U bety ``` -You should see the following prompt if you have succesfully logged into the test database: +You should see the following prompt if you have successfully logged into the test database: ```sh psql (11.2, server 9.5.16) Type "help" for help. diff --git a/sensors/01-meteorological-data.Rmd b/sensors/01-meteorological-data.Rmd index 3828721..5434cf3 100644 --- a/sensors/01-meteorological-data.Rmd +++ b/sensors/01-meteorological-data.Rmd @@ -147,7 +147,7 @@ The data represent 5 minute summaries aggregated from 1/s observations. ### Download data using the command line Data can be downloaded from Clowder using the command line program Curl. If the -following is typed into the commmand line, it will download the datapoints data +following is typed into the command line, it will download the datapoints data that we're interested in as a file which we have chosen to call `spectra.json`. ```{sh eval=FALSE} diff --git a/sensors/06-list-datasets-by-plot.Rmd b/sensors/06-list-datasets-by-plot.Rmd index 8318ff0..ffadee6 100644 --- a/sensors/06-list-datasets-by-plot.Rmd +++ b/sensors/06-list-datasets-by-plot.Rmd @@ -28,10 +28,10 @@ The *key* is a unique access key for the Clowder API. ## Getting the sensor list -The first thing to get is the sensor name. This can be retreived using the +The first thing to get is the sensor name. This can be retrieved using the `get_sensor_list` function. This function returns the full record which may be useful in some cases but primarily includes sensor names that include -a plot id number. The utility function `unique_sensor_names` accpets the +a plot id number. The utility function `unique_sensor_names` accepts the sensor list and provides a list of names suitable for use in the `get_file_listing` function. @@ -96,7 +96,7 @@ of requests against the API it's possible to determine the files of interest and Each of the API URL's have the same beginning (https://terraref.ncsa.illinois.edu/clowder/api), followed by the data needed for a specific request. As we step through the process you will be able -to see how then end of the URL changes depending upon the reuqest. +to see how then end of the URL changes depending upon the request. Below is what the API looks like as a URL. Try pasting it into your browser. @@ -156,7 +156,7 @@ STREAM_ID=11586 curl -o datasets.json -X GET "https://terraref.ncsa.illinois.edu/clowder/api/geostreams/datapoints?stream_id=${STREAM_ID}" ``` -After the call succeeds, a file named *datasets.json* is created containing the returned JSON onject. As part of the +After the call succeeds, a file named *datasets.json* is created containing the returned JSON object. As part of the JSON object there are one or more `properties` fields containing *source_dataset* parameters. ```{javascript eval=FALSE} @@ -211,7 +211,7 @@ of the files in the dataset with their IDs, and other data if available: ### Retrieving the files -Given that a large number of files may be contained in a dataset, it may be desireable to automate the process of pulling down files +Given that a large number of files may be contained in a dataset, it may be desirable to automate the process of pulling down files to the local system. For each file to be retrieved, the unique file ID is needed on the URL. diff --git a/sensors/09-vnir-radiometer.Rmd b/sensors/09-vnir-radiometer.Rmd index fea7a01..949e537 100644 --- a/sensors/09-vnir-radiometer.Rmd +++ b/sensors/09-vnir-radiometer.Rmd @@ -10,7 +10,7 @@ output: html_document An Ocean Optics STS Spectrometer measures downwelling solar spectral radiance every 5s on top of the Gantry. -Lets look at the output from this sensor over the course of the day (and then we will see how to acces the data): +Lets look at the output from this sensor over the course of the day (and then we will see how to access the data): ```{r one-downwelling-spectra} library(tidyverse) diff --git a/traits/00-BETYdb-getting-started.Rmd b/traits/00-BETYdb-getting-started.Rmd index e4dc960..5baea47 100644 --- a/traits/00-BETYdb-getting-started.Rmd +++ b/traits/00-BETYdb-getting-started.Rmd @@ -1,4 +1,4 @@ -# (PART\*) Secton 2: Traits {-} +# (PART\*) Section 2: Traits {-} # Getting Started with BETYdb diff --git a/traits/06-agronomic-metadata.Rmd b/traits/06-agronomic-metadata.Rmd index 6af2f6a..f90f445 100644 --- a/traits/06-agronomic-metadata.Rmd +++ b/traits/06-agronomic-metadata.Rmd @@ -1,6 +1,6 @@ # Querying Agronomic Meta-data -In previous tutorials you have learned how to query trait data using a variety of different methods, including the web interface, an API, and the R traits package. Here you will continute to use the R traits package, and learn how to access meta-data from other tables in the database. +In previous tutorials you have learned how to query trait data using a variety of different methods, including the web interface, an API, and the R traits package. Here you will continue to use the R traits package, and learn how to access meta-data from other tables in the database. While the basic search query that we have used in previous sections provides the key information that you may need for an analysis - the genotype name, the location, date, and method, there are other tables that contain more specific metadata. diff --git a/traits/07-betydb-sql-access.Rmd b/traits/07-betydb-sql-access.Rmd index 1bce7bd..4da438a 100644 --- a/traits/07-betydb-sql-access.Rmd +++ b/traits/07-betydb-sql-access.Rmd @@ -4,7 +4,7 @@ will be derived from https://github.com/pi4-uiuc/2017-bootcamp/blob/master/conte ## On workbench -On the TERRA REF Workbench, you have access to the database. These connnections will only work if you are on the workbench. +On the TERRA REF Workbench, you have access to the database. These connections will only work if you are on the workbench. ## Using PostgresSQL Studio diff --git a/traits/10-simulated-sorghum.Rmd b/traits/10-simulated-sorghum.Rmd index 013c82e..4a250ca 100644 --- a/traits/10-simulated-sorghum.Rmd +++ b/traits/10-simulated-sorghum.Rmd @@ -31,7 +31,7 @@ The 'genotypes' are based on five-hundred quasi-random parameterizations of a bi _Note that these data sets contain numerical artifacts and scientific reinterpretations for illustrative purpose._ -All of these simulated datasets are released with an unrestrive [copyright](https://creativecommons.org/publicdomain/zero/1.0/). This means you can copy, modify, and share the data. Please keep in mind that the data sets are not production quality - they have been developed solely to inspire and solicit feedback. +All of these simulated datasets are released with an unrestrictive [copyright](https://creativecommons.org/publicdomain/zero/1.0/). This means you can copy, modify, and share the data. Please keep in mind that the data sets are not production quality - they have been developed solely to inspire and solicit feedback. ### Design of Simulation Experiment @@ -72,7 +72,7 @@ Two-hundred and twenty-seven lines were grown at each site. Each line is identif The phenotypes associated with each genotype is in the file `phenotypes.csv`. -These 'phenotypes' are used as input parameters to the simulation model. We often refer to these as 'traits' (as opposed to biomass or growth rates, which are states and proceses). In this example, we assume that 'phenotypes' are time-invariant. +These 'phenotypes' are used as input parameters to the simulation model. We often refer to these as 'traits' (as opposed to biomass or growth rates, which are states and processes). In this example, we assume that 'phenotypes' are time-invariant. | variable_id|name |standard_name |units |Description | @@ -293,7 +293,7 @@ Now, join the traits data frame with the new max_height data frame trait data we traits_height <- traits_wide %>% left_join(max_height, by = 'genotype') ``` -Which traits are related to height? We can discover this in a few way, for example, a pairs plot that shows correltations: +Which traits are related to height? We can discover this in a few way, for example, a pairs plot that shows correlations: ```{r trait_pairs, fig.height = 8, fig.width = 8, warning = FALSE} ggpairs(traits_height %>% select(-genotype), diff --git a/videos/first_walkthrough.Rmd b/videos/first_walkthrough.Rmd index a1b4595..a1de7b7 100644 --- a/videos/first_walkthrough.Rmd +++ b/videos/first_walkthrough.Rmd @@ -22,7 +22,7 @@ These live walkthroughs will be turned into video tutorials that can be viewed a #### Project overview TERRA REF is a project funded by the Advanced Research Projects Agency for Energy (ARPA-E). The objective is to advance the science and technology of high throughput field phenotyping, which is the automated measurement of plants. TERRA REF has deployed an exceptionally large robot capable of collecting data from a range of sensors at 1 mm spatial resolution every other day. You can learn more about the project from the website, terraref.org. The suite of sensors include visible light cameras, hyperspectral cameras, a laser 3d scanner, and environmental data. In addition there are weather stations and high resolution genome sequences for hundreds of varieties of Sorghum being monitored. -These tutorials are to make these data accessible and useable. I will provide an overview of how to use computational tools to get these data, and combine, plot, and analyze them. +These tutorials are to make these data accessible and usable. I will provide an overview of how to use computational tools to get these data, and combine, plot, and analyze them. We have written online tutorials [here](https://terraref.github.io/tutorials/). These walkthroughs and videos complement those materials, which are growing and improving continually. @@ -249,7 +249,7 @@ We'll calculate a metric from temperature called growing degree days. Basically First we just want the date, not including time of day. Pull out date using built-in `as.Date` and put into new column `day`. -Also create new column with air temperature in Celcius instead of Kelvin. +Also create new column with air temperature in Celsius instead of Kelvin. ```{r} weather_daily <- weather_all %>% diff --git a/videos/first_walkthrough.md b/videos/first_walkthrough.md index fd55e38..7007a6e 100644 --- a/videos/first_walkthrough.md +++ b/videos/first_walkthrough.md @@ -35,7 +35,7 @@ cameras, hyperspectral cameras, a laser 3d scanner, and environmental data. In addition there are weather stations and high resolution genome sequences for hundreds of varieties of Sorghum being monitored. -These tutorials are to make these data accessible and useable. I will +These tutorials are to make these data accessible and usable. I will provide an overview of how to use computational tools to get these data, and combine, plot, and analyze them. @@ -415,7 +415,7 @@ over time during the year. First we just want the date, not including time of day. Pull out date using built-in `as.Date` and put into new column `day`. -Also create new column with air temperature in Celcius instead of +Also create new column with air temperature in Celsius instead of Kelvin. ``` r diff --git a/videos/second_walkthrough.Rmd b/videos/second_walkthrough.Rmd index 4a8bbb3..5a2b667 100644 --- a/videos/second_walkthrough.Rmd +++ b/videos/second_walkthrough.Rmd @@ -58,7 +58,7 @@ These data are summarized from camera data, can see that by unselecting "Heat Ma We can download these individual files. TERRA REF data are on two platforms. We pulled data from the first platform Clowder last week. Other platform is called Globus, let's work with that first. -All data are on Globus. Go to [website](globus.org) and log in. Need to set up account and get acccess to Terraref collection. Instructions are [here](https://docs.terraref.org/user-manual/how-to-access-data/using-globus-sensor-and-genomics-data). +All data are on Globus. Go to [website](globus.org) and log in. Need to set up account and get access to Terraref collection. Instructions are [here](https://docs.terraref.org/user-manual/how-to-access-data/using-globus-sensor-and-genomics-data). In File Manager section. Click "Start here", "Shared With You" tab, and then "Terraref" option. This gives access to a bunch of files for the project. diff --git a/vignettes/00-introduction.Rmd b/vignettes/00-introduction.Rmd index 823153f..25ace57 100644 --- a/vignettes/00-introduction.Rmd +++ b/vignettes/00-introduction.Rmd @@ -22,7 +22,7 @@ Information for getting a beta user account is available in the section on [User ### R In order to use the vignettes, you will need to have R installed along with the extra packages that are used. -All of the packages except 'trait' are available on CRAN and can be installed using the `install.packages` funciton: +All of the packages except 'trait' are available on CRAN and can be installed using the `install.packages` function: ```r install.packages('tidyverse', 'jsonlite', 'knitr', 'lubridate', 'raster', 'sf') diff --git a/vignettes/01-get-trait-data-R.Rmd b/vignettes/01-get-trait-data-R.Rmd index 8348bbf..91a2d1c 100644 --- a/vignettes/01-get-trait-data-R.Rmd +++ b/vignettes/01-get-trait-data-R.Rmd @@ -13,7 +13,7 @@ In this chapter you will learn: In this chapter, we go over how to query TERRA REF trait data using the [`traits`](https://github.com/ropensci/traits) package. The `traits` package is a way to query for various sources of species trait data, including [BETYdb](https://www.betydb.org/), [NCBI](http://www.ncbi.nlm.nih.gov/), [Coral Traits Disease](http://coraltraits.org/) and others. In this chapter we use BETYdb as our trait source, as it contains the TERRA REF data that we are interested in. -Our example will show how to query for season 6 data and visualize canopy height. In addition to the `traits` package we will also be using some of the [`tidyverse`](https://www.tidyverse.org/) packages, which allow us to maniupulate the data in an efficient, understandable way. If you are unfamiliar with `tidyverse` syntax, we recommend checking out some of the resources [here](https://www.tidyverse.org/learn/). +Our example will show how to query for season 6 data and visualize canopy height. In addition to the `traits` package we will also be using some of the [`tidyverse`](https://www.tidyverse.org/) packages, which allow us to manipulate the data in an efficient, understandable way. If you are unfamiliar with `tidyverse` syntax, we recommend checking out some of the resources [here](https://www.tidyverse.org/learn/). @@ -99,7 +99,7 @@ kable(traits) ### Querying season 6 canopy height data -You may find after constructing a general query as above that you want to only query a specific trait. Here, we query for the canopy height trait by adding the key-value pair `trait = "canopy_height"` to our query function. Note that the limit is also set to return only 250 records, shown here for demostration purposes. +You may find after constructing a general query as above that you want to only query a specific trait. Here, we query for the canopy height trait by adding the key-value pair `trait = "canopy_height"` to our query function. Note that the limit is also set to return only 250 records, shown here for demonstration purposes. ```{r traits-vig-canopy-query, message = FALSE} diff --git a/vignettes/03-get-images-python.Rmd b/vignettes/03-get-images-python.Rmd index ebb36d5..64c481d 100644 --- a/vignettes/03-get-images-python.Rmd +++ b/vignettes/03-get-images-python.Rmd @@ -25,7 +25,7 @@ All the `terrautils` functions are now available in Python, although we will onl ## Retrieving sensor names In this section we retrieve the names of different sensor types that are available. -This will allow you to understand what files may be avaialable other than just those containing RBG image data. +This will allow you to understand what files may be available other than just those containing RBG image data. In order to run Python functions, including those from the `terrautils` library, within this Rmarkdown, we have to install and set up `reticulate`. diff --git a/vignettes/04-synthesis-data.Rmd b/vignettes/04-synthesis-data.Rmd index e36c43c..badea2e 100644 --- a/vignettes/04-synthesis-data.Rmd +++ b/vignettes/04-synthesis-data.Rmd @@ -166,7 +166,7 @@ ggplot(data.frame(inf_points = unique(all_cultivars$inf_x))) + ## Get image data -In this examnple we will extract our plot data from a series of images taken in May of Season 6, measure its "greeness" annd plot that against the plant heights from above in this vignette. +In this example we will extract our plot data from a series of images taken in May of Season 6, measure its "greeness" annd plot that against the plant heights from above in this vignette. The chosen statistic here is the normalised green-red difference index, $\textrm{NGRDI}=\frac{R-G}/{R+G}$ (Rasmussen et al., 2016), which uses the red and green bands from the image raster.