This repository was archived by the owner on Jan 9, 2020. It is now read-only.
Refactor satellite notebook#379
Merged
Merged
Conversation
dopplershift
suggested changes
Nov 20, 2018
Member
dopplershift
left a comment
dopplershift
left a comment
There was a problem hiding this comment.
In general looks good and I think helps get this one in good shape to teach, applying many, many lessons learned. 😁 Just a few minor comments.
| "For current data, you could navigate to the `Test Datasets` directory, then `GOES Products` and `GOES-16`. There are subfolders for the CONUS, full disk, mesoscale sector images, and other products. In each of these is a folder for each [channel of the ABI](http://www.goes-r.gov/education/ABI-bands-quick-info.html). In each channel there is a folder for every day in the approximately month-long rolling archive. As you can see, there are a massive amount of data coming down from GOES-16!\n", | ||
| "\n", | ||
| "We can use Unidata's [Siphon](https://github.com/Unidata/siphon) package to parse the catalog from the TDS. This provides us a nice programmatic way of accessing the data. We start by importing the `TDSCatalog` class from siphon and giving it the URL to the catalog we just surfed to manually. **Note:** Instead of giving it the link to the HTML catalog, we change the extension to XML, which asks the TDS for the XML version of the catalog. This is much better to work with in code. If you forget, the extension will be changed for you with a warning being issued from siphon.\n", | ||
| "In the next section we'll be downloading the data in a pythonic way, so our first task is to build a URL that matches that we manually navigated to in the web browser. To make it as flexible as possible, we'll want to use variables for the sector name (CONUS, full-disk, mesoscale-1, etc.), the date, and the ABI channel number." |
Member
There was a problem hiding this comment.
"That matches that we manually navigated"
missing a word...or something
| "outputs": [], | ||
| "source": [ | ||
| "cat.datasets[-5:]" | ||
| "# !load solutions/data_url.py\n", |
Member
There was a problem hiding this comment.
Solution is actually in the notebook.
| "Rather than just giving back the raw array of data, this gives back a `DataArray` object; from here not only\n", | ||
| "can we get the raw data values, but there are useful metadata as well. We can see just what additional information\n", | ||
| "is present by printing out the `DataArray` object:" | ||
| "We're finally ready to get the actual data. We could download the file, then open that, but there is no need! We'll get it and just hold it in memory while we're working with it. No mess to clean up afterwards! Notice that we're using the XArray accessor which will make life much nicer that dealing with the raw netCDF (like we used to back in the days of early 2018)." |
Member
There was a problem hiding this comment.
It's not actually downloading anything initially. It's using opendap or cdmremote, but either way at this point all it does it download some metadata and maybe coordinates.
| "metadata": {}, | ||
| "source": [ | ||
| "This reveals several useful pieces of information, but we're going to focus on the `grid_mapping` data. This attribute is defined by the [NetCDF Climate and Forecast (CF) Metadata Conventions](http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html) and specifies a variable that contains information about the grid's projection." | ||
| "Yes, we reused the `ds` name - mostly just to help keep our namespace realtively uncluttered, but no need to. This is one of the powers of Python - `ds` can be a string, a class, an integer, or anything else all during the course of one program." |
Member
There was a problem hiding this comment.
Good explanation. But is this something we should be doing?
Contributor
Author
There was a problem hiding this comment.
I was following my predecessors pattern 👿 - I'm open to changing it certainly, but is that useful to keep the old handle around?
Member
There was a problem hiding this comment.
Yeah, your predecessor made mistakes. 😉 I don’t think we need to keep the old handle around, but I don’t think we should be demonstrating re-using the variable name. We wouldn’t want that in MetPy code and it also leads to confusing behavior in the notebook—it makes re-running cells problematic.
| "metadata": {}, | ||
| "source": [ | ||
| "This is a nice start, but it would be nice to have better geographic references for the image. For example, where are the states? Cartopy's `feature` module has support for adding geographic features to plots, with many features are built in. The `BORDERS` built-in feature contains country borders and `STATES` contains US state boundaries. There is also support for creating \"custom\" features from the [Natural Earth](http://www.naturalearthdata.com/) set of free vector and raster map data (CartoPy will automatically download the necessary data and cache it locally)." | ||
| "Before we get to plotting, let's go ahead and get a handle to the x and y coordinates that we'll need. This is wasy to do with the accessor on our data variable DataArray." |
| "\n", | ||
| "Colormapping in matplotlib (which backs CartoPy) is handled through two pieces:\n", | ||
| "\n", | ||
| "- The colormap controls how values are converted from floating point values in the range [0, 1] to colors (think colortable)\n", |
Member
There was a problem hiding this comment.
If you flip the order of these bullets, they would read in the order of how the steps are actually applied to convert from their original data to an image. I'm not sure if you had another intention in mind here behind the ordering, though.
Contributor
Author
|
Revised per your suggestions @dopplershift |
jrleeman
force-pushed
the
satellite_refactor
branch
from
67b6a02 to
2052e5f
Compare
dopplershift
approved these changes
Nov 27, 2018
Member
|
Travis failure looks real. |
jrleeman
force-pushed
the
satellite_refactor
branch
from
2052e5f to
a1e3ace
Compare
jrleeman
force-pushed
the
satellite_refactor
branch
from
a1e3ace to
1cf5da7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refactor and clean up exercises.