Improve docs tutorial #164

spencerahill · 2017-03-29T06:24:57Z

Closes #151

@spencerkclark with apologies for poor git practices, I ran out of steam before I could finish the last chunk of the Examples page (I noted the point where I stopped).

Will return to this tomorrow. I feel like it's mostly there, but of course any suggestions welcome.

spencerkclark

@spencerahill this looks very good! Thanks for all your effort here cleaning up the internal links in the docs, fleshing out the submit_mult_calcs docstring, and greatly improving the examples page by adding more description of the aospy workflow, and porting it to mirror (and ultimately use) the example_obj_lib. Adding the scientific details as footnotes is also a really nice touch.

I've gone through your changes so far and offered some comments/suggestions; hopefully you find them helpful.

spencerkclark · 2017-03-29T10:58:11Z

aospy/automate.py

+            - None : no vertical reduction
+            - 'vert_av' : mass-weighted vertical average
+            - 'vert_int' : mass-weighted vertical integral
+        input_time_intervals : {'annual', 'monthly', 'daily', '6hr', '3hr'}


Maybe put an "e.g." here to note that these are not the only options (and that the values one needs to specify here are controlled by how one sets up their DataLoaders).

spencerkclark · 2017-03-29T11:00:09Z

aospy/automate.py

+              calculations to be performed and prompt user to confirm before
+              submitting for execution.
+        - parallelize : (default False) If True, submit calculations in
+              parallel.


Should we note here that the way this is set up currently, this requires installing an optional dependency (pip install multiprocess)?

spencerkclark · 2017-03-29T11:06:30Z

aospy/automate.py

+        - parallelize : (default False) If True, submit calculations in
+              parallel.
+        - write_to_tar : (default True) If True, write results of calculations
+              to .tar files, one for each object.  These tar files have an


"one for each object" feels somewhat ambiguous here; I think you mean "one for each :py:classaospy.Run"?

spencerkclark · 2017-03-29T11:11:25Z

aospy/automate.py

+              parallel.
+        - write_to_tar : (default True) If True, write results of calculations
+              to .tar files, one for each object.  These tar files have an
+              identical directory structures the standard output relative to


This is possibly a bug, but I'll just note that this is not entirely true at the moment. For example the example main script, aospy/examples/aospy_main.py, produces the following directory structures:

Normal output:
example-output/example_proj/example_model/example_run

Tar output (note the data directory under example_proj):
example-tar-output/example_proj/data/example_model/example_run

Good catch! I agree, this is a bug in my view. I will fix it within this PR if you agree.

spencerkclark · 2017-03-29T11:23:29Z

docs/using-aospy.rst

@@ -239,7 +233,8 @@ and in the directory structure within which they are saved.
 - File name :
  ``varname.intvl_out.dtype_out_time.'from_'intvl_in'_'dtype_in_time.model.run.date_range.nc``

-See the API reference documentation of ``CalcInterface`` for explanation of each of these components of the path and file name.
+See the API reference documentation of ``CalcInterface`` for


Should you use :py:class:aospy.CalcInterface here?

spencerkclark · 2017-03-29T11:47:40Z

docs/examples.rst

-            '../aospy/test/data/netcdf/00040101.precip_monthly.nc',
-            '../aospy/test/data/netcdf/im.landmask.nc'
+            rootdir + '/aospy/test/data/netcdf/00040101.precip_monthly.nc',
+            rootdir + '/aospy/test/data/netcdf/im.landmask.nc'


It might be confusing to users why we are specifying a land mask for an aquaplanet simulation; should we make a note about this requirement? In this case it should¹ be all zeros at all latitudes and longitudes.

¹Good thing I checked...it turns out that it isn't. This is a land mask for a rectangular continent. I will submit a PR to correct this.

See #165 for the fix

spencerkclark · 2017-03-29T12:11:39Z

docs/examples.rst

+    calcs[0].path_out
+    calcs[0].data_out
+
+**(S. Hill: Still need to finish up everything below this point)**


Sorry, I recognize you're not done here, but I'm adding some comments anyway :)

spencerkclark · 2017-03-29T12:11:41Z

docs/examples.rst

+
+   You may have noticed that ``subset_`` and ``raw_`` coordinates have
+   years 1678 and later, when our data was from model years 4
+   through 6.  This is because technical details upstream (in numpy)


This issue is complicated and there are multiple levels of upstream dependencies; instead of calling out numpy in particular, I might just link to the discussion in this issue: pandas-dev/pandas#7307

spencerkclark · 2017-03-29T12:11:43Z

docs/examples.rst

+   You may have noticed that ``subset_`` and ``raw_`` coordinates have
+   years 1678 and later, when our data was from model years 4
+   through 6.  This is because technical details upstream (in numpy)
+   limit the range of supported years to roughly 1677 to 2234.


Let's link directly to this part of the pandas docs here: http://pandas.pydata.org/pandas-docs/stable/timeseries.html#timestamp-limitations

Also the upper bound year should be 2262 (instead of 2234).

spencerkclark · 2017-03-29T12:16:05Z

docs/examples.rst

+We can re-use our object library at will to perform new calculations
+or re-compute old ones.  We can also add new objects.  For example,
+suppose we performed a new simulation in which we increased the
+surface albedo or introduced a rectangular continent.  All we would


In this case of a rectangular continent, I might say "not so fast!" :)

Currently the safest thing to do in that situation would be to create a new Model with a different land mask (to make sure regional calculations that depended on land would be accurate). Changing surface albedo is fine for the same Model, however.

If you want some other parameters to tweak as an example (while keeping the same Model object) you could say increase the solar constant or turn off the convection scheme.

spencerahill · 2017-03-29T17:13:45Z

@spencerkclark thanks a lot! Won't bother responding in-line, because it all sounds good. Will merge #165 also.

spencerahill · 2017-03-29T20:23:32Z

@spencerkclark ok I've addressed all your comments and finished the Examples section.

I will go through all of the docs pages later today one more time...I suspect there are some out of date parts in Using aospy especially, given all the recent modifications.

After that I'd say ready to merge.

spencerkclark · 2017-03-29T20:29:52Z

This looks great to me; go ahead and merge! Thanks again.

I will go through all of the docs pages later today one more time...I suspect there are some out of date parts in Using aospy especially, given all the recent modifications.

Sounds good; I'll take care of updating the example Jupyter notebook. I think I should be able to get our CI configured to test that as well.

spencerahill · 2017-03-30T00:58:13Z

Ok, finished going through all the docs again. Merging pending CI pass

spencerkclark

Looks good! Just caught a minor typo.

spencerkclark · 2017-03-30T01:18:05Z

docs/using-aospy.rst

+
+:py:func:`aospy.submit_mult_calcs` creates a :py:class:`aospy.CalcSuite`
+object that permutes over the provided lists of calculation
+specifications, encoding each permutations into


Typo: "each permutations"

good catch, thanks

Spencer Hill added 3 commits March 28, 2017 23:17

Add aospy.examples.example_obj_lib

e335268

Update submit_mult_calcs docstring, Calc repr, & example lib formatting

0506885

DOC use example_obj_lib in Examples; other less major doc edits

cbaf9f9

spencerkclark reviewed Mar 29, 2017

View reviewed changes

Spencer Hill added 3 commits March 29, 2017 13:18

Remove "data" level from tar output directory

a246a39

Finish examples section; improve submit_mult_calcs docstring

57a28f5

Formatting improvements to API and Using aospy

4699b16

Spencer Hill added 2 commits March 29, 2017 15:02

Update what's new

cbf1559

DOC Semi-major updates to 'Using aospy'; other more minor doc edits

1a1015b

spencerkclark reviewed Mar 30, 2017

View reviewed changes

fix typo

95c3140

spencerahill merged commit b5edf48 into develop Mar 30, 2017

spencerahill deleted the improve-docs-tutorial branch March 30, 2017 03:11

spencerahill mentioned this pull request Mar 30, 2017

Update Examples page of docs: use main script to submit calculations #150

Closed

spencerahill changed the title ~~WIP Improve docs tutorial~~ Improve docs tutorial Mar 30, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve docs tutorial #164

Improve docs tutorial #164

spencerahill commented Mar 29, 2017

spencerkclark left a comment

spencerkclark Mar 29, 2017

spencerkclark Mar 29, 2017

spencerkclark Mar 29, 2017

spencerkclark Mar 29, 2017

spencerahill Mar 29, 2017

spencerkclark Mar 29, 2017

spencerkclark Mar 29, 2017

spencerkclark Mar 29, 2017

spencerkclark Mar 29, 2017

spencerkclark Mar 29, 2017

spencerkclark Mar 29, 2017

spencerkclark Mar 29, 2017

spencerkclark Mar 29, 2017

spencerahill commented Mar 29, 2017

spencerahill commented Mar 29, 2017

spencerkclark commented Mar 29, 2017

spencerahill commented Mar 30, 2017

spencerkclark left a comment

spencerkclark Mar 30, 2017

spencerahill Mar 30, 2017

Improve docs tutorial #164

Improve docs tutorial #164

Conversation

spencerahill commented Mar 29, 2017

spencerkclark left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

spencerahill commented Mar 29, 2017

spencerahill commented Mar 29, 2017

spencerkclark commented Mar 29, 2017

spencerahill commented Mar 30, 2017

spencerkclark left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment