Code

Feature/one step save baseline#193

Code

Merged

nbren12 merged 83 commits intodevelop-one-stepsai2cm/fv3net:develop-one-stepsfrom feature/one-step-save-baselineai2cm/fv3net:feature/one-step-save-baselineCopy head branch name to clipboardMar 26, 2020

+475-187

Commits

Commits on Mar 17, 2020

• Refactore online_modules to fv3net

  nbren12

  committed
  8ca8819

  Copy full SHA for 8ca8819

• black

  nbren12

  committed
  5909c73

  Copy full SHA for 5909c73

• Create us.gcr.io/vcm-ml/prognostic_run:v0.1.0

  Show description for cee28fd
  nbren12

  committed
  cee28fd

  Copy full SHA for cee28fd

• Refactor us.gcr.io/vcm-ml/fv3net image build code

  nbren12

  committed
  305629f

  Copy full SHA for 305629f

• Add build_images makefile target

  nbren12

  committed
  ff9f938

  Copy full SHA for ff9f938

• Add __version__ to fv3net init

  nbren12

  committed
  b31a099

  Copy full SHA for b31a099

• update prognostic_run_diags configuration

  nbren12

  committed
  8f9cb3c

  Copy full SHA for 8f9cb3c

• black

  nbren12

  committed
  31930d9

  Copy full SHA for 31930d9

• update readme

  nbren12

  committed
  f994766

  Copy full SHA for f994766

• Fix table in README

  nbren12

  committed
  871be05

  Copy full SHA for 871be05

• fix yaml bug

  nbren12

  committed
  db16a39

  Copy full SHA for db16a39

• pin pandas version to 1.0.1

  nbren12

  committed
  47d69e3

  Copy full SHA for 47d69e3

Commits on Mar 18, 2020

• save and process outputs at different stages

  nbren12

  committed
  cfe4523

  Copy full SHA for cfe4523

• post process runfile

  nbren12

  committed
  1b120ad

  Copy full SHA for 1b120ad

• Initialize the zarr store

  nbren12

  committed
  61c2036

  Copy full SHA for 61c2036

• write code to insert output in zarr store

  nbren12

  committed
  46b4f0f

  Copy full SHA for 46b4f0f

• get data for one time-step saved in the cloud

  nbren12

  committed
  f6827c0

  Copy full SHA for f6827c0

• test for full set of steps

  nbren12

  committed
  7eb6314

  Copy full SHA for 7eb6314

Commits on Mar 19, 2020

• update fv3config submodule

  nbren12

  committed
  2981824

  Copy full SHA for 2981824

• update fv3config to master

  nbren12

  committed
  3f6edda

  Copy full SHA for 3f6edda

• Refactor config generation

  Show description for a735a2e
  nbren12

  committed
  a735a2e

  Copy full SHA for a735a2e

• print more info in zarr_stat

  nbren12

  committed
  8a7a49c

  Copy full SHA for 8a7a49c

• remove some files

  nbren12

  committed
  ff62ff3

  Copy full SHA for ff62ff3

• Separate kube_config and fv3config handling

  Show description for a152ed7
  nbren12

  committed
  a152ed7

  Copy full SHA for a152ed7

• black and add workflow to job lables

  nbren12

  committed
  e397c13

  Copy full SHA for e397c13

• linter

  nbren12

  committed
  71f7300

  Copy full SHA for 71f7300

• save coordinate information and attrs to the big zarr

  nbren12

  committed
  74d5c9b

  Copy full SHA for 74d5c9b

• add coordinate info in runfile

  nbren12

  committed
  27cb13d

  Copy full SHA for 27cb13d

• debug

  nbren12

  committed
  b8a9c04

  Copy full SHA for b8a9c04

• move writing into runfile

  nbren12

  committed
  0cf8cff

  Copy full SHA for 0cf8cff

• black

  nbren12

  committed
  8f443f7

  Copy full SHA for 8f443f7

• log zarr creation stuff

  nbren12

  committed
  fdac3bc

  Copy full SHA for fdac3bc

• initialize array with timesteps

  nbren12

  committed
  c6ad40b

  Copy full SHA for c6ad40b

• save many more outputs

  nbren12

  committed
  b7de07f

  Copy full SHA for b7de07f

• parallelize opening writing across variables

  nbren12

  committed
  32ff298

  Copy full SHA for 32ff298

Commits on Mar 20, 2020

• write data with only the master thread

  nbren12

  committed
  c8bff73

  Copy full SHA for c8bff73

• debug out of memory error

  nbren12

  committed
  cad7e08

  Copy full SHA for cad7e08

• fix store call

  Show description for 49334f5
  nbren12

  committed
  49334f5

  Copy full SHA for 49334f5

• change versioning info in makefile

  nbren12

  committed
  d0c77d5

  Copy full SHA for d0c77d5

• add tracer variables

  nbren12

  committed
  cfed20f

  Copy full SHA for cfed20f

• change image version and chunking

  nbren12

  committed
  ae2e383

  Copy full SHA for ae2e383

• change chunks size

  nbren12

  committed
  97e4627

  Copy full SHA for 97e4627

• fix out of memory errors

  nbren12

  committed
  34426c0

  Copy full SHA for 34426c0

• show dataset size in summary script

  nbren12

  committed
  fba67f5

  Copy full SHA for fba67f5

Commits on Mar 23, 2020

• Add surface variables

  nbren12

  committed
  4da489b

  Copy full SHA for 4da489b

Commits on Mar 24, 2020

• merge in two d data

  nbren12

  committed
  dc1f502

  Copy full SHA for dc1f502

• reformat

  nbren12

  committed
  669df7e

  Copy full SHA for 669df7e

• Merge branch 'master' into feature/one-step-save-baseline

  nbren12

  committed
  5a2d1a1

  Copy full SHA for 5a2d1a1

• remove get_runfile_config

  nbren12

  committed
  6f698bc

  Copy full SHA for 6f698bc

• update fv3config to master

  nbren12

  committed
  56fe153

  Copy full SHA for 56fe153

• fix time dimension

  nbren12

  committed
  e57c543

  Copy full SHA for e57c543

• add test of runfile

  nbren12

  committed
  b85d0e9

  Copy full SHA for b85d0e9

• comment on need for fill_value=Nan

  nbren12

  committed
  92f2353

  Copy full SHA for 92f2353

• black

  nbren12

  committed
  1bc45b5

  Copy full SHA for 1bc45b5

• add provenance information to file

  nbren12

  committed
  b72a670

  Copy full SHA for b72a670

Commits on Mar 25, 2020

• add git provenance info

  nbren12

  committed
  08a7dc5

  Copy full SHA for 08a7dc5

• change order of arguments following upstream changes

  nbren12

  committed
  f2a4f0a

  Copy full SHA for f2a4f0a

• lint

  nbren12

  committed
  7a0d7b0

  Copy full SHA for 7a0d7b0

• fix runfile

  nbren12

  committed
  09cc500

  Copy full SHA for 09cc500

• comment out git logging, discovering the reop fails

  nbren12

  committed
  669bc68

  Copy full SHA for 669bc68

• another runfile fix

  nbren12

  committed
  513562d

  Copy full SHA for 513562d

• another bug 30 mins later...ugh

  nbren12

  committed
  cf0e5ff

  Copy full SHA for cf0e5ff

• delete submit_jobs.py

  Show description for f685dc9
  nbren12

  committed
  f685dc9

  Copy full SHA for f685dc9

• add some type hints

  nbren12

  committed
  597aab2

  Copy full SHA for 597aab2

• lint

  nbren12

  committed
  732128f

  Copy full SHA for 732128f

• Merge branch 'develop-one-steps' into feature/one-step-save-baseline

  nbren12

  committed
  e524d4e

  Copy full SHA for e524d4e

Commits on Mar 26, 2020

• unify the naming of the monitors and step names

  nbren12

  committed
  558011d

  Copy full SHA for 558011d

• Add out of memory troubleshooting info

  nbren12

  committed
  9f41b4b

  Copy full SHA for 9f41b4b

• Update info about submission

  nbren12

  committed
  a962ef9

  Copy full SHA for a962ef9

• add comment clarifying the local upload dir

  nbren12

  committed
  0e537bb

  Copy full SHA for 0e537bb

• lint

  nbren12

  committed
  3805291

  Copy full SHA for 3805291

• fix typo

  nbren12

  committed
  9873e54

  Copy full SHA for 9873e54

• Fix another bug

  nbren12

  committed
  eb11110

  Copy full SHA for eb11110

• another typo

  nbren12

  committed
  3dd8cf0

  Copy full SHA for 3dd8cf0

• fix another bug

  nbren12

  committed
  c144a5b

  Copy full SHA for c144a5b

• fix key

  nbren12

  committed
  2c5f2fe

  Copy full SHA for 2c5f2fe

• pass index to write_zarr_store

  nbren12

  committed
  643757f

  Copy full SHA for 643757f

• remove prototyping functions

  nbren12

  committed
  d40b044

  Copy full SHA for d40b044

• lint

  nbren12

  committed
  4a953da

  Copy full SHA for 4a953da

• bake the runfile into the submission script

  nbren12

  committed
  92df973

  Copy full SHA for 92df973

• print logging information

  nbren12

  committed
  d4fa24f

  Copy full SHA for d4fa24f

• lint

  nbren12

  committed
  71cc1ec

  Copy full SHA for 71cc1ec

• update yaml with brian's code

  nbren12

  committed
  5a6fa36

  Copy full SHA for 5a6fa36