Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Main module blending #233

Merged
merged 96 commits into from
Jan 14, 2022
Merged

Commits on Apr 3, 2019

  1. Configuration menu
    Copy the full SHA
    b01fac3 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2019

  1. Configuration menu
    Copy the full SHA
    236fb2e View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2021

  1. Configuration menu
    Copy the full SHA
    d38e26d View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2021

  1. Configuration menu
    Copy the full SHA
    92b8ecc View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2021

  1. pysteps.io with xarray (pySTEPS#219)

    * Add xarray dependency
    
    * MCH importer returns an xarray Dataset
    
    * Remove plot lines
    
    * Remove import
    
    * Adapt readers to xarray format
    
    * Rewrite as more general decorator
    
    * Add missing metadata
    
    * Adapt io tests
    
    * Mrms bounding box (pySTEPS#222)
    
    * Fix bounding box coordinates
    
    * Add missing metadata
    
    * Import xarray DataArray
    
    Ignore quality field
    
    * Black
    
    * Do not hardcode metadata
    
    * Address review comments by ruben
    
    * Add a legacy option to the io functions
    
    A "legacy" options is added to revert back the importers and readers behavior to version 1. This is a temporary solution to allow the examples, and other functions, to run as usual (v1.*).
    
    Hopefully, this is will allow a easier transition into version 2 during the development process and will allow testing functions that were not updated to v2.
    
    * Fix compatibility problems with tests
    
    Many of the tests were updated to use the legacy data structures (v1). The tests that still contains issues, were tagged with a TODO message and they are skipped.
    
    This will allow incremental changes to be tested in the new v2.
    
    IMPORTANT: once the v2 branch is stable, we may remove the legacy compatibility from the code base and the tests.
    
    * Update dependencies
    
    * Ignore plugins test
    
    Co-authored-by: Andres Perez Hortal <16256571+aperezhortal@users.noreply.github.com>
    dnerini and aperezhortal committed Aug 4, 2021
    Configuration menu
    Copy the full SHA
    0082738 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2021

  1. Configuration menu
    Copy the full SHA
    3802f8d View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2021

  1. Configuration menu
    Copy the full SHA
    f5750b0 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2021

  1. Add blend_optical_flow

    cvelascof committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    b58d806 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    6b1de65 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    a568e4a View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    fff9c7e View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    b209139 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2021

  1. Configuration menu
    Copy the full SHA
    1552a47 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0c3a260 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    912282b View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    5209631 View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2021

  1. Configuration menu
    Copy the full SHA
    2c2b807 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    29a6450 View commit details
    Browse the repository at this point in the history
  3. Change back to original BPS2006 blending formulation and add regressi…

    …on of skill values to climatological values for weights determination
    RubenImhoff committed Aug 12, 2021
    Configuration menu
    Copy the full SHA
    ce46582 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    193d90f View commit details
    Browse the repository at this point in the history
  5. Reformat code with Black

    RubenImhoff committed Aug 12, 2021
    Configuration menu
    Copy the full SHA
    1b15c4b View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    94995cf View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2021

  1. Configuration menu
    Copy the full SHA
    add0931 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    029a9a9 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    79a4e61 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    624bc7c View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2021

  1. Configuration menu
    Copy the full SHA
    694c9d3 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    6e3db59 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    6feec67 View commit details
    Browse the repository at this point in the history
  4. Small change to docstring

    RubenImhoff committed Aug 16, 2021
    Configuration menu
    Copy the full SHA
    1fe5ff3 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    2dfd531 View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2021

  1. Bom import xarray (pySTEPS#228)

    * Add import_bom_rf3  using xarray
    
    * Add tests to xarray version
    
    * Fix mrms importer tests
    
    * Pass **kwargs to internal functions
    
    * Add nwp_importers to read bom nwp sample data
    
    * Add bom nwp data to source file
    
    * Add tests for bom_nwp reader
    
    * Fix pystepsrc
    
    Co-authored-by: Andres Perez Hortal <16256571+aperezhortal@users.noreply.github.com>
    cvelascof and aperezhortal committed Aug 17, 2021
    Configuration menu
    Copy the full SHA
    2b8ee58 View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2021

  1. Functions to store and compute climatological weights (pySTEPS#231)

    * Implement the functions get_default_weights, save_weights, calc_clim_weights. 
    
    These functions are used to evolve the weights in the scale- and skill-dependent blending with NWP in the STEPS blending algorithm. The current weights, based on the correlations per cascade level, are regressed towards these climatological weights in the course of the forecast.
    
    These functions save the current and compute the climatological weights (a running mean of the weights of the past n days, where typically n=30). First daily averages are stored and these are then averaged over the running window of n days.
    
    * Add tests for pysteps climatological weight io and calculations.
    
    * Add path_workdir to outputs section in pystepsrc file and use it as a default path to store/retrieve blending weights.
    
    * Minor changes to docstrings, changes to skill scores and testing scripts
    
    * Completed documentation for blending clim module, cleanup.
    
    Co-authored-by: RubenImhoff <r.o.imhoff@live.nl>
    ladc and RubenImhoff committed Aug 18, 2021
    Configuration menu
    Copy the full SHA
    c6368ab View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    4b2ca89 View commit details
    Browse the repository at this point in the history

Commits on Aug 24, 2021

  1. Configuration menu
    Copy the full SHA
    ec65725 View commit details
    Browse the repository at this point in the history
  2. Add simple tests

    RubenImhoff committed Aug 24, 2021
    Configuration menu
    Copy the full SHA
    3fa773d View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    f9ed749 View commit details
    Browse the repository at this point in the history

Commits on Aug 25, 2021

  1. Add utilities to decompose, store and load NWP cascades for use in bl…

    …ending (pySTEPS#232)
    
    * First version of NWP decomposition
    
    * Added saving to netCDF
    
    * Completed functions for saving and loading decomposed NWP data
    
    * Added example files for the decomposed NWP functions
    
    * Added compatibility with numpy datetime
    
    * Use default output path_workdir for tmp files in blending/utils.py.
    
    * Update documentation of NWP decomposition functions in utils.py
    
    Co-authored-by: Wout Dewettinck <wout.dewettinck@ugent.be>
    Co-authored-by: wdewettin <87696913+wdewettin@users.noreply.github.com>
    3 people committed Aug 25, 2021
    Configuration menu
    Copy the full SHA
    81e88fe View commit details
    Browse the repository at this point in the history
  2. Add importer for RMI NWP data (pySTEPS#234)

    Add importer for netcdf NWP data from RMI using xarrays.
    
    * Add test for RMI NWP data importer.
    
    * Add entry for RMI NWP data in pystepsrc.
    
    * Run black on everything: fix formatting.
    ladc committed Aug 25, 2021
    Configuration menu
    Copy the full SHA
    56bd0e9 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    e39e025 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    d86cd7b View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    f9f09ae View commit details
    Browse the repository at this point in the history
  6. Changes to v_models to make use of multiple timesteps. Changes in the…

    … velocity field over time in the NWP forecast will be taken into account now.
    RubenImhoff committed Aug 25, 2021
    Configuration menu
    Copy the full SHA
    296ef53 View commit details
    Browse the repository at this point in the history
  7. Fixes for KNMI importer:

    Add forgotten @postprocess_import()
    Don't call dropna on NWP data.
    ladc committed Aug 25, 2021
    Configuration menu
    Copy the full SHA
    4d1fb51 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    779ff18 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2021

  1. Configuration menu
    Copy the full SHA
    e9059cb View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    cf2d585 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2021

  1. Configuration menu
    Copy the full SHA
    7e30217 View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2021

  1. Configuration menu
    Copy the full SHA
    40bdd69 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    01ef1e6 View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2021

  1. Cleanup imports

    dnerini committed Sep 15, 2021
    Configuration menu
    Copy the full SHA
    fb3ab19 View commit details
    Browse the repository at this point in the history
  2. More cleanup

    dnerini committed Sep 15, 2021
    Configuration menu
    Copy the full SHA
    5566c1e View commit details
    Browse the repository at this point in the history
  3. Update docstrings

    dnerini committed Sep 15, 2021
    Configuration menu
    Copy the full SHA
    5c26e10 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2021

  1. Merge pull request #11 from pySTEPS/steps_blending

    Merge steps_blending into local branch
    RubenImhoff committed Sep 28, 2021
    Configuration menu
    Copy the full SHA
    a763b59 View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2021

  1. Configuration menu
    Copy the full SHA
    8ddeeb2 View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2021

  1. Configuration menu
    Copy the full SHA
    a101e55 View commit details
    Browse the repository at this point in the history
  2. Reprojection functionality (pySTEPS#236)

    * Added Lesley's reprojection module to this branch
    
    * Added compatibility for three-dimensional xarrays
    
    * Add commentary to reprojection util
    
    * Changes to make reprojection of KNMI data possible
    
    * Changes after Daniele's review
    
    * Add dependencies
    
    * Changes to importers, see issue pySTEPS#215
    
    * Add tests
    
    * Fix some issues
    
    * documentation
    
    * Fixes for tests
    
    * Set requirements again
    
    * Some fixes
    
    * Changes to nwp_importers after Carlos' response
    
    * Remove wrong example script
    
    * Remove rasterio dependencies from lists
    
    * First try to prevent testing error
    
    * Changes Daniele and fix knmi nwp importer
    
    * Add rasterio to tox.ini
    
    * Aesthetics
    
    * rasterio import test
    
    * Add rasterio to the test dependencies
    
    * Reset try-except functionality for rasterio import
    
    * Fix for failing test on windows python 3.6
    
    * add importerskip rasterio
    
    Co-authored-by: Wout Dewettinck <wout.dewettinck@ugent.be>
    RubenImhoff and wdewettin committed Oct 5, 2021
    Configuration menu
    Copy the full SHA
    bccb8fc View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2021

  1. Configuration menu
    Copy the full SHA
    d1b1ac4 View commit details
    Browse the repository at this point in the history
  2. Merge pull request #12 from pySTEPS/pysteps-v2

    Merge Pysteps v2 in branch
    RubenImhoff committed Oct 6, 2021
    Configuration menu
    Copy the full SHA
    9ff3970 View commit details
    Browse the repository at this point in the history
  3. Fixes in nwp importers

    RubenImhoff committed Oct 6, 2021
    Configuration menu
    Copy the full SHA
    ce65906 View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2021

  1. Configuration menu
    Copy the full SHA
    2c639f8 View commit details
    Browse the repository at this point in the history
  2. Revert "Merge branch 'steps_blending' into pysteps-v2" (pySTEPS#239)

    This reverts commit 2c639f8, reversing
    changes made to bccb8fc.
    RubenImhoff committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    a342b02 View commit details
    Browse the repository at this point in the history
  3. Merge latest version pysteps-v2 into steps_blending branch (pySTEPS#237)

    * Update docstrings
    
    * More cleanup
    
    * Cleanup imports
    
    * Cleanup imports
    
    * More cleanup
    
    * Update docstrings
    
    * Update references
    
    Mention the work of Ravuri et al (2021, Nature) as an example of work using cGANs to generate ensembles
    
    * Clean up page
    
    * Reprojection functionality (pySTEPS#236)
    
    * Added Lesley's reprojection module to this branch
    
    * Added compatibility for three-dimensional xarrays
    
    * Add commentary to reprojection util
    
    * Changes to make reprojection of KNMI data possible
    
    * Changes after Daniele's review
    
    * Add dependencies
    
    * Changes to importers, see issue pySTEPS#215
    
    * Add tests
    
    * Fix some issues
    
    * documentation
    
    * Fixes for tests
    
    * Set requirements again
    
    * Some fixes
    
    * Changes to nwp_importers after Carlos' response
    
    * Remove wrong example script
    
    * Remove rasterio dependencies from lists
    
    * First try to prevent testing error
    
    * Changes Daniele and fix knmi nwp importer
    
    * Add rasterio to tox.ini
    
    * Aesthetics
    
    * rasterio import test
    
    * Add rasterio to the test dependencies
    
    * Reset try-except functionality for rasterio import
    
    * Fix for failing test on windows python 3.6
    
    * add importerskip rasterio
    
    Co-authored-by: Wout Dewettinck <wout.dewettinck@ugent.be>
    
    * Revert "Merge branch 'steps_blending' into pysteps-v2" (pySTEPS#239)
    
    This reverts commit 2c639f8, reversing
    changes made to bccb8fc.
    
    Co-authored-by: ned <daniele.nerini@meteoswiss.ch>
    Co-authored-by: dnerini <daniele.nerini@gmail.com>
    Co-authored-by: Wout Dewettinck <wout.dewettinck@ugent.be>
    4 people committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    82a9d18 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    cfcb572 View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2021

  1. Configuration menu
    Copy the full SHA
    b432013 View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2021

  1. Update docs

    RubenImhoff committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    c831878 View commit details
    Browse the repository at this point in the history
  2. resolve conflicts

    RubenImhoff committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    64a0a37 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    85807ca View commit details
    Browse the repository at this point in the history
  4. Fix docstrings example

    RubenImhoff committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    faa8396 View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2021

  1. Configuration menu
    Copy the full SHA
    671592b View commit details
    Browse the repository at this point in the history
  2. Fixes for the tests

    RubenImhoff committed Oct 19, 2021
    Configuration menu
    Copy the full SHA
    41f69b7 View commit details
    Browse the repository at this point in the history
  3. update docs

    RubenImhoff committed Oct 19, 2021
    Configuration menu
    Copy the full SHA
    b367608 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2021

  1. Configuration menu
    Copy the full SHA
    7648372 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2021

  1. Update contributing guidelines (pySTEPS#241)

    - Improve grammar.
    - Make the guide more concise. Remove unused/unnecessary rules.
    - Indicate more clearly which parts of the guidelines are inspired by other projects (before they were only mentioned at the end).
    - Change "Travis-CI" references by "GitHub Actions".
    aperezhortal committed Oct 25, 2021
    Configuration menu
    Copy the full SHA
    9b72689 View commit details
    Browse the repository at this point in the history
  2. Advect noise cascade

    RubenImhoff committed Oct 25, 2021
    Configuration menu
    Copy the full SHA
    9aba683 View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2021

  1. Configuration menu
    Copy the full SHA
    048fd88 View commit details
    Browse the repository at this point in the history
  2. minor fixes

    RubenImhoff committed Nov 2, 2021
    Configuration menu
    Copy the full SHA
    1b8d952 View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2021

  1. Linear blending (pySTEPS#229)

    * Implemented linear blending function
    * Added example file and test
    * Added compatibility for NWP ensembles
    
    The PR is ready to go. Making the code xarray ready will be done in a separate PR. 
    
    Co-authored-by: RubenImhoff <r.o.imhoff@live.nl>
    wdewettin and RubenImhoff committed Nov 3, 2021
    Configuration menu
    Copy the full SHA
    d30b07e View commit details
    Browse the repository at this point in the history
  2. merge pysteps-v2

    RubenImhoff committed Nov 3, 2021
    Configuration menu
    Copy the full SHA
    63ec111 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    053edf4 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    dabd10b View commit details
    Browse the repository at this point in the history

Commits on Nov 17, 2021

  1. Configuration menu
    Copy the full SHA
    ae6b826 View commit details
    Browse the repository at this point in the history

Commits on Dec 2, 2021

  1. Configuration menu
    Copy the full SHA
    d8ba4ea View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    daf1998 View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2021

  1. Configuration menu
    Copy the full SHA
    4479e69 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    ac4363b View commit details
    Browse the repository at this point in the history

Commits on Dec 10, 2021

  1. Configuration menu
    Copy the full SHA
    60cc1ea View commit details
    Browse the repository at this point in the history
  2. remove unnecessary lines

    RubenImhoff committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    dc05cb9 View commit details
    Browse the repository at this point in the history

Commits on Dec 16, 2021

  1. Configuration menu
    Copy the full SHA
    9ef4052 View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2021

  1. minor changes to blending climatology, blending weights and remove pa…

    …th_workdir from pystepsrc
    RubenImhoff committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    855935b View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    9391c90 View commit details
    Browse the repository at this point in the history

Commits on Dec 22, 2021

  1. Use pathlib

    dnerini committed Dec 22, 2021
    Configuration menu
    Copy the full SHA
    760f2bf View commit details
    Browse the repository at this point in the history
  2. Extract methods

    dnerini committed Dec 22, 2021
    Configuration menu
    Copy the full SHA
    b84191b View commit details
    Browse the repository at this point in the history

Commits on Jan 3, 2022

  1. Configuration menu
    Copy the full SHA
    a036e0f View commit details
    Browse the repository at this point in the history

Commits on Jan 10, 2022

  1. Merge pull request #13 from RubenImhoff/refactor-blending

    Refactoring of main steps blending code
    RubenImhoff committed Jan 10, 2022
    Configuration menu
    Copy the full SHA
    fd04a39 View commit details
    Browse the repository at this point in the history

Commits on Jan 13, 2022

  1. Access climatological skill file for multiple NWP model and date stri…

    …ng changes to prevent errors in blending.utils
    RubenImhoff committed Jan 13, 2022
    Configuration menu
    Copy the full SHA
    dddf02e View commit details
    Browse the repository at this point in the history