Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restore the time series to the variable summaries #408

Merged

Commits on Jun 16, 2023

  1. Update diag.history to read from parquet

    Update the tests and the implementation for retrieving historical data.
    In the process, I simplified the response type. I think, in the future,
    if we want to select unadjusted O - F or just the historical
    observations, we should build that into the URL, rather than sending all
    of the data back to the client and letting the client pick what to view.
    This cuts down on data transfer and allows us to simplify our components
    when they read data, since they don't have to parse a nested JavaScript
    object.
    esheehan-gsl committed Jun 16, 2023
    Configuration menu
    Copy the full SHA
    691c4cf View commit details
    Browse the repository at this point in the history
  2. Remove some unused code from the history implementation

    Removed some of the dataclasses and functions that had been part of the
    history API which are no longer used, now that we have the Parquet
    implementation.
    esheehan-gsl committed Jun 16, 2023
    Configuration menu
    Copy the full SHA
    4807605 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2023

  1. Test with a file:// prototocol

    The diag.history function fails when there's a file:// prefix in the
    diag file path because apparently Pandas gets a little confused and
    tries to pass it to urllib instead of just opening the file.
    esheehan-gsl committed Jun 20, 2023
    Configuration menu
    Copy the full SHA
    0b984f2 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2023

  1. Revert "Remove the time series for now"

    This reverts commit 716a400.
    esheehan-gsl committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    6e295ed View commit details
    Browse the repository at this point in the history
  2. Use env variable to find Parquet files in routes

    Instead of deriving the Parquet file path from the Zarr path, the route
    reads in the FLASK_DIAG_PARQUET variable from the environment and passes
    that explicitly to diag.history. diag.history uses that to find the
    Parquet file for this model, and strips out `file://` if it's present
    because Pandas is a bit silly about that.
    esheehan-gsl committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    b480a0d View commit details
    Browse the repository at this point in the history
  3. Update property access for time series charts

    The charts really need to be refactored (in part) to eliminate these
    hard-coded property accesses, but for now we can just update the
    properties that the time series component expects so that we can include
    test out the time series data.
    esheehan-gsl committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    13bdd7f View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2023

  1. Configuration menu
    Copy the full SHA
    79aa304 View commit details
    Browse the repository at this point in the history
  2. Use tmp_path to store parquet fixtures

    Instead of deriving the path to save parquet fixtures for tests, we use
    tmp_path, basically the same way we would for production.
    esheehan-gsl committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    9237851 View commit details
    Browse the repository at this point in the history
  3. Strip file protocol from diag zarr config

    It seems like pathlib.Path is having an issue with the `file://`
    protocol on our environment variables, although I don't think it used
    to. It's easy enough to strip this, although this may require more
    robust path/uri handling in our tests.
    esheehan-gsl committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    6cf5524 View commit details
    Browse the repository at this point in the history
  4. Fix data type for is_used in test fixtures

    When we process diag files in our pipeline, we convert the integers in
    the diag files for is_used into booleans, so should treat them the same
    way in our test fixtures. I think this is a sign that our test fixtures
    are poorly set up, since they can get out of sync with reality. The
    result of this problem was that either the application worked, or our
    tests passed. Prior to this change, we needed to compare is_used to a
    boolean for Parquet files generated with our ETL code, but to an integer
    for test files generated with our fixtures.
    esheehan-gsl committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    5d93be0 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    fbb6527 View commit details
    Browse the repository at this point in the history