Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix: Reconcile setting of METPLOTPY_BASE for use cases #1713

Closed
4 of 27 tasks
georgemccabe opened this issue Jul 26, 2022 · 0 comments · Fixed by #1715 or #1716
Closed
4 of 27 tasks

Bugfix: Reconcile setting of METPLOTPY_BASE for use cases #1713

georgemccabe opened this issue Jul 26, 2022 · 0 comments · Fixed by #1715 or #1716
Assignees
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED MORE DEFINITION Not yet actionable, additional definition required component: build process Build process issue METplus: Configuration priority: blocker Blocker reporting: DTC NCAR Base NCAR Base DTC Project reporting: DTC NOAA BASE NOAA Office of Atmospheric Research DTC Project requestor: METplus Team METplus Development Team requestor: NOAA/EMC NOAA Environmental Modeling Center required: FOR DEVELOPMENT RELEASE Required to be completed in the development release for the assigned project type: bug Fix something that is not working

Comments

@georgemccabe
Copy link
Collaborator

georgemccabe commented Jul 26, 2022

This was discovered in discussion #1703. The Hovmoeller use case failed to run on Hera due to the current approach.

Currently the METPLOTPY_BASE environment variable is set in 2 use case config files.

s2s/UserScript_obsPrecip_obsOnly_Hovmoeller.conf:
METPLOTPY_BASE = {METPLUS_BASE}/../METplotpy

s2s/UserScript_obsPrecip_obsOnly_CrossSpectraPlot.conf:
METPLOTPY_BASE = {METPLUS_BASE}/METplotpy

This may not always be the location of the METplotpy install. Also, the information is inconsistent between the two. Apparently it is not actually used for the CrossSpectraPlot use case.

Describe the Problem

Come up with a better way to set this environment variable on install so that it is consistent for all use cases.

  • Remove setting of env var from individual use case config files
  • For the automated tests, this could be set via [user_env_vars] in the .github/parm/test_settings.conf file so it is used for all use cases:
[user_env_vars]
METPLOTPY_BASE = {METPLUS_BASE}/../METplotpy
  • Consider adding the [user_env_vars] section and the variable commented out in the parm/metplus_config/defaults.conf file with instructions to uncomment and set to the appropriate location upon install
  • On Hera, the environment variable is set in the module load script for METplotpy, so it doesn't need to be set in METplus defaults.conf. This would allow users to easily load a different METplotpy module and METplus would use that version. Variables set in [user_env_vars] take precedence over environment variables set in the user's environment.

Expected Behavior

Use cases will run properly regardless of location, i.e. Hera

Environment

Describe your runtime environment:
1. Machine: Hera

To Reproduce

Describe the steps to reproduce the behavior:
1. Log into Hera
2. Run the module load commands to obtain METplus, METplotpy, METcalcpy, and METdatadb (METdataio)
3. Run the Hovmoeller use case and confirm it fails

Note: I commented out setting METPLOTPY_BASE in the use case config file for METplus-4.1.0 so other users could run the use case. To test the original behavior, copy the conf file, uncomment the variable, and run the use case.

Relevant Deadlines

5.0.0 and 4.1.3

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Select Organization level Project for support of the current coordinated release
  • Select Repository level Project for development toward the next official release or add alert: NEED PROJECT ASSIGNMENT label
  • Select Milestone as the next bugfix version

Define Related Issue(s)

Consider the impact to the other METplus components.

Bugfix Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of main_<Version>.
    Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
  • Fix the bug and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Add any new Python packages to the METplus Components Python Requirements table.
  • Push local changes to GitHub.
  • Submit a pull request to merge into main_<Version>.
    Pull request: bugfix <Issue Number> main_<Version> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Linked issues
    Select: Organization level software support Project for the current coordinated release
    Select: Milestone as the next bugfix version
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Complete the steps above to fix the bug on the develop branch.
    Branch name: bugfix_<Issue Number>_develop_<Description>
    Pull request: bugfix <Issue Number> develop <Description>
    Select: Reviewer(s) and Linked issues
    Select: Repository level development cycle Project for the next official release
    Select: Milestone as the next official version
  • Close this issue.
@georgemccabe georgemccabe added type: bug Fix something that is not working priority: blocker Blocker requestor: NOAA/EMC NOAA Environmental Modeling Center alert: NEED MORE DEFINITION Not yet actionable, additional definition required alert: NEED ACCOUNT KEY Need to assign an account key to this issue component: build process Build process issue requestor: METplus Team METplus Development Team required: FOR DEVELOPMENT RELEASE Required to be completed in the development release for the assigned project METplus: Configuration labels Jul 26, 2022
@georgemccabe georgemccabe added this to the METplus-5.0.0 milestone Jul 26, 2022
@georgemccabe georgemccabe self-assigned this Jul 26, 2022
georgemccabe added a commit that referenced this issue Jul 26, 2022
…config file and instead set it in the global config settings for the automated test environment, ci-run-all-cases
georgemccabe added a commit that referenced this issue Jul 26, 2022
…s] in the default METplus config file upon install if necessary
georgemccabe added a commit that referenced this issue Jul 26, 2022
…config file and instead set it in the global config settings for the automated test environment, ci-run-all-cases
georgemccabe added a commit that referenced this issue Jul 26, 2022
…s] in the default METplus config file upon install if necessary
georgemccabe added a commit that referenced this issue Jul 26, 2022
georgemccabe added a commit that referenced this issue Jul 26, 2022
This was linked to pull requests Jul 26, 2022
bikegeek pushed a commit that referenced this issue Jul 27, 2022
* per #1713, remove setting of METPLOTPY_BASE env var in each use case config file and instead set it in the global config settings for the automated test environment, ci-run-all-cases

* per #1713, added instructions to set METPLOTPY_BASE via [user_env_vars] in the default METplus config file upon install if necessary

* per #1713, added documentation to describe METPLOTPY_BASE variable that was added to the defaults.conf file
bikegeek pushed a commit that referenced this issue Jul 27, 2022
* per #1713, remove setting of METPLOTPY_BASE env var in each use case config file and instead set it in the global config settings for the automated test environment, ci-run-all-cases

* per #1713, added instructions to set METPLOTPY_BASE via [user_env_vars] in the default METplus config file upon install if necessary

* per #1713, added documentation to describe METPLOTPY_BASE variable that was added to the defaults.conf file
@georgemccabe georgemccabe changed the title Reconcile setting of METPLOTPY_BASE for use cases Bugfix: Reconcile setting of METPLOTPY_BASE for use cases Jul 27, 2022
georgemccabe pushed a commit that referenced this issue Apr 6, 2023
Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
Co-authored-by: johnhg <johnhg@ucar.edu>
Co-authored-by: cristianastan2 <cstan@gmu.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: bikegeek <minnawin@ucar.edu>
Co-authored-by: Lisa Goodrich <lisag@ucar.edu>
Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: Hank Fisher <fisherh@ucar.edu>
Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: Tracy Hertneky <hertneky@seneca.rap.ucar.edu>
Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com>
Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com>
Co-authored-by: bikegeek <3753118+bikegeek@users.noreply.github.com>
Co-authored-by: j-opatz <jopatz@ucar.edu>
Co-authored-by: Will Mayfield <59745143+willmayfield@users.noreply.github.com>
Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
Co-authored-by: Tracy Hertneky <39317287+hertneky@users.noreply.github.com>
fixing some spacing #770
Fixing formatting, typos etc.  #770
Fixing formatting  #770
fixing typos, spacing, etc. #770
fixing typos, grammar and punctuation. #770
Fixing TOC and coding block #770
fixing note #770
fix.  Updating bolding and italics for file names and directories #770
fixing space #770
fixing conflict #2 #770
fixing typos #770
fixing typos #2 #770
fixing formatting #770
fixing errors #650
fixing errors take 2 #650
fixing errors take 3 #650
fixing errors take 4 #650
fixing errors take 5 #650
fixing question order #650
fix #1706 fix PhaseDiagram use case to avoid writing into INPUT_BASE (#1708)
fix #1713 develop METPLOTPY_BASE (#1715)
fix #1691 remove whitespace from output file paths (#1721)
fix Contributor's Guide GitHub Workflow page (#1774)
fix release (#1790)
fix GitHub Actions warnings (#1864)
fix #1884 develop PCPCombine {custom} in subtract method (#1887)
fix #1939 develop - failure reading obs when zipped file also exists (#1941)
Closes #1986
fix develop Fix broken documentation links (#2004)
fix #2026 develop StatAnalysis looping (#2028)
fix priority of obs_window config variables so that wrapper-specific version is preferred over generic OBS_WINDOW_BEGIN/END (#2062)
fix #2070 var list numeric order (#2072)
fix #2087 develop docs_pdf (#2091)
fix #2096/#2098 develop - fix skip if output exists and do not error if no commands were run (#2099)
georgemccabe pushed a commit that referenced this issue Apr 28, 2023
Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
Co-authored-by: johnhg <johnhg@ucar.edu>
Co-authored-by: cristianastan2 <cstan@gmu.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: bikegeek <minnawin@ucar.edu>
Co-authored-by: Lisa Goodrich <lisag@ucar.edu>
Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: Hank Fisher <fisherh@ucar.edu>
Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: Tracy Hertneky <hertneky@seneca.rap.ucar.edu>
Co-authored-by: Giovanni Rosa <giovanni.rosa@unimol.it>
Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com>
Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com>
Co-authored-by: bikegeek <3753118+bikegeek@users.noreply.github.com>
Co-authored-by: j-opatz <jopatz@ucar.edu>
Co-authored-by: Will Mayfield <59745143+willmayfield@users.noreply.github.com>
Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
Co-authored-by: Tracy Hertneky <39317287+hertneky@users.noreply.github.com>
Co-authored-by: Giovanni Rosa <g.rosa1@studenti.unimol.it>
fixing errors #650
fixing errors take 2 #650
fixing errors take 3 #650
fixing errors take 4 #650
fixing errors take 5 #650
fixing question order #650
fix #1706 fix PhaseDiagram use case to avoid writing into INPUT_BASE (#1708)
fix #1713 develop METPLOTPY_BASE (#1715)
fix #1691 remove whitespace from output file paths (#1721)
fix Contributor's Guide GitHub Workflow page (#1774)
fix release (#1790)
fix GitHub Actions warnings (#1864)
fix #1884 develop PCPCombine {custom} in subtract method (#1887)
fix #1939 develop - failure reading obs when zipped file also exists (#1941)
Closes #1986
fix develop Fix broken documentation links (#2004)
fix #2026 develop StatAnalysis looping (#2028)
fix priority of obs_window config variables so that wrapper-specific version is preferred over generic OBS_WINDOW_BEGIN/END (#2062)
fix #2070 var list numeric order (#2072)
fix #2087 develop docs_pdf (#2091)
fix #2096/#2098 develop - fix skip if output exists and do not error if no commands were run (#2099)
Fix for Dockerfile smell DL4000 (#2112)
fix #2082 develop regrid.convert/censor_thresh/censor_val (#2140)
fix #2082 main_v5.0 regrid.convert/censor_thresh/censor_val (#2101)
fix #2137 develop PointStat -obs_valid_beg/end (#2141)
JohnHalleyGotway pushed a commit that referenced this issue May 11, 2023
Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
Co-authored-by: johnhg <johnhg@ucar.edu>
Co-authored-by: cristianastan2 <cstan@gmu.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: bikegeek <minnawin@ucar.edu>
Co-authored-by: Lisa Goodrich <lisag@ucar.edu>
Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: Hank Fisher <fisherh@ucar.edu>
Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: Tracy Hertneky <hertneky@seneca.rap.ucar.edu>
Co-authored-by: Giovanni Rosa <giovanni.rosa@unimol.it>
Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com>
Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com>
Co-authored-by: bikegeek <3753118+bikegeek@users.noreply.github.com>
Co-authored-by: j-opatz <jopatz@ucar.edu>
Co-authored-by: Will Mayfield <59745143+willmayfield@users.noreply.github.com>
Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
Co-authored-by: Tracy Hertneky <39317287+hertneky@users.noreply.github.com>
Co-authored-by: Giovanni Rosa <g.rosa1@studenti.unimol.it>
fixing errors #650
fixing errors take 2 #650
fixing errors take 3 #650
fixing errors take 4 #650
fixing errors take 5 #650
fixing question order #650
fix #1706 fix PhaseDiagram use case to avoid writing into INPUT_BASE (#1708)
fix #1713 develop METPLOTPY_BASE (#1715)
fix #1691 remove whitespace from output file paths (#1721)
fix Contributor's Guide GitHub Workflow page (#1774)
fix release (#1790)
fix GitHub Actions warnings (#1864)
fix #1884 develop PCPCombine {custom} in subtract method (#1887)
fix #1939 develop - failure reading obs when zipped file also exists (#1941)
Closes #1986
fix develop Fix broken documentation links (#2004)
fix #2026 develop StatAnalysis looping (#2028)
fix priority of obs_window config variables so that wrapper-specific version is preferred over generic OBS_WINDOW_BEGIN/END (#2062)
fix #2070 var list numeric order (#2072)
fix #2087 develop docs_pdf (#2091)
fix #2096/#2098 develop - fix skip if output exists and do not error if no commands were run (#2099)
Fix for Dockerfile smell DL4000 (#2112)
fix #2082 develop regrid.convert/censor_thresh/censor_val (#2140)
fix #2082 main_v5.0 regrid.convert/censor_thresh/censor_val (#2101)
fix #2137 develop PointStat -obs_valid_beg/end (#2141)
fix failured introduced by urllib3 (see urllib3/urllib3#2168)
fix #2161 develop PCPCombine additional field arguments in -subtract mode (#2162)
georgemccabe pushed a commit that referenced this issue Jun 5, 2023
Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
Co-authored-by: johnhg <johnhg@ucar.edu>
Co-authored-by: cristianastan2 <cstan@gmu.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: bikegeek <minnawin@ucar.edu>
Co-authored-by: Lisa Goodrich <lisag@ucar.edu>
Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: Hank Fisher <fisherh@ucar.edu>
Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: Tracy Hertneky <hertneky@seneca.rap.ucar.edu>
Co-authored-by: Giovanni Rosa <giovanni.rosa@unimol.it>
Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com>
Co-authored-by: Mrinal Biswas <biswas@seneca.rap.ucar.edu>
Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com>
Co-authored-by: bikegeek <3753118+bikegeek@users.noreply.github.com>
Co-authored-by: j-opatz <jopatz@ucar.edu>
Co-authored-by: Will Mayfield <59745143+willmayfield@users.noreply.github.com>
Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
Co-authored-by: Tracy Hertneky <39317287+hertneky@users.noreply.github.com>
Co-authored-by: Giovanni Rosa <g.rosa1@studenti.unimol.it>
Co-authored-by: mrinalbiswas <biswas@ucar.edu>
fixing errors #650
fixing errors take 2 #650
fixing errors take 3 #650
fixing errors take 4 #650
fixing errors take 5 #650
fixing question order #650
fix #1706 fix PhaseDiagram use case to avoid writing into INPUT_BASE (#1708)
fix #1713 develop METPLOTPY_BASE (#1715)
fix #1691 remove whitespace from output file paths (#1721)
fix Contributor's Guide GitHub Workflow page (#1774)
fix release (#1790)
fix GitHub Actions warnings (#1864)
fix #1884 develop PCPCombine {custom} in subtract method (#1887)
fix #1939 develop - failure reading obs when zipped file also exists (#1941)
Closes #1986
fix develop Fix broken documentation links (#2004)
fix #2026 develop StatAnalysis looping (#2028)
fix priority of obs_window config variables so that wrapper-specific version is preferred over generic OBS_WINDOW_BEGIN/END (#2062)
fix #2070 var list numeric order (#2072)
fix #2087 develop docs_pdf (#2091)
fix #2096/#2098 develop - fix skip if output exists and do not error if no commands were run (#2099)
Fix for Dockerfile smell DL4000 (#2112)
fix #2082 develop regrid.convert/censor_thresh/censor_val (#2140)
fix #2082 main_v5.0 regrid.convert/censor_thresh/censor_val (#2101)
fix #2137 develop PointStat -obs_valid_beg/end (#2141)
fix failured introduced by urllib3 (see urllib3/urllib3#2168)
fix #2161 develop PCPCombine additional field arguments in -subtract mode (#2162)
fix #2168 develop - StatAnalysis time shift (#2169)
fix releases. (#2183)
fix #2189 develop - spaces in complex thresholds (#2191)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED MORE DEFINITION Not yet actionable, additional definition required component: build process Build process issue METplus: Configuration priority: blocker Blocker reporting: DTC NCAR Base NCAR Base DTC Project reporting: DTC NOAA BASE NOAA Office of Atmospheric Research DTC Project requestor: METplus Team METplus Development Team requestor: NOAA/EMC NOAA Environmental Modeling Center required: FOR DEVELOPMENT RELEASE Required to be completed in the development release for the assigned project type: bug Fix something that is not working
Projects
No open projects
1 participant