Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remaining reference to ARIMA in utils.py compute_std_err_from_median_variance_inflation_factor() #254

Closed
22 tasks
bikegeek opened this issue Jan 6, 2023 · 2 comments
Assignees
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle alert: NEED MORE DEFINITION Not yet actionable, additional definition required type: bug Fix something that is not working

Comments

@bikegeek
Copy link
Collaborator

bikegeek commented Jan 6, 2023

Describe the Problem

In METcalcpy/metcalcpy/util/utils.py, the compute_std_err_from_median_variance_inflation_factor function is referencing the ARIMA function from statsmodels (line 958):

    arima = ARIMA(data_excursions, order=(1, 0, 0))
    ar_1 = arima.fit().arparams[0]

Expected Behavior

If a user is invoking this function, there will be a module not found error (or something to that effect) due to no ARIMA

Environment

Describe your runtime environment:
1. Machine: (e.g. HPC name, Linux Workstation, Mac Laptop)
2. OS: (e.g. RedHat Linux, MacOS)
3. Software version number(s)

To Reproduce

Describe the steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
Post relevant sample data following these instructions:
https://dtcenter.org/community-code/model-evaluation-tools-met/met-help-desk#ftp

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Select Organization level Project for support of the current coordinated release
  • Select Repository level Project for development toward the next official release or add alert: NEED PROJECT ASSIGNMENT label
  • Select Milestone as the next bugfix version

Define Related Issue(s)

Consider the impact to the other METplus components.

Bugfix Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of main_<Version>.
    Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
  • Fix the bug and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into main_<Version>.
    Pull request: bugfix <Issue Number> main_<Version> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Development issues
    Select: Organization level software support Project for the current coordinated release
    Select: Milestone as the next bugfix version
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Complete the steps above to fix the bug on the develop branch.
    Branch name: bugfix_<Issue Number>_develop_<Description>
    Pull request: bugfix <Issue Number> develop <Description>
    Select: Reviewer(s) and Development issues
    Select: Repository level development cycle Project for the next official release
    Select: Milestone as the next official version
  • Close this issue.
@bikegeek bikegeek added type: bug Fix something that is not working alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED MORE DEFINITION Not yet actionable, additional definition required alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle labels Jan 6, 2023
@bikegeek bikegeek added this to the METcalcpy 2.0.0-bugfix milestone Jan 6, 2023
@bikegeek bikegeek modified the milestones: METcalcpy 2.0.1-bugfix, METcalcpy 2.1.0 beta1 Jan 31, 2023
@bikegeek
Copy link
Collaborator Author

following what was done in util.comput_std_err_from_mean():
replace this:

arima = ARIMA(data_excursions, order=(1, 0, 0))
ar_1 = arima.fit().arparams[0]

with this:
ar_1 = autocor_coef(data)

@bikegeek bikegeek self-assigned this Feb 3, 2023
@bikegeek bikegeek modified the milestones: METcalcpy 2.1.0 beta1, METcalcpy 2.1.0-beta2 Mar 1, 2023
bikegeek added a commit that referenced this issue Mar 3, 2023
…_err_from_median_variance_inflation_factor
@bikegeek
Copy link
Collaborator Author

bikegeek commented Mar 3, 2023

also removed code that is no longer used (needed when invoking ARIMA):

Use excursions from the median to compute the first order auto-correlation coefficient.

    data_excursions = list()
    median = st.median(data)
    for val in data:
        if val >= median:
            data_excursions.append(1)
        else:
            data_excursions.append(0)

    arima = ARIMA(data_excursions, order=(1, 0, 0))
    ar_1 = arima.fit().arparams[0]

replaced by:
ar_1 = autocor_coef(data)

bikegeek added a commit that referenced this issue Mar 3, 2023
…d_err_from_median_variance_inflation_factor function in util/utils.py
bikegeek added a commit that referenced this issue Mar 3, 2023
…o longer used in compute_std_err_from_median_variance_inflation_factor()
bikegeek added a commit that referenced this issue Mar 7, 2023
* Issue #254 use scorecard data stat_values for testing the compute_std_err_from_median_variance_inflation_factor

* Issue #254 test to verify ARIMA is no longer in use in the compute_std_err_from_median_variance_inflation_factor function in util/utils.py

* Issue #254 replace call to ARIMA and remove associated code that is no longer used in  compute_std_err_from_median_variance_inflation_factor()
@bikegeek bikegeek closed this as completed Mar 7, 2023
@JohnHalleyGotway JohnHalleyGotway modified the milestones: METcalcpy 2.1.0-beta2, METcalcpy 2.1.0 Mar 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle alert: NEED MORE DEFINITION Not yet actionable, additional definition required type: bug Fix something that is not working
Projects
No open projects
Status: Done
Development

When branches are created from issues, their pull requests are automatically linked.

2 participants