Rcv0 2 0 #30

rstoneback · 2022-02-18T00:39:41Z

Description

Addresses #3, #15, #28, #29

A large range of changes to get pysatSeasons generally up to the modern era.

Switched to workflows and pytest
Condensed testing
Added Consellation support to more functions, with testing
Added xarray support to functions, with testing
Removed deprecated library use
Improved docstrings and general style
Adopted updated setup.py and setup.cfg
Removed collections.deque use
computational_form -> to_xarray_dataset
Simplified code for seasonal averaging functions
Updated examples

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details for
your test configuration

Existing unit tests
Developed additional unit tests
Ran updated documentation examples

Test Configuration:

Operating system: MacOS
Version number: Python 3.9
Any details about your local setup that are relevant: pysat 'develop' branch and v3.0.1 release.

Checklist:

Update develop with all RC0.1.3 changes

jklenzing

Some updates to the core documents. We should probably finalize this soon, as dependabot is getting cranky about the numpy version cap in main.

.github/workflows/main.yml

jklenzing · 2022-08-02T15:17:09Z

CHANGELOG.md

@@ -2,6 +2,31 @@
 All notable changes to this project will be documented in this file.
 This project adheres to [Semantic Versioning](http://semver.org/).

+## [0.2.0] - 2022-06-10


make sure to update

README.md

CHANGELOG.md

Co-authored-by: Jeff Klenzing <19592220+jklenzing@users.noreply.github.com>

rstoneback · 2022-08-02T15:55:37Z

hmm........ we broke something

mkdir pysatData
[3](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:3)
  python -c "import pysat; pysat.params['data_dirs'] = 'pysatData'"
[4](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:4)
  shell: /usr/bin/bash -e {0}
[5](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:5)
  env:
[6](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:6)
    pythonLocation: /opt/hostedtoolcache/Python/3.8.13/x64
[7](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:7)
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.8.13/x64/lib
[8](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:9)
Traceback (most recent call last):
[9](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:10)
  File "<string>", line 1, in <module>
[10](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:11)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/pysat/__init__.py", line 49, in <module>
[11](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:12)
    from pysat import _params
[12](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:13)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/pysat/_params.py", line 13, in <module>
[13](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:14)
    import pysat.utils
[14](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:15)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/pysat/utils/__init__.py", line 9, in <module>
[15](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:16)
    from pysat.utils._core import available_instruments
[16](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:17)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/pysat/utils/_core.py", line 15, in <module>
[17](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:18)
    import xarray as xr
[18](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:19)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/xarray/__init__.py", line 1, in <module>
[19](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:20)
    from . import testing, tutorial
[20](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:21)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/xarray/testing.py", line 9, in <module>
[21](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:22)
    from xarray.core import duck_array_ops, formatting, utils
[22](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:23)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/xarray/core/duck_array_ops.py", line 26, in <module>
[23](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:24)
    from . import dask_array_compat, dask_array_ops, dtypes, npcompat, nputils
[24](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:25)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/xarray/core/npcompat.py", line 72, in <module>
[25](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:26)
    _SupportsDType[np.dtype],
[26](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:27)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/typing.py", line 261, in inner
[27](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:28)
    return func(*args, **kwds)
[28](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:29)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/typing.py", line 897, in __class_getitem__
[29](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:30)
    _check_generic(cls, params)
[30](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:31)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/typing.py", line 211, in _check_generic
[31](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:32)
    raise TypeError(f"{cls} is not a generic class")
[32](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:33)
TypeError: <class 'numpy.typing._dtype_like._SupportsDType'> is not a generic class
[33](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:34)
Error: Process completed with exit code 1.

rstoneback · 2022-08-04T20:02:41Z

Tests are passing with latest pysat.

jklenzing

I know I already approved, but this would reduce the meta deprecation warnings throughout.

pysatSeasons/tests/test_core.py

Co-authored-by: Jeff Klenzing <19592220+jklenzing@users.noreply.github.com>

aburrell · 2022-08-05T17:02:13Z

README.md

-The caps on numpy and python stem from compatibility with the maximum
-supported pandas version.  Version 0.2.0 will rewrite the routines to remove
-`Panel`.
+and for the Space Physics community.  

 | Common modules | Community modules |
 | -------------- | ----------------- |
 | matplotlib     | pysat             |


Should pysat be version limited?

Yes, after some digging I found the we need Constellation instantiation support given a list of instruments, which was added to pysat in v3.0.1. Tests were passing here with that pysat version. Added the req to requirements.txt and the readme.

demo/cosmic_and_ivm_demo.py

pysatSeasons/avg.py

Co-authored-by: Angeline Burrell <aburrell@users.noreply.github.com>

aburrell · 2022-08-09T00:01:12Z

README.md

-| numpy<1.20     |                   |
-| pandas<0.24    |                   |
-
+| matplotlib     | pysat >= 3.0.1    |


Seems like the safest thing? If not needed, that's fine.

Needed for the unit tests since we added the use_header kwarg to those. I don't think it's needed in the core code.

aburrell · 2022-08-09T00:02:17Z

requirements.txt

+pysat>=3.0.1
+xarray<2022.06.0


If going this direction, update the xarray limit in the README

aburrell · 2022-08-11T21:20:29Z

setup.cfg

+  numpy
+  pandas
+  pysat>=3.0.2
+  xarray<2022.06.0


There shouldn't be any version caps here

the version limits aren't applied, or checked, during installation without specifying them here

From docs on install_requires, https://packaging.python.org/en/latest/discussions/install-requires-vs-requirements/,

Additionally, it’s best practice to indicate any known lower or upper bounds.

The root version requirement on xarray should be placed on pysat, and re-released, so that the environments here work more out of the box.

From your link:

Lastly, it’s important to understand that install_requires is a listing of “Abstract” requirements, i.e just names and version restrictions that don’t determine where the dependencies will be fulfilled from (i.e. from what index or source). The where (i.e. how they are to be made “Concrete”) is to be determined at install time using pip options. 1

Requirements files
Requirements Files described most simply, are just a list of pip install arguments placed into a file.

Whereas install_requires defines the dependencies for a single project, Requirements Files are often used to define the requirements for a

While the limits don't appear to be breaking the setup the way that they used to, it still seems cleaner to use the requirements file for this. Might be best to discuss our standards at the next meeting.

From your link:

Lastly, it’s important to understand that install_requires is a listing of “Abstract” requirements, i.e just names and version restrictions

While the limits don't appear to be breaking the setup the way that they used to, it still seems cleaner to use the requirements file for this. Might be best to discuss our standards at the next meeting.

I'll put it on the agenda for Wed. though it is possible I may not be available. Government depending.

I checked the linked discussion from pysatSpaceWeather: pypa/setuptools#1951

This particular comment notes the anti-pattern is loading requirements.txt into install_requires. Further notes that the biggest issue with loading requirements.txt is the potential for pinned versions set with a ==. Notes a compromise would be to allow requirements.txt to be loaded into install_requires as long as pinned versions aren't used.
pypa/setuptools#1951 (comment)

The python docs https://packaging.python.org/en/latest/discussions/install-requires-vs-requirements/ say

install_requires is a [setuptools](https://packaging.python.org/en/latest/key_projects/#setuptools) setup.py keyword that should be used to specify what a project minimally needs to run correctly. When the project is installed by [pip](https://packaging.python.org/en/latest/key_projects/#pip), this is the specification that is used to install its dependencies.

I checked to see what was above this doc for greater context to see if this install_requires discussion was limited to a particular sub-situation but didn't see anything.

From what I've read setting minimal version requirements in install_requires in setup.cfg for pip installs is the correct thing to do. pip uses the information from setup to determine dependencies. For user pip installs of pysatPenumbra packages to work reliably we should specify a minimum pysat version in setup, 3.0.4, when that version is released. A similar condition would also be in requirements.txt, but strictly speaking the requirements and setup restrictions are there for different reasons thus may not always be the same. requirements.txt is not loaded into install_requires.

I'm not opposed to using a .toml file but I don't understand why that is better.

I think it comes down to the idea that the format of a cfg is loosely defined. See https://peps.python.org/pep-0518/#sticking-with-setup-cfg for why toml was recommended above cfg

rstoneback · 2022-08-17T19:47:18Z

check if matplotlib dependence is real

rstoneback · 2022-08-31T22:41:19Z

Tests stopped working due to some kind of numpy and xarray issue. Not knowing what is going on, I bumped the numpy version up to 1.21....
https://github.com/pysat/pysatSeasons/runs/8124117985?check_suite_focus=true

How is it that working software always breaks just before release? 🤷

rstoneback · 2022-09-01T01:14:37Z

Now a lone windows test failure from matplotlib with the suggestion that perhaps Tcl wasn't installed properly.

jklenzing and others added 30 commits June 15, 2021 14:27

Merge pull request #25 from pysat/main

fa836a9

Update develop with all RC0.1.3 changes

STY: Improve clarity of iteration Instruments

7e044bb

STY: Comments pass

e7437a7

ENH: Remove pandas version limit

212cfe5

ENH: First pass Panel -> Dataset

fe04c05

ENH: First pass Panel -> Dataset

d596c76

DOC: Updated changelog

720269f

STY: Moved file docstring

bd7c7ea

ENH: Refactor computational_form, return xr.DataSets

719639b

STY: Generalize

982476b

TST: Add tests for computational_form

8567953

BUG: Update test function name

51dbf05

TST: tweak

c005fb6

BUG: Better pysat compatibility

0e72bdd

BUG: Corrected import

182a43e

STY: Tweaks

cc81b10

TST/BUG: Fixed use of inst_module and added test for list of numbers.

f184478

STY: pep8 whitespace

86de9ce

STY: pep8 whitespace

fc68da1

TST: Remove commented test.

d44aa63

STY: Placeholder date

4588b27

STY: Update comments

14fd8ec

STY: computational_form -> to_xarray_dataset

ed79ad5

STY: Update docstring

323bb63

STY: Remove __future__ import

dd942d6

STY: Remove __future__ import

6fdc97c

STY: Remove capital X in docstrings

6fb92bb

STY: minor style improvements

4ca45a1

STY: Update changelog

138db72

STY: Removed new lines

ac64ee9

rstoneback added 2 commits July 19, 2022 14:43

STY: More calculate in docstrings

dc837cb

STY: Clarify docstring

a1873d3

jklenzing approved these changes Aug 2, 2022

View reviewed changes

rstoneback commented Aug 2, 2022

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

Apply suggestions from code review

bfaa1ed

Co-authored-by: Jeff Klenzing <19592220+jklenzing@users.noreply.github.com>

BUG: xarray version limit

20aefa5

rstoneback requested a review from aburrell August 4, 2022 20:02

jklenzing reviewed Aug 4, 2022

View reviewed changes

pysatSeasons/tests/test_core.py Outdated Show resolved Hide resolved

pysatSeasons/tests/test_core.py Outdated Show resolved Hide resolved

Apply suggestions from code review

76c3ee6

Co-authored-by: Jeff Klenzing <19592220+jklenzing@users.noreply.github.com>

aburrell reviewed Aug 5, 2022

View reviewed changes

rstoneback and others added 2 commits August 5, 2022 14:34

Apply suggestions from code review

dc3b18a

Co-authored-by: Angeline Burrell <aburrell@users.noreply.github.com>

BUG: pysat version requirement

93b2ac8

aburrell reviewed Aug 9, 2022

View reviewed changes

rstoneback added 3 commits August 9, 2022 10:57

STY: Reviewer feedback

d3bc16e

STY: Updated date

c751c87

BUG: Version limits in setup

1365170

aburrell reviewed Aug 11, 2022

View reviewed changes

rstoneback mentioned this pull request Aug 12, 2022

Xarray version limit, round 2 pysat/pysat#1034

Closed

10 tasks

STY: Remove setup.cfg version requirements

44c6140

rstoneback requested a review from aburrell August 31, 2022 19:52

aburrell approved these changes Aug 31, 2022

View reviewed changes

rstoneback added 2 commits August 31, 2022 14:54

BUG: Update

ec31f45

BUG: Tweak numpy verison?

73a49ed

rstoneback merged commit bde709b into main Sep 1, 2022

rstoneback deleted the rcv0-2-0 branch October 28, 2022 16:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rcv0 2 0 #30

Rcv0 2 0 #30

rstoneback commented Feb 18, 2022 •

edited

jklenzing left a comment

jklenzing Aug 2, 2022

rstoneback commented Aug 2, 2022

rstoneback commented Aug 4, 2022

jklenzing left a comment

aburrell Aug 5, 2022

rstoneback Aug 5, 2022

aburrell Aug 9, 2022

aburrell Aug 9, 2022

jklenzing Aug 9, 2022

aburrell Aug 9, 2022

aburrell Aug 11, 2022

rstoneback Aug 11, 2022

rstoneback Aug 11, 2022

aburrell Aug 15, 2022 •

edited

rstoneback Aug 15, 2022

rstoneback Aug 18, 2022

jklenzing Aug 18, 2022

rstoneback commented Aug 17, 2022

rstoneback commented Aug 31, 2022

rstoneback commented Sep 1, 2022

	\| matplotlib \| pysat >= 3.0.1 \|
	\| matplotlib \| pysat >= 3.0.2 \|

		pysat>=3.0.1
		xarray<2022.06.0

Rcv0 2 0 #30

Rcv0 2 0 #30

Conversation

rstoneback commented Feb 18, 2022 • edited

Description

Type of change

How Has This Been Tested?

Checklist:

jklenzing left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rstoneback commented Aug 2, 2022

rstoneback commented Aug 4, 2022

jklenzing left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aburrell Aug 15, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rstoneback commented Aug 17, 2022

rstoneback commented Aug 31, 2022

rstoneback commented Sep 1, 2022

rstoneback commented Feb 18, 2022 •

edited

aburrell Aug 15, 2022 •

edited