Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

change pandas engine to be numpy>2 compat #1690

Merged
merged 27 commits into from
Jun 20, 2024
Merged

change pandas engine to be numpy>2 compat #1690

merged 27 commits into from
Jun 20, 2024

Conversation

cosmicBboy
Copy link
Collaborator

@cosmicBboy cosmicBboy commented Jun 17, 2024 •
edited

fixes #1656

This PR:

  • Updates the pandas/numpy type engine to stop using deprecated numpy types
  • Updates CI to run the extras packages in a separate job
  • Moves extra package (pyspark, dask, modin, etc) out of the top-level module-loading execution path of pandera
  • Pins numpy<2 in extra package CI
  • Adds a temporary numpy2 patch until dependent packages (pyspark, modin, etc) support numpy2
  • Disables decimal type test for modin
  • Pins numpy<2 in readthedocs config yaml

@cosmicBboy
change pandas engine to be numpy>2 compat
8835bc5 
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
@cosmicBboy
remove numpy from mypy pre-commit type stubs
96f1a66 
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
@cosmicBboy
install numpy<2.0.0 in readthedocs build
514047a 
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
@cosmicBboy
remove check docs ci test
47e1985 
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
@cosmicBboy cosmicBboy mentioned this pull request Jun 17, 2024
Closed
3 tasks
@cosmicBboy
update
9105027 
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
@cosmicBboy
debug
0637cf5 
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
@cosmicBboy
update requirements
bdbe2e0 
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
@cosmicBboy
update requirements, only use numpy2 for latest pandas
bcd9198 
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
@cosmicBboy
move pyspark config
63838ad 
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
@cosmicBboy
remove typing from root __init__
03a8b25 
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
@cmarshak
Copy link

Thank you for fixing this!

@cosmicBboy
[wip] try patching numpy2 with NaN
c4e06ee 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy
[wip] remove pyspark import from typing.pandas
c809bed 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy
isolate dask, modin, pyspark, etc imports, update ci
2aca4be 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy
update numpy patch
515d65f 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy
fix ci numpy pin
e6cd14a 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy
update ci
11f2aee 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy
update
63d6203 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy
update
8230cc3 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy
debug
00f568e 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy
debug
d51a372 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy
debug
f72c0a3 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy
update ci test for mypy
877059a 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy
add numpy patch to root pandera
e93842b 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy
debug
cdf1a14 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy
debug
527f79a 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy
debug
5347a5a 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy
skip modin test for decimals, remove pandas2.0.3
c7a9b29 
Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>
@cosmicBboy cosmicBboy merged commit 47c236e into main Jun 20, 2024
154 checks passed
@cosmicBboy cosmicBboy deleted the bugfix/1656 branch June 20, 2024 14:13
@cmarshak cmarshak mentioned this pull request Jun 20, 2024
Merged
@kr-hansen kr-hansen mentioned this pull request Jun 25, 2024
Closed
@cosmicBboy cosmicBboy mentioned this pull request Jul 17, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

pandera not compatible with numpy 2.0
2 participants
@cosmicBboy @cmarshak