Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pandas 2.0 update #3322

Merged
merged 10 commits into from
Apr 13, 2023
Merged

Pandas 2.0 update #3322

merged 10 commits into from
Apr 13, 2023

Conversation

jeffkinnison
Copy link
Contributor

@jeffkinnison jeffkinnison commented Apr 4, 2023

Updates to make Ludwig compatible with Pandas 2.0. The current changes include:

  • Remove experiment_impact_tracker as a dependency
  • Update LudwigProfiler to replicate experiment_impact_tracker CPU/GPU attribute collection
  • df.iteritems() -> df.items()
  • df.append(df2) -> df_engine.concat([df, df2])
  • Support for writing xls files was removed. This only seems to impact one integration test.

@github-actions
Copy link

github-actions bot commented Apr 5, 2023

Unit Test Results

  6 files  +  2    6 suites  +2   1h 1m 56s ⏱️ + 22m 14s
33 tests ±  0  30 ✔️ ±  0  3 💤 ±0  0 ±0 
99 runs  +33  90 ✔️ +30  9 💤 +3  0 ±0 

Results for commit b04c261. ± Comparison against base commit d68d96e.

♻️ This comment has been updated with latest results.

Copy link
Collaborator

@justinxzhao justinxzhao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet! Just one comment, otherwise LGTM.

def get_gpu_info():
"""Gathers general hardware information about an nvidia GPU.

:param args:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Remove empty docs.

If this is copied from the other repo, please reference it as a comment i.e.
Adapted from <link>, with additional adjustments for compatibility with Pandas 2.0.

@@ -20,19 +22,38 @@
from ludwig.globals import LUDWIG_VERSION
from ludwig.utils.data_utils import save_json

# disabling print because the following imports are verbose
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice cleanup.

@jeffkinnison
Copy link
Contributor Author

The changes listed above should cover the 2.0 update, and as far as I can tell we should still be able to support 1.x because df.items and pd.concat have been in the API since at least 1.0.

requirements.txt does allow for versions all the way back to 0.25, though, and concat doesn't exist that far back. Do we want to bump the minimum version to 1.0?

@justinxzhao

@jeffkinnison jeffkinnison changed the title [Draft] Pandas 2.0 update Pandas 2.0 update Apr 10, 2023
@jeffkinnison jeffkinnison marked this pull request as ready for review April 10, 2023 16:15
@justinxzhao
Copy link
Collaborator

The changes listed above should cover the 2.0 update, and as far as I can tell we should still be able to support 1.x because df.items and pd.concat have been in the API since at least 1.0.

requirements.txt does allow for versions all the way back to 0.25, though, and concat doesn't exist that far back. Do we want to bump the minimum version to 1.0?

@justinxzhao

Bumping the min version to 1.0 SGTM!

@jeffkinnison jeffkinnison merged commit b8f7f3a into master Apr 13, 2023
10 checks passed
@jeffkinnison jeffkinnison deleted the pandas-2.0-update branch April 13, 2023 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants