Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix CI failures #145

Merged
merged 7 commits into from
Apr 17, 2024
Merged

Conversation

IzerOnadimQC
Copy link
Collaborator

@IzerOnadimQC IzerOnadimQC commented Apr 14, 2024

This PR should fix the current CI failures. There were several issues, listed below from most trivial to least, all but the first were related to tests involving nightly builds of pandas (caused by changes that will be introduced in pandas 3).

  1. reference-data was needed for pyarrow 15.0.2.
  2. Pandas 3 will remove the infer_datetime_format argument from to_datetime. According to the docs this has been deprecated since version 2 and passing it has no effect.
  3. The _is_view property will be removed from the DataFrame class. Obviously using a private property is always a bit dubious, but it was only used for testing. My no-less-dubious solution was to pull the same information out of another private property :)
  4. In two places in the tests, numpy.array_split is used to split a DataFrame. As far as I can tell this only works because array_split calls numpy.swapaxes here, which in turn uses calls swapaxes on the underlying object through a getattr call. This will no longer work once the (currently deprecated) swapaxes method is removed in Pandas 3. This is known to numpy maintainers (see this issue), and verdict appears to be that it was never intended usage and will not be supported going forward. Therefore, I added a workaround.
  • Changelog entry

@IzerOnadimQC IzerOnadimQC changed the title [WIP] Investigate and fix CI failures Fix CI failures Apr 16, 2024
@IzerOnadimQC IzerOnadimQC marked this pull request as ready for review April 16, 2024 21:01
@xhochy xhochy merged commit 69167c1 into data-engineering-collective:main Apr 17, 2024
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants