Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-16547: [Python] to_pandas fails with FixedOffset timezones when timestamp_as_object is used #14448

Merged
merged 11 commits into from Oct 27, 2022

Conversation

AlenkaF
Copy link
Member

@AlenkaF AlenkaF commented Oct 18, 2022

This PR tries to substitute fromutc method in ConvertTimezoneAware (arrow_to_pandas.cc) as it errors when tzinfo is an instance of pytz timezone. Basically does the change from:

tzinfo.fromutc(dt)

to

dt.replace(tzinfo=datetime.timezone.utc).astimezone(tzinfo)

but in C++.

Thank you @milesgranger for the help!

@github-actions
Copy link

@github-actions
Copy link

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@AlenkaF
Copy link
Member Author

AlenkaF commented Oct 19, 2022

Ha, I am getting the strange behaviour on CI now (not locally):
https://github.com/apache/arrow/actions/runs/3280427766/jobs/5401187533#step:6:5793

Edit: actually I think the expected df is wrong - need to look into that.

@AlenkaF AlenkaF marked this pull request as ready for review October 24, 2022 09:58
Copy link
Member

@jorisvandenbossche jorisvandenbossche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

python/pyarrow/tests/test_pandas.py Outdated Show resolved Hide resolved
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
@jorisvandenbossche
Copy link
Member

The remaining failures are ones are known ones, so merging!

@jorisvandenbossche jorisvandenbossche merged commit 31e8eeb into apache:master Oct 27, 2022
@AlenkaF AlenkaF deleted the ARROW-16547 branch October 27, 2022 08:02
@ursabot
Copy link

ursabot commented Oct 29, 2022

Benchmark runs are scheduled for baseline = c56934b and contender = 31e8eeb. 31e8eeb is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.0% ⬆️0.0%] test-mac-arm
[Finished ⬇️0.82% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.07% ⬆️0.04%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 31e8eebd ec2-t3-xlarge-us-east-2
[Failed] 31e8eebd test-mac-arm
[Finished] 31e8eebd ursa-i9-9960x
[Finished] 31e8eebd ursa-thinkcentre-m75q
[Finished] c56934b5 ec2-t3-xlarge-us-east-2
[Failed] c56934b5 test-mac-arm
[Finished] c56934b5 ursa-i9-9960x
[Finished] c56934b5 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@ursabot
Copy link

ursabot commented Oct 29, 2022

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants