Segfault in comparison between datetime.timezone.utc and pytz.utc

[TomAugspurger]

BPO	37915
Nosy	@abalkin, @ned-deily, @ambv, @serhiy-storchaka, @pganssle, @pablogsal, @miss-islington, @tirkarthi, @TomAugspurger
PRs	bpo-37915: Fix comparison between tzinfo objects and timezone objects #15390 [3.8] bpo-37915: Fix comparison between tzinfo objects and timezone objects (GH-15390) #15399 [3.7] bpo-37915: Fix comparison between tzinfo objects and timezone objects (GH-15390) #15417

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-08-23.10:12:53.927>
created_at = <Date 2019-08-22.11:18:39.483>
labels = ['3.7', '3.8', '3.9', 'type-crash', 'release-blocker']
title = 'Segfault in comparison between datetime.timezone.utc and pytz.utc'
updated_at = <Date 2019-08-23.11:53:21.433>
user = 'https://github.com/TomAugspurger'

bugs.python.org fields:

activity = <Date 2019-08-23.11:53:21.433>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2019-08-23.10:12:53.927>
closer = 'pablogsal'
components = []
creation = <Date 2019-08-22.11:18:39.483>
creator = 'tomaugspurger'
dependencies = []
files = []
hgrepos = []
issue_num = 37915
keywords = ['3.8regression']
message_count = 10.0
messages = ['350184', '350186', '350189', '350194', '350195', '350196', '350227', '350267', '350270', '350277']
nosy_count = 9.0
nosy_names = ['belopolsky', 'ned.deily', 'lukasz.langa', 'serhiy.storchaka', 'p-ganssle', 'pablogsal', 'miss-islington', 'xtreak', 'tomaugspurger']
pr_nums = ['15390', '15399', '15417']
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue37915'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

[TomAugspurger]

The following crashes with Python 3.8b3

import sys
import pytz
import datetime

print(sys.version_info)
print(pytz.__version__)
print(datetime.timezone.utc == pytz.utc)

When run with -X faulthandler, I see

sys.version_info(major=3, minor=8, micro=0, releaselevel='beta', serial=3)
2019.2
Fatal Python error: Segmentation fault

Current thread 0x00000001138dc5c0 (most recent call first):
  File "foo.py", line 8 in <module>
Segmentation fault: 11

[tirkarthi]

I am adding 3.8 regression since the code works with 3.7 though it involves pytz.

[tirkarthi]

Seems this is due to bpo-37685 (dde944f) on bisecting datetime related changes. Adding Serhiy. I guess this could be marked as release blocker.

Here is a simplified reproducer on extracting pytz.utc source [0] which is an object of simple subclass of datetime.tzinfo :

import sys
import datetime

print(sys.version)
class UTC(datetime.tzinfo):
    pass

print(datetime.timezone.utc == UTC())
datetime.timezone.utc == datetime.tzinfo() # This also segfaults without a subclass and should be False

# Segfaults

➜ cpython git:(dde944f) git checkout dde944f && make -s -j4 > /dev/null
HEAD is now at dde944f bpo-37685: Fixed comparisons of datetime.timedelta and datetime.timezone. (GH-14996)
➜ cpython git:(dde944f) ./python.exe ../backups/bpo37915.py
3.8.0b3+ (tags/v3.8.0b3-30-gdde944f9df:dde944f9df, Aug 22 2019, 17:55:14)
[Clang 7.0.2 (clang-700.1.81)]
[1] 33988 segmentation fault ./python.exe ../backups/bpo37915.py

# Commit before works fine

➜ cpython git:(dde944f) git checkout dde944f~1 && make -s -j4 > /dev/null
Previous HEAD position was dde944f bpo-37685: Fixed comparisons of datetime.timedelta and datetime.timezone. (GH-14996)
HEAD is now at 4e402d3 Correct description of HTTP status code 308. (GH-15098)
➜ cpython git:(4e402d3) ./python.exe ../backups/bpo37915.py
3.8.0b3+ (tags/v3.8.0b3-30-gdde944f9df:dde944f9df, Aug 22 2019, 17:55:14)
[Clang 7.0.2 (clang-700.1.81)]
False

[0] https://github.com/stub42/pytz/blob/62f872054dde69e5c510094093cd6e221d96d5db/src/pytz/__init__.py#L256

[TomAugspurger]

Thanks for debugging this Karthikeyan and for the quick fix Pablo!

[pablogsal]

This is due to an incorrect type check in the timezone_richcompare, it should be a comparison against PyDateTime_TimeZoneType not against PyDateTime_TZInfoType. The segfault is due to an invalid casting to PyDateTime_TimeZoneType (the child) from the parent (PyDateTime_TZInfoType).

[pablogsal]

Thanks, Tom and Karthikeyan for the finding and the debugging :)

[miss-islington]

New changeset 4be11c0 by Miss Islington (bot) (Pablo Galindo) in branch 'master':
bpo-37915: Fix comparison between tzinfo objects and timezone objects (GH-15390)
4be11c0

[miss-islington]

New changeset 5c77730 by Miss Islington (bot) in branch '3.8':
bpo-37915: Fix comparison between tzinfo objects and timezone objects (GH-15390)
5c77730

[pablogsal]

New changeset 1b1796d by Pablo Galindo in branch '3.7':
[3.7] bpo-37915: Fix comparison between tzinfo objects and timezone objects (GH-15390) (GH-15417)
1b1796d

[ned-deily]

(Removing “3.7regression” as the problematic change for 3.7 had not yet been released.)