re._compile should check if the argument is a compiled pattern before checking cache and flags

[Recursing]

BPO	39195
Nosy	@serhiy-storchaka, @Recursing
PRs	bpo-39195: Check earlier if pattern is compiled in _compile #17799

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-01-02.20:10:00.588>
created_at = <Date 2020-01-02.18:59:30.340>
labels = ['3.8', 'library', '3.9', 'performance']
title = 're._compile should check if the argument is a compiled pattern before checking cache and flags'
updated_at = <Date 2020-01-02.20:10:00.587>
user = 'https://github.com/Recursing'

bugs.python.org fields:

activity = <Date 2020-01-02.20:10:00.587>
actor = 'Recursing'
assignee = 'none'
closed = True
closed_date = <Date 2020-01-02.20:10:00.588>
closer = 'Recursing'
components = ['Library (Lib)']
creation = <Date 2020-01-02.18:59:30.340>
creator = 'Recursing'
dependencies = []
files = []
hgrepos = []
issue_num = 39195
keywords = ['patch']
message_count = 4.0
messages = ['359210', '359213', '359215', '359218']
nosy_count = 2.0
nosy_names = ['serhiy.storchaka', 'Recursing']
pr_nums = ['17799']
priority = 'normal'
resolution = None
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue39195'
versions = ['Python 3.8', 'Python 3.9']

[Recursing]

In the re module, re._compile gets called when using most re methods.
In my use case (which I think is not rare) I have a small number of compiled patterns that I have to match against a large number of short strings

profiling showed that half of the total runtime was still spent in re._compile, checking for the type of the flags and trying to get the pattern in a cache

Example code that exhibits this behavior:

import re

pattern = re.compile("spam")
string = "Monty pythons"
for _ in range(1000000):
    re.search(pattern, string)

[Recursing]

I now know that the correct and fastest way to do this is to use pattern.search, but I still think this change would be an improvement

[serhiy-storchaka]

PR 17799 improves the performance of an uncommon case at the cost of reducing the performance of a common case. I doubt this is a good change.

If you have a compiled pattern, it is better to use its methods.

[Recursing]

Rereading the documentation it's actually well documented, and I fully agree.

Thanks for your time