Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-104104: Optimize pathlib.Path.glob() by avoiding repeated calls to os.path.normcase() #104105

Merged

Conversation

barneygale
Copy link
Contributor

@barneygale barneygale commented May 2, 2023

Use re.IGNORECASE to implement case-insensitive matching. This restores behaviour from before #31691.

$ ./python -m timeit -s 'from pathlib import Path; p = Path()' 'list(p.glob("Lib/*.py"))'
500 loops, best of 5: 403 usec per loop   # before
1000 loops, best of 5: 389 usec per loop  # after

…calls to `os.path.normcase()`

Use `re.IGNORECASE` to implement case-insensitive matching. This
restores behaviour from before python#31691.
Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great speedup! LGTM.

What was the motivation for moving to the normcase-based implementation in #31691? Did it just feel simpler at the time?

@AlexWaygood AlexWaygood added type-feature A feature request or enhancement 3.12 bugs and security fixes labels May 2, 2023
@barneygale
Copy link
Contributor Author

What was the motivation for moving to the normcase-based implementation in #31691? Did it just feel simpler at the time?

Mostly because os.path doesn't have an attribute indicating whether it's case sensitive or not, and it didn't occur to me at the time to use normcase('Aa') == 'Aa'.

@barneygale barneygale merged commit 47770a1 into python:main May 2, 2023
23 checks passed
@barneygale
Copy link
Contributor Author

Thanks for the review! :-)

@AlexWaygood
Copy link
Member

I think you should add an entry to "What's new in 3.12" about all the great optimisation work you've been doing on pathlib. It's a really good improvement that's worth celebrating :)

carljm added a commit to carljm/cpython that referenced this pull request May 2, 2023
* main: (760 commits)
  pythonGH-104102: Optimize `pathlib.Path.glob()` handling of `../` pattern segments (pythonGH-104103)
  pythonGH-104104: Optimize `pathlib.Path.glob()` by avoiding repeated calls to `os.path.normcase()` (pythonGH-104105)
  pythongh-103822: [Calendar] change return value to enum for day and month APIs (pythonGH-103827)
  pythongh-65022: Fix description of tuple return value in copyreg (python#103892)
  pythonGH-103525: Improve exception message from `pathlib.PurePath()` (pythonGH-103526)
  pythongh-84436: Add integration C API tests for immortal objects (pythongh-103962)
  pythongh-103743: Add PyUnstable_Object_GC_NewWithExtraData (pythonGH-103744)
  pythongh-102997: Update Windows installer to SQLite 3.41.2. (python#102999)
  pythonGH-103484: Fix redirected permanently URLs (python#104001)
  Improve assert_type phrasing (python#104081)
  pythongh-102997: Update macOS installer to SQLite 3.41.2. (pythonGH-102998)
  pythonGH-103472: close response in HTTPConnection._tunnel (python#103473)
  pythongh-88496: IDLE - fix another test on macOS (python#104075)
  pythongh-94673: Hide Objects in PyTypeObject Behind Accessors (pythongh-104074)
  pythongh-94673: Properly Initialize and Finalize Static Builtin Types for Each Interpreter (pythongh-104072)
  pythongh-104016: Skip test for deeply neste f-strings on wasi (python#104071)
  pythongh-104057: Fix direct invocation of test_super (python#104064)
  pythongh-87092: Expose assembler to unit tests (python#103988)
  pythongh-97696: asyncio eager tasks factory (python#102853)
  pythongh-84436: Immortalize in _PyStructSequence_InitBuiltinWithFlags() (pythongh-104054)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.12 bugs and security fixes performance Performance or resource usage topic-pathlib type-feature A feature request or enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants