21.11b1 is still broken due to regex dependency

[pablogamboa]

Describe the bug

21.11b0 had to be re-released because the regex package didn't include Pattern in __all__ and this caused a typing annotation to fail with name error. 21.11b1 updated regex to 2021.4.4 to solve it. Unfortunately it didn't solve it. The required version should be 2021.11.9.

Initial PR

To Reproduce

Install black and run it:

Traceback (most recent call last):
  File "/Users/pablo/Library/Caches/pypoetry/virtualenvs/coin-tracker-server-Wu_bm3d4-py3.9/bin/black", line 5, in <module>
    from black import patched_main
  File "/Users/pablo/Library/Caches/pypoetry/virtualenvs/coin-tracker-server-Wu_bm3d4-py3.9/lib/python3.9/site-packages/black/__init__.py", line 39, in <module>
    from black.nodes import STARS, syms, is_simple_decorator_expression
  File "/Users/pablo/Library/Caches/pypoetry/virtualenvs/coin-tracker-server-Wu_bm3d4-py3.9/lib/python3.9/site-packages/black/nodes.py", line 31, in <module>
    from black.strings import has_triple_quotes
  File "/Users/pablo/Library/Caches/pypoetry/virtualenvs/coin-tracker-server-Wu_bm3d4-py3.9/lib/python3.9/site-packages/black/strings.py", line 159, in <module>
    def _cached_compile(pattern: str) -> re.Pattern:
AttributeError: module 'regex' has no attribute 'Pattern'

Expected behavior

Black running properly

Environment

• Black's version: 21.11b1
• OS and Python version: OSX aarch64, python 3.9.6

Additional context

[JelleZijlstra]

I can't reproduce this locally. Black works fine with regex==2021.4.4 for me. It's been there since October 2020 in the code:

22c5f461 regex_3/regex.py        (Matthew Barnett 2020-10-22 23:43:57 +0100 722) Pattern = type(_pat)

It doesn't matter whether it's in __all__ since we're not using import *.

[JelleZijlstra]

A few of us have tried to reproduce this on various OSes and Python versions but haven't been able to. Could anyone who has reported or reacted to this issue please provide more information to confirm that this issue exists?

[onerandomusername]

Tried recreating the issue, could not recreate on black 21.11b1
I did ensure that regex was at version 2021.4.4.

Perhaps the installed regex library is not up to date in your env?

[pablogamboa]

Here's a log reproducing the issue in a fresh environment using python 3.10:

https://gist.github.com/pablogamboa/6fae6e1cd099addc8c0806ee0a4f4f8a

If I install black, it works fine because just installs the latest regex library. But if I force regex to be 2021.4.4, then it won't work

[JelleZijlstra]

Thanks for the log! I still can't reproduce this, and I looked at the raw tar and whl files for regex 2021.4.4 and Pattern is definitely there.

[JelleZijlstra]

@ichard26 figured it out (thanks!). It turns out that the sdist for 2021.4.4 does not contain the line needed to expose regex.Pattern, but the wheels do (at least the wheel for manylinux/py3.8). In 2021.7.1, though, both the wheel and the sdist appear to have the right code. It's possible that some other 2021.4.4 wheels are also broken but I don't have time to check right now.

regex provides wheels for manylinux, Windows, and MacOS, so this issue likely doesn't affect a lot of users.

[ichard26]

So yeah, 2021.7.1 is the first release to be all clear and 2021.4.4 would be all good if its sdist wasn't different. Unfortunately regex 2021.4.4 doesn't have a wheel for Apple ARM or Linux musl so the sdist is more popular than ideal :/ Though it seems like the newer releases of regex do have wheels for at least Apple ARM (and anyway their sdist isn't broken so building from source should work fine). Could y'all use a newer version of regex? (you might have to upgrade pip so it recognizes the rather young Apple ARM wheel tag).

I'm open to do another hotfix to bump regex's minimum to 2021.7.1 but I'm afraid we would be eating too much of our churn budget.

[pablogamboa]

Thanks @ichard26 glad to hear it can be reproduced, I'm indeed on aarm64 (M1) and thus I was using sdist. This also explains why other team members weren't affected. I agree that with the popularity of aarm64 more folks might be affected by this.

[aldanor]

Confirming same error on osx-x64 (not M1), py38, 21.11b0, installed via conda-forge:

  File ".../lib/python3.8/site-packages/black/strings.py", line 159, in <module>
    def _cached_compile(pattern: str) -> re.Pattern:
AttributeError: module 'regex' has no attribute 'Pattern'

[ichard26]

Hmm, why are y'all picking up the sdist for regex? Is your pip too old or something or regex 2021.4.4 straight up doesn't provide a compatible wheel for your system?.

[pablogamboa]

@ichard26 In my case pip was a bit old, but not that old:

You are using pip version 21.1.3; however, version 21.3.1 is available.