Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cache uses filenames that might be too long on some systems #4172

Closed
hottwaj opened this issue Jan 26, 2024 · 3 comments · Fixed by #4176
Closed

cache uses filenames that might be too long on some systems #4172

hottwaj opened this issue Jan 26, 2024 · 3 comments · Fixed by #4176
Labels
T: bug Something isn't working

Comments

@hottwaj
Copy link

hottwaj commented Jan 26, 2024

Describe the bug

Running black 24.1.0 (without --check) can produce filenames in user's cache folder that are too long for some filesystems e.g. eCryptFS only allows filenames that are 143 chars long

The resulting error is:

Traceback (most recent call last):
  File "/home/hottwaj/.pyenv/versions/3.10.2/envs/test-venv/bin/black", line 8, in <module>
    sys.exit(patched_main())
  File "src/black/__init__.py", line 1593, in patched_main
  File "/home/hottwaj/.pyenv/versions/3.10.2/envs/test-venv/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/hottwaj/.pyenv/versions/3.10.2/envs/test-venv/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/hottwaj/.pyenv/versions/3.10.2/envs/test-venv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/hottwaj/.pyenv/versions/3.10.2/envs/test-venv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/hottwaj/.pyenv/versions/3.10.2/envs/test-venv/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "src/black/__init__.py", line 703, in main
  File "/home/hottwaj/.pyenv/versions/3.10.2/envs/test-venv/lib/python3.10/site-packages/black/concurrency.py", line 101, in reformat_many
    loop.run_until_complete(
  File "/home/hottwaj/.pyenv/versions/3.10.2/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
    return future.result()
  File "/home/hottwaj/.pyenv/versions/3.10.2/envs/test-venv/lib/python3.10/site-packages/black/concurrency.py", line 137, in schedule_formatting
    cache = Cache.read(mode)
  File "src/black/cache.py", line 67, in read
  File "/home/hottwaj/.pyenv/versions/3.10.2/lib/python3.10/pathlib.py", line 1288, in exists
    self.stat()
  File "/home/hottwaj/.pyenv/versions/3.10.2/lib/python3.10/pathlib.py", line 1095, in stat
    return self._accessor.stat(self, follow_symlinks=follow_symlinks)
OSError: [Errno 36] File name too long: '/home/hottwaj/.cache/black/24.1.0/cache.-.130.0.0.0.0.1.e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.0.e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.pickle'

Environment

  • Black's version: 24.1.0
  • OS and Python version: Linux, Python 3.10.2, home folder encrypted with eCryptFS
@hottwaj hottwaj added the T: bug Something isn't working label Jan 26, 2024
@JelleZijlstra
Copy link
Collaborator

Thanks for reporting. #4174 has a more detailed writeup.

I think we should do a few things:

  • Merge the two hashed pieces in the cache key in one, so the name gets shorter.
  • Add a test to ensure the cache filename isn't too long.
  • As eCryptfs incompatibility introduced in 24.1.0 #4174 suggests, make it so reformatting still succeeds (with a warning) if cache creation fails.

We can put out an extra release (24.1.1) once that's done.

JelleZijlstra added a commit to JelleZijlstra/black that referenced this issue Jan 26, 2024
- Ensure total file length stays under 96
- Hash the path only if it's too long
- Proceed normally (with a warning) if the cache can't be read

Fixes psf#4172
JelleZijlstra added a commit that referenced this issue Jan 26, 2024
- Ensure total file length stays under 96
- Hash the path only if it's too long
- Proceed normally (with a warning) if the cache can't be read

Fixes #4172
@JelleZijlstra
Copy link
Collaborator

Fix is merged. I think I'll release 24.1.1 either later today or tomorrow, depending on we get other bug reports that we'd want to address.

@khamaileon
Copy link

Fixed for me. thanks @JelleZijlstra and @hottwaj for reporting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T: bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants