Invalid Cross-device Link

[Kurt-von-Laven]

The pre-commit hook sometimes crashes when run from the Windows file system within Windows Subsystem for Linux (WSL):

Update markdown table-of-contents.............................................Failed
 - hook id: md-toc
 - exit code: 1
Traceback (most recent call last):
File "~/.cache/pre-commit/repogo0t_l7x/py_env-python3.10.4/lib/python3.10/site-packages/md_toc/__main__.py", line 35, in main
                                                                                                                          result = args.func(args)
File "~/.cache/pre-commit/repogo0t_l7x/py_env-python3.10.4/lib/python3.10/site-packages/md_toc/cli.py", line 77, in write_toc
                                                                                                                          write_strings_on_files_between_markers(
File "~/.cache/pre-commit/repogo0t_l7x/py_env-python3.10.4/lib/python3.10/site-packages/md_toc/api.py", line 141, in write_strings_on_files_between_markers
                                                                                                                          write_string_on_file_between_markers(f, strings[file_id], marker, newline_string)
File "~/.cache/pre-commit/repogo0t_l7x/py_env-python3.10.4/lib/python3.10/site-packages/md_toc/api.py", line 83, in write_string_on_file_between_markers
                                                                                                                          fpyutils.filelines.remove_line_interval(
File "~/.cache/pre-commit/repogo0t_l7x/py_env-python3.10.4/lib/python3.10/site-packages/fpyutils/filelines.py", line 242, in remove_line_interval
                                                                                                                          with atomic_write(output_file, overwrite=True) as f:
File "~/.asdf/installs/python/3.10.4/lib/python3.10/contextlib.py", line 142, in __exit__
                                                                                                                          next(self.gen)
File "~/.cache/pre-commit/repogo0t_l7x/py_env-python3.10.4/lib/python3.10/site-packages/atomicwrites/__init__.py", line 169, in _open
                                                                                                                          self.commit(f)
File "~/.cache/pre-commit/repogo0t_l7x/py_env-python3.10.4/lib/python3.10/site-packages/atomicwrites/__init__.py", line 202, in commit
                                                                                                                          replace_atomic(f.name, self._path)
File "~/.cache/pre-commit/repogo0t_l7x/py_env-python3.10.4/lib/python3.10/site-packages/atomicwrites/__init__.py", line 99, in replace_atomic
                                                                                                                          return _replace_atomic(src, dst)
File "~/.cache/pre-commit/repogo0t_l7x/py_env-python3.10.4/lib/python3.10/site-packages/atomicwrites/__init__.py", line 55, in _replace_atomic
                                                                                                                          os.rename(src, dst)
OSError: [Errno 18] Invalid cross-device link: '/mnt/c/Users/<username>/source/repos/scribemd-uwp/tmpkpv7awhg' -> 'README.md'

I am not yet certain under what circumstances this issue occurs; if it's happening to others I suggest starting by re-running the hook.

[frnmst]

This has something to do with atomicwites which is used by fpyutils in the write operation. See this comment in atomicwites documentation:

Windows support, although not well-tested.

Does this happen occasionally, for example when the readme file changed or is it totally random?

[Kurt-von-Laven]

That sounds like the right idea, yeah. Indeed, it most recently happened when I modified the README within WSL on the Windows file system. I have seen it before, most likely in the same scenario, but it’s been too long for me to be sure.

[frnmst]

This issue should be fixed with version 8.1.7 since atomicwrites is no longer a dependency of fpyutils. I don't have Windows so I can't test this

[dichn]

Hello @frnmst ,
It happened to me as well, It failed when calling from https://pre-commit.ci/

Update markdown table-of-contents........................................Failed
- hook id: md-toc
- exit code: 1

Traceback (most recent call last):
  File "/pc/clone/VOaz3usUQ4aoEcz6Rnavwg/py_env-python3/lib/python3.11/site-packages/md_toc/__main__.py", line 36, in main
    result = args.func(args)
             ^^^^^^^^^^^^^^^
  File "/pc/clone/VOaz3usUQ4aoEcz6Rnavwg/py_env-python3/lib/python3.11/site-packages/md_toc/cli.py", line 79, in write_toc
    write_strings_on_files_between_markers(
  File "/pc/clone/VOaz3usUQ4aoEcz6Rnavwg/py_env-python3/lib/python3.11/site-packages/md_toc/api.py", line 170, in write_strings_on_files_between_markers
    write_string_on_file_between_markers(f, strings[file_id], marker,
  File "/pc/clone/VOaz3usUQ4aoEcz6Rnavwg/py_env-python3/lib/python3.11/site-packages/md_toc/api.py", line 101, in write_string_on_file_between_markers
    fpyutils.filelines.remove_line_interval(
  File "/pc/clone/VOaz3usUQ4aoEcz6Rnavwg/py_env-python3/lib/python3.11/site-packages/fpyutils/filelines.py", line 275, in remove_line_interval
    os.replace(f.name, output_file)
OSError: [Errno 18] Invalid cross-device link: '/tmp/tmpcht2mdqp' -> 'README.md'

And it also failed locally on my fedora 36

$ cat .pre-commit-config.yaml | grep md-toc -A 7
-   repo: https://github.com/frnmst/md-toc
    rev: 8.1.7
    hooks:
    - id: md-toc

-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.4.0
    hooks:
    - id: end-of-file-fixer
    - id: trailing-whitespace

$ pre-commit run --all-files
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
black....................................................................Passed
isort....................................................................Passed
Update markdown table-of-contents........................................Failed
- hook id: md-toc
- exit code: 1

Traceback (most recent call last):
  File "/home/dichen/.cache/pre-commit/repoo10urf49/py_env-python3.10/lib/python3.10/site-packages/md_toc/__main__.py", line 36, in main
    result = args.func(args)
  File "/home/dichen/.cache/pre-commit/repoo10urf49/py_env-python3.10/lib/python3.10/site-packages/md_toc/cli.py", line 79, in write_toc
    write_strings_on_files_between_markers(
  File "/home/dichen/.cache/pre-commit/repoo10urf49/py_env-python3.10/lib/python3.10/site-packages/md_toc/api.py", line 170, in write_strings_on_files_between_markers
    write_string_on_file_between_markers(f, strings[file_id], marker,
  File "/home/dichen/.cache/pre-commit/repoo10urf49/py_env-python3.10/lib/python3.10/site-packages/md_toc/api.py", line 101, in write_string_on_file_between_markers
    fpyutils.filelines.remove_line_interval(
  File "/home/dichen/.cache/pre-commit/repoo10urf49/py_env-python3.10/lib/python3.10/site-packages/fpyutils/filelines.py", line 275, in remove_line_interval
    os.replace(f.name, output_file)
OSError: [Errno 18] Invalid cross-device link: '/tmp/tmpk_ukykyv' -> 'README.md'

fix end of files.........................................................Passed
trim trailing whitespace.................................................Passed

[frnmst]

Interesting. I'll try with Fedora to see if I can reproduce the bug and maybe change os.replace with shutil.move or something...

[frnmst]

OK, I did a quick test: I installed Fedora Workstation 36 and indeed the problem shows up immediately, even if I install md-toc from source. I can see that a real difference is the Python version (3.11 vs 3.9 on Debian). Changing os.replace with shutil.move in fpyutils solves the problem. New releases will be out soon...

[frnmst]

Tested the latest md-toc (8.1.8) on Fedora and Arch Linux. The exception did not appear anymore during testing.

Please update to the latest versions of md-toc and fpyutils.

[frnmst]

I hope this issue is finally solved so I'm going to close it