Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

link_or_copy_directory() error - Invalid cross-device link #744

Closed
aborruso opened this issue Apr 28, 2020 · 28 comments · Fixed by #769
Closed

link_or_copy_directory() error - Invalid cross-device link #744

aborruso opened this issue Apr 28, 2020 · 28 comments · Fixed by #769

Comments

@aborruso
Copy link

aborruso commented Apr 28, 2020

Hi,
when I run

datasette  publish heroku -n myapp --template-dir ./template mydb.db

I have this error

Traceback (most recent call last):
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/utils/__init__.py", line 607, in link_or_copy_directory
    shutil.copytree(src, dst, copy_function=os.link)
  File "/usr/lib/python3.7/shutil.py", line 365, in copytree
    raise Error(errors)
shutil.Error: [('/myfolder/youtubeComunePalermo/processing/./template/base.html', '/tmp/tmps9_4mzc4/templates/base.html', "[Errno 18] Invalid cross-device link: '/myfolder/youtubeComunePalermo/processing/./template/base.html' -> '/tmp/tmps9_4mzc4/templates/base.html'"), ('/myfolder/youtubeComunePalermo/processing/./template/index.html', '/tmp/tmps9_4mzc4/templates/index.html', "[Errno 18] Invalid cross-device link: '/myfolder/youtubeComunePalermo/processing/./template/index.html' -> '/tmp/tmps9_4mzc4/templates/index.html'")]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aborruso/.local/bin/datasette", line 8, in <module>
    sys.exit(cli())
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/publish/heroku.py", line 103, in heroku
    extra_metadata,
  File "/usr/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/publish/heroku.py", line 191, in temporary_heroku_directory
    os.path.join(tmp.name, "templates"),
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/utils/__init__.py", line 609, in link_or_copy_directory
    shutil.copytree(src, dst)
  File "/usr/lib/python3.7/shutil.py", line 321, in copytree
    os.makedirs(dst)
  File "/usr/lib/python3.7/os.py", line 221, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/tmp/tmps9_4mzc4/templates'

I'm attaching my very basic template folder.

Thank you

template.zip

@simonw
Copy link
Owner

simonw commented Apr 29, 2020

Is it possible that your /tmp directory is on a different volume from the template folder? That could cause a problem with the symlinks.

Here's the code in question:

def link_or_copy(src, dst):
# Intended for use in populating a temp directory. We link if possible,
# but fall back to copying if the temp directory is on a different device
# https://github.com/simonw/datasette/issues/141
try:
os.link(src, dst)
except OSError:
shutil.copyfile(src, dst)
def link_or_copy_directory(src, dst):
try:
shutil.copytree(src, dst, copy_function=os.link)
except OSError:
shutil.copytree(src, dst)

It looks to me like we were expecting an OSError but in your environment you got a shutil.Error instead. So a fix could be to capture that error as well.

@simonw
Copy link
Owner

simonw commented Apr 29, 2020

https://docs.python.org/3/library/shutil.html#shutil.copytree says that shutil.Error should be raised here, so I think this is a bug in the link_or_copy_directory function.

I don't have an environment that can replicate this bug. @aborruso I'm going to ship a fix in a branch that you can test against.

@simonw simonw changed the title datasette heroku publish: errore using template-dir option link_or_copy_directory() error - catches the wrong exception Apr 29, 2020
@simonw
Copy link
Owner

simonw commented Apr 29, 2020

This exception handling was introduced in #141

@simonw
Copy link
Owner

simonw commented Apr 29, 2020

@aborruso I think I have a branch with a fix - could you try it out?

Install the new branch like this:

pip install https://github.com/simonw/datasette/archive/issue-744.zip

Then try running datasette publish heroku again.

If this fixes your bug I'll merge that pull request!

@aborruso
Copy link
Author

aborruso commented Apr 29, 2020

Hi @simonw , I have installed it and I have the below errors.

Is it possible that your /tmp directory is on a different volume from the template folder? That could cause a problem with the symlinks.

No, /tmp folder is in the same volume.

Thank you

Traceback (most recent call last):
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/utils/__init__.py", line 607, in link_or_copy_directory
    shutil.copytree(src, dst, copy_function=os.link)
  File "/usr/lib/python3.7/shutil.py", line 365, in copytree
    raise Error(errors)
shutil.Error: [('/var/youtubeComunePalermo/processing/./template/base.html', '/tmp/tmpcqv_1i5d/templates/base.html', "[Errno 18] Invalid cross-device link: '/var/youtubeComunePalermo/processing/./template/base.html' -> '/tmp/tmpcqv_1i5d/templates/base.html'"), ('/var/youtubeComunePalermo/processing/./template/index.html', '/tmp/tmpcqv_1i5d/templates/index.html', "[Errno 18] Invalid cross-device link: '/var/youtubeComunePalermo/processing/./template/index.html' -> '/tmp/tmpcqv_1i5d/templates/index.html'")]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aborruso/.local/bin/datasette", line 8, in <module>
    sys.exit(cli())
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 610, in invoke    return callback(*args, **kwargs)
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/publish/heroku.py", line 103, in heroku
    extra_metadata,
  File "/usr/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/publish/heroku.py", line 191, in temporary_heroku_directory
    os.path.join(tmp.name, "templates"),
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/utils/__init__.py", line 609, in link_or_copy_directory
    shutil.copytree(src, dst)
  File "/usr/lib/python3.7/shutil.py", line 321, in copytree
    os.makedirs(dst)
  File "/usr/lib/python3.7/os.py", line 221, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/tmp/tmpcqv_1i5d/templates'

@aborruso
Copy link
Author

aborruso commented Apr 29, 2020

A stupid note: I have no tmpcqv_1i5d folder in in /tmp.

It seems to me that it does not create any /tmp/tmpcqv_1i5d/templates folder (or other name folder, inside /tmp)

@simonw
Copy link
Owner

simonw commented Apr 29, 2020

What operating system are you using? I need to figure out a way to replicate this bug.

@aborruso
Copy link
Author

aborruso commented Apr 29, 2020

Hi @simonw it's debian as Windows Subsystem for Linux

PRETTY_NAME="Pengwin"
NAME="Pengwin"
VERSION_ID="10"
VERSION="10 (buster)"
ID=debian
ID_LIKE=debian
HOME_URL="https://github.com/whitewaterfoundry/Pengwin"
SUPPORT_URL="https://github.com/whitewaterfoundry/Pengwin"
BUG_REPORT_URL="https://github.com/whitewaterfoundry/Pengwin"
VERSION_CODENAME=buster

@aborruso
Copy link
Author

aborruso commented May 7, 2020

Hi @simonw probably I could try to do it in Python for windows. I do not like to do these things in win enviroment.

Because probably WSL Linux env (in which I do a lot of great things) is not an environment that will be tested for datasette.

In win I shouldn't have any problems. Am I right?

@simonw
Copy link
Owner

simonw commented May 7, 2020

Have a try and let me know what happens! I'd like this stuff to just work on Windows but I need to figure out how to get an environment working.

@aborruso
Copy link
Author

aborruso commented May 7, 2020

@simonw another error starting from Windows.

I run

datasette  publish heroku -n comunepa --template-dir template commissioniComunePalermo.db

And I have

Traceback (most recent call last):
  File "c:\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\Scripts\datasette.exe\__main__.py", line 9, in <module>
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\datasette\publish\heroku.py", line 53, in heroku
    line.split()[0] for line in check_output(["heroku", "plugins"]).splitlines()
  File "c:\python37\lib\subprocess.py", line 395, in check_output
    **kwargs).stdout
  File "c:\python37\lib\subprocess.py", line 472, in run
    with Popen(*popenargs, **kwargs) as process:
  File "c:\python37\lib\subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "c:\python37\lib\subprocess.py", line 1178, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] The specified file could not be found

files.zip

@simonw
Copy link
Owner

simonw commented May 7, 2020

I have a Windows 10 gaming PC in the house: what steps can I take to get an environment in there that matches yours? I've not installed Python on Windows before.

@aborruso
Copy link
Author

aborruso commented May 7, 2020

In Windows I'm not very strong. I use debian (inside WSL).

However these are the possible steps:

  • I have installed Python 3 for win (I have 3.7.3);
  • I have installed heroku cli for win64 and logged in;
  • I have installed datasette running python -m pip install --upgrade --user datasette.

It's a very basic Python env that I do not use. This time only to reach my goal: try to publish using custom template

@aborruso
Copy link
Author

aborruso commented May 7, 2020

I have installed heroku plugins:install heroku-builds, but I have the same error.

Then I have removed from datasette\publish\heroku.py

        # Check for heroku-builds plugin
        plugins = [
            line.split()[0] for line in check_output(["heroku", "plugins"]).splitlines()
        ]
        if b"heroku-builds" not in plugins:
            click.echo(
                "Publishing to Heroku requires the heroku-builds plugin to be installed."
            )
            click.confirm(
                "Install it? (this will run `heroku plugins:install heroku-builds`)",
                abort=True,
            )
            call(["heroku", "plugins:install", "heroku-builds"])

And now I have

Traceback (most recent call last):
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\datasette\publish\heroku.py", line 210, in temporary_heroku_directory
    yield
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\datasette\publish\heroku.py", line 96, in heroku
    list_output = check_output(["heroku", "apps:list", "--json"]).decode(
  File "c:\python37\lib\subprocess.py", line 395, in check_output
    **kwargs).stdout
  File "c:\python37\lib\subprocess.py", line 472, in run
    with Popen(*popenargs, **kwargs) as process:
  File "c:\python37\lib\subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "c:\python37\lib\subprocess.py", line 1178, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] The specified file could not be found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\Scripts\datasette.exe\__main__.py", line 9, in <module>
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\datasette\publish\heroku.py", line 120, in heroku
    call(["heroku", "builds:create", "-a", app_name, "--include-vcs-ignore"])
  File "c:\python37\lib\contextlib.py", line 130, in __exit__
    self.gen.throw(type, value, traceback)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\datasette\publish\heroku.py", line 213, in temporary_heroku_directory
    tmp.cleanup()
  File "c:\python37\lib\tempfile.py", line 809, in cleanup
    _shutil.rmtree(self.name)
  File "c:\python37\lib\shutil.py", line 513, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\python37\lib\shutil.py", line 401, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "c:\python37\lib\shutil.py", line 399, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] Unable to access file. The file is being used by another process: 'C:\\Users\\aborr\\AppData\\Local\\Temp\\tmpkcxy8i_q'

@simonw
Copy link
Owner

simonw commented May 21, 2020

I just hit this bug myself, or at least a variant of it!

https://github.com/simonw/museums/runs/697063068?check_suite_focus=true

2020-05-21T17:14:02.2596471Z Traceback (most recent call last):
2020-05-21T17:14:02.2599146Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/datasette/utils/__init__.py", line 605, in link_or_copy_directory
2020-05-21T17:14:02.2599861Z     shutil.copytree(src, dst, copy_function=os.link)
2020-05-21T17:14:02.2600377Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/shutil.py", line 554, in copytree
2020-05-21T17:14:02.2600786Z     return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
2020-05-21T17:14:02.2601196Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/shutil.py", line 510, in _copytree
2020-05-21T17:14:02.2601580Z     raise Error(errors)
2020-05-21T17:14:02.2604565Z shutil.Error: [('/home/runner/work/museums/museums/templates/row-browse-museums.html', '/tmp/tmpq47xi96y/datasette/templates/row-browse-museums.html',
"[Errno 18] Invalid cross-device link: '/home/runner/work/museums/museums/templates/row-browse-museums.html' -> '/tmp/tmpq47xi96y/datasette/templates/row-browse-museums.html'"), ('/home/runner/work/museums/museums/templates/index.html', '/tmp/tmpq47xi96y/datasette/templates/index.html',
"[Errno 18] Invalid cross-device link: '/home/runner/work/museums/museums/templates/index.html' -> '/tmp/tmpq47xi96y/datasette/templates/index.html'"), ('/home/runner/work/museums/museums/templates/query-browse-search.html', '/tmp/tmpq47xi96y/datasette/templates/query-browse-search.html',
"[Errno 18] Invalid cross-device link: '/home/runner/work/museums/museums/templates/query-browse-search.html' -> '/tmp/tmpq47xi96y/datasette/templates/query-browse-search.html'"), ('/home/runner/work/museums/museums/templates/_analytics.html', '/tmp/tmpq47xi96y/datasette/templates/_analytics.html', "[Errno 18] Invalid cross-device link: '/home/runner/work/museums/museums/templates/_analytics.html' -> '/tmp/tmpq47xi96y/datasette/templates/_analytics.html'"), ('/home/runner/work/museums/museums/templates/_museum_card.html', '/tmp/tmpq47xi96y/datasette/templates/_museum_card.html', "[Errno 18] Invalid cross-device link: '/home/runner/work/museums/museums/templates/_museum_card.html' -> '/tmp/tmpq47xi96y/datasette/templates/_museum_card.html'"), ('/home/runner/work/museums/museums/templates/pages/about.html', '/tmp/tmpq47xi96y/datasette/templates/pages/about.html', "[Errno 18] Invalid cross-device link: '/home/runner/work/museums/museums/templates/pages/about.html' -> '/tmp/tmpq47xi96y/datasette/templates/pages/about.html'"), ('/home/runner/work/museums/museums/templates/pages/map.html', '/tmp/tmpq47xi96y/datasette/templates/pages/map.html', "[Errno 18] Invalid cross-device link: '/home/runner/work/museums/museums/templates/pages/map.html' -> '/tmp/tmpq47xi96y/datasette/templates/pages/map.html'")]
2020-05-21T17:14:02.2605437Z 
2020-05-21T17:14:02.2605797Z During handling of the above exception, another exception occurred:
2020-05-21T17:14:02.2606102Z 
2020-05-21T17:14:02.2606423Z Traceback (most recent call last):
2020-05-21T17:14:02.2606817Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/bin/datasette", line 8, in <module>
2020-05-21T17:14:02.2607189Z     sys.exit(cli())
2020-05-21T17:14:02.2607907Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 829, in __call__
2020-05-21T17:14:02.2608347Z     return self.main(*args, **kwargs)
2020-05-21T17:14:02.2609024Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 782, in main
2020-05-21T17:14:02.2609451Z     rv = self.invoke(ctx)
2020-05-21T17:14:02.2610116Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
2020-05-21T17:14:02.2610550Z     return _process_result(sub_ctx.command.invoke(sub_ctx))
2020-05-21T17:14:02.2611451Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
2020-05-21T17:14:02.2611989Z     return _process_result(sub_ctx.command.invoke(sub_ctx))
2020-05-21T17:14:02.2612682Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
2020-05-21T17:14:02.2613117Z     return ctx.invoke(self.callback, **ctx.params)
2020-05-21T17:14:02.2613801Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 610, in invoke
2020-05-21T17:14:02.2614235Z     return callback(*args, **kwargs)
2020-05-21T17:14:02.2614946Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/datasette/publish/cloudrun.py", line 111, in cloudrun
2020-05-21T17:14:02.2615371Z     with temporary_docker_directory(
2020-05-21T17:14:02.2615769Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/contextlib.py", line 113, in __enter__
2020-05-21T17:14:02.2616135Z     return next(self.gen)
2020-05-21T17:14:02.2616957Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/datasette/utils/__init__.py", line 392, in temporary_docker_directory
2020-05-21T17:14:02.2617403Z     link_or_copy_directory(
2020-05-21T17:14:02.2618298Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/datasette/utils/__init__.py", line 607, in link_or_copy_directory
2020-05-21T17:14:02.2619270Z     shutil.copytree(src, dst)
2020-05-21T17:14:02.2619872Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/shutil.py", line 554, in copytree
2020-05-21T17:14:02.2622179Z     return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
2020-05-21T17:14:02.2622609Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/shutil.py", line 455, in _copytree
2020-05-21T17:14:02.2622878Z     os.makedirs(dst, exist_ok=dirs_exist_ok)
2020-05-21T17:14:02.2623157Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/os.py", line 223, in makedirs
2020-05-21T17:14:02.2623405Z     mkdir(name, mode)
2020-05-21T17:14:02.2623978Z FileExistsError: [Errno 17] File exists: '/tmp/tmpq47xi96y/datasette/templates'

@simonw simonw changed the title link_or_copy_directory() error - catches the wrong exception link_or_copy_directory() error - Invalid cross-device link May 21, 2020
@simonw
Copy link
Owner

simonw commented May 21, 2020

Actually maybe the answer here is to use dirs_exist_ok=True when calling shutil.copytree.

@aborruso
Copy link
Author

aborruso commented May 21, 2020

@simonw can I test it know? What I must do to update it?

Thank you

@simonw
Copy link
Owner

simonw commented May 21, 2020

I think this is the fix! I just landed it in master.

@aborruso you can install Datasette master to test it out like this:

pip install https://github.com/simonw/datasette/archive/cee671a58f417f827d1735b1abaa40716534ea67.zip

@aborruso
Copy link
Author

aborruso commented May 21, 2020

Thank you very much!!

I will try and I write you here

@aborruso
Copy link
Author

aborruso commented May 21, 2020

@simonw now I have

Traceback (most recent call last):
  File "/home/aborruso/.local/bin/datasette", line 8, in <module>
    sys.exit(cli())
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/publish/heroku.py", line 103, in heroku
    extra_metadata,
  File "/usr/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/publish/heroku.py", line 191, in temporary_heroku_directory
    os.path.join(tmp.name, "templates"),
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/utils/__init__.py", line 605, in link_or_copy_directory
    shutil.copytree(src, dst, copy_function=os.link, dirs_exist_ok=True)
TypeError: copytree() got an unexpected keyword argument 'dirs_exist_ok'

Do I must open a new issue?

Thank you

@simonw
Copy link
Owner

simonw commented May 26, 2020

What version of Python are you running?

@simonw
Copy link
Owner

simonw commented May 26, 2020

Oh no! It looks like dirs_exist_ok is Python 3.8 only. This is a bad fix, it needs to work on older Python's too. Re-opening.

@simonw simonw reopened this May 26, 2020
@aborruso
Copy link
Author

aborruso commented May 26, 2020

Oh no! It looks like dirs_exist_ok is Python 3.8 only. This is a bad fix, it needs to work on older Python's too. Re-opening.

Thank you very much

@simonw
Copy link
Owner

simonw commented May 27, 2020

OK, here's a new branch you can try. Install it like this:

pip install https://github.com/simonw/datasette/archive/shutil-backport.zip

If it works for you I'll merge that branch into master.

@aborruso
Copy link
Author

aborruso commented May 27, 2020

Dear @simonw thank you for your time, now IT WORKS!!!

I hope that this edit to datasette code is not for an exceptional case (my PC configuration) and that it will be useful to other users.

Thank you again!!

@simonw
Copy link
Owner

simonw commented May 27, 2020

Thanks for helping test this @aborruso!

simonw added a commit that referenced this issue May 28, 2020
@simonw
Copy link
Owner

simonw commented May 28, 2020

This is now released. pip install datasette==0.43 should get the fix.

@aborruso
Copy link
Author

aborruso commented May 28, 2020

Wow, I'm in some way very proud!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants