inveniomanage collect OSError

[jalavik]

OSError happens after running inveniomanage collect.

Offending file is:
/home/jlavik/envs/pudemosite/var/invenio.base-instance/static/js/workflows/actions/approval.js

Fix suggested in #240.

See full output here (I added a ipdb statement to print some values):
https://gist.github.com/jalavik/d718d8ae404af25d042b

CC @greut @jirikuncar

[jalavik]

Sorry, I forgot to put the original exception

Traceback (most recent call last):
  File "/home/jlavik/envs/pudemosite/bin/inveniomanage", line 9, in <module>
    load_entry_point('Invenio==1.9999', 'console_scripts', 'inveniomanage')()
  File "/home/jlavik/envs/pudemosite/src/invenio/invenio/base/manage.py", line 103, in main
    manager.run()
  File "/home/jlavik/envs/pudemosite/local/lib/python2.7/site-packages/flask_script/__init__.py", line 405, in run
    result = self.handle(sys.argv[0], sys.argv[1:])
  File "/home/jlavik/envs/pudemosite/local/lib/python2.7/site-packages/flask_script/__init__.py", line 384, in handle
    return handle(app, *positional_args, **kwargs)
  File "/home/jlavik/envs/pudemosite/local/lib/python2.7/site-packages/flask_script/commands.py", line 145, in handle
    return self.run(*args, **kwargs)
  File "/home/jlavik/envs/pudemosite/src/invenio/invenio/ext/script/__init__.py", line 137, in wrapper
    result = f(*args, **kwds)
  File "/home/jlavik/envs/pudemosite/local/lib/python2.7/site-packages/flask_collect/collect.py", line 82, in collect
    return self.collect(verbose=verbose)
  File "/home/jlavik/envs/pudemosite/local/lib/python2.7/site-packages/flask_collect/collect.py", line 95, in collect
    return storage.run()
  File "/home/jlavik/envs/pudemosite/src/invenio/invenio/ext/collect/storage/link.py", line 50, in run
    os.symlink(f, destination)
OSError: [Errno 17] File exists

[greut]

thanks for the pdb output... gist truncated it so I missed it at first.

(<Flask 'invenio.base'>,
 '/home/jlavik/envs/pudemosite/var/invenio.base-instance/static/js/workflows/actions/approval.js',
 'js/workflows/actions/approval.js')

The problem seems to be that the invenio.base blueprint points to the instance_paths static folder instead of its own!?! Here is the code collecting the files: flask_collect.storage.base.

I don't get what's going on here. What's sure is that adding a if won't solve anything, only hidding it away.

[jalavik]

Yeah, I am not sure. All I know it gives this error. I can probably fix it by cleaning the folder. Just wanted to give the heads-up.

Anyway, asking os.symlink to symlink a to b where a == b is also not cool, so we could do something about it.

[jirikuncar]

Anyway, asking os.symlink to symlink a to b where a == b is also not cool, so we could do something about it.

assert a != b should be enough

[jalavik]

Alright, so if you want to reproduce it, I think I have found how. It happens when renaming a static file. It is possible (on my machine) like this:

workon pu
cdvirtualenv var/invenio.base-instance/static/js
rm -rf ./*
cdvirtualenv src/invenio
git checkout pu
inveniomanage collect

Now rename any static file

git mv invenio/modules/deposit/static/js/deposit/form.js invenio/modules/deposit/static/js/deposit/form_new_name.js

Run inveniomanage collect again and voila:

js/formatter/quicktags.js is already present!
Traceback (most recent call last):
  File "/home/jlavik/envs/pudemosite/bin/inveniomanage", line 9, in <module>
    load_entry_point('Invenio==1.9999', 'console_scripts', 'inveniomanage')()
  File "/home/jlavik/envs/pudemosite/src/invenio/invenio/base/manage.py", line 103, in main
    manager.run()
  File "/home/jlavik/envs/pudemosite/local/lib/python2.7/site-packages/flask_script/__init__.py", line 405, in run
    result = self.handle(sys.argv[0], sys.argv[1:])
  File "/home/jlavik/envs/pudemosite/local/lib/python2.7/site-packages/flask_script/__init__.py", line 384, in handle
    return handle(app, *positional_args, **kwargs)
  File "/home/jlavik/envs/pudemosite/local/lib/python2.7/site-packages/flask_script/commands.py", line 145, in handle
    return self.run(*args, **kwargs)
  File "/home/jlavik/envs/pudemosite/src/invenio/invenio/ext/script/__init__.py", line 137, in wrapper
    result = f(*args, **kwds)
  File "/home/jlavik/envs/pudemosite/local/lib/python2.7/site-packages/flask_collect/collect.py", line 82, in collect
    return self.collect(verbose=verbose)
  File "/home/jlavik/envs/pudemosite/local/lib/python2.7/site-packages/flask_collect/collect.py", line 95, in collect
    return storage.run()
  File "/home/jlavik/envs/pudemosite/src/invenio/invenio/ext/collect/storage/link.py", line 50, in run
    os.symlink(f, destination)
OSError: [Errno 17] File exists

Is this true for any of you? Otherwise it may be my environment.

[greut]

Bug confirmed!

[greut]

The troublemaker is this little guy:

for bp in [self.collect.app] + list(self.collect.blueprints.values()):

https://github.com/klen/Flask-Collect/blob/develop/flask_collect/storage/base.py#L21

self.collect.app gives it var/invenio.base-instance/static.

[jirikuncar]

It was me O:-) https://github.com/klen/Flask-Collect/blame/develop/flask_collect/storage/base.py#L21