sagenb: SandboxViolation in webassets

[vbraun]

Processing webassets-0.9.tar.gz
Writing /tmp/easy_install-hklIUJ/webassets-0.9/setup.cfg
Running webassets-0.9/setup.py -q bdist_egg --dist-dir /tmp/easy_install-hklIUJ/webassets-0.9/egg-dist-tmp-Cwkw_W
error: SandboxViolation: chmod('/home/buildslave-sage/slave/sage_git/dot_sage/.python-eggs/Pillow-2.2.2-py2.7-linux-x86_64.egg-tmp/PIL/tmpelb5a7.$extract', 493) {}

CC: @ppurka @nexttime @rwst @kiwifb

Component: packages: standard

Reviewer: Volker Braun

Issue created by migration from https://trac.sagemath.org/ticket/16736

[vbraun]

comment:1

The reason is that webasset setup.py imports sphinx to do some crap. Which ultimately imports PIL/Pillow which causes the egg to be unpacked, violating the sandbox.

Why does sagenb come with webassets to start with? Its not used anywhere, it seems. Some packaged js files were generated with it?

[vbraun]

Branch: u/vbraun/sagenb__sandboxviolation_in_webassets

[vbraun]

comment:3

I guess the ideal solution would be to use distutils / pip instead of easy_install (who cares about binary eggs)? Though that would probably be a bigger task. For now, just allow chmod outside of the setuptools sandbox (and, really, fixing permissions isn't so bad).

Updating to the latest setuptools alone didn't fix it, but we should do that anyway.

---

New commits:

425ebcc	Update to setuptools 5.4.1
a487a0d	Allow chmod outside of the sandbox

[vbraun]

Author: Volker Braun

[vbraun]

Commit: a487a0d

[kiwifb]

comment:6

Not sure what webassets does either. Is it just me or there is a violent inflation in setuptools version numbers - worse than chrome even.

[kiwifb]

comment:7

In fact I hadn't noticed that webassets had been upgraded past 0.7.1.

[vbraun]

comment:8

The whole packages-inside-sagenb thing needs to be fixed at one point, too. But this is also not the scope of this ticket, but make a new beta that can be installed from scratch.

[vbraun]

comment:9

This seems to be caused by #16733, so I'll unmerge that upgrade until somebody can review this.

[kiwifb]

comment:11

I am not convinced by the solution I will try something in the next few hours before accepting this.

[kiwifb]

comment:12

So far I haven't been able to reproduce this. I am guessing this is a from scratch install while I am working on top of beta7 which may have an impact.

[vbraun]

comment:13

You probably need to (re-)install by hand the new docutils and pillow. Or rebuild from scratch (with #16733 merged in).

[strogdon]

comment:14

This is very curious. I merged #16733 and this ticket w/o the sandbox.patch (the patch was removed and the change was committed with git commit -a). The following sequence did not give a SandboxViolation in webassets:

• ./sage -f docutils
• ./sage -f setuptools
• ./sage -f sagenb
  But the following sequence did yield the violation (new docutils installed)
• ./sage -f pillow
• ./sage -f setuptools
• ./sage -f sagenb
  Then adding the sandbox patch in and doing
• ./sage -f setuptools
• ./sage -f sagenb
  seemed to resolve things. So where is the real problem? Maybe it doesn't matter. The sandbox patch does work.

[kiwifb]

comment:15

Interesting digging Steve. I am sure the patch works, I just think it is the wrong thing to do but I don't know any alternative, apart from trying to upgrade webassets since there is a newer release - of course it is likely it will suffer the same fate.

[vbraun]

comment:16

I tried upgrading webassets and it didn't work. The setup.py is till importing sphinx, so it'll die if that ends up pulling in pil(low)

[kiwifb]

comment:17

OK so that potentially pushes the problem in sphinx and upgrading that brings its own problems. Or may be pillow, that would be easier to try that.

[strogdon]

comment:18

I've installed sphinx-1.2.2, merged #16733 and this ticket w/o the sandbox patch and I have no SandboxViolation. However, #16396 can't be merged w/o some work. Is it possible that sphinx-1.2.2 doesn't import pillow? I believe what I've done is correct. But maybe the sandbox patch is the thing to do in the short-term.

[kiwifb]

comment:19

I really think that shooting an element of the sandbox for just one package is not good. It would be better if an exception could be made until we fix the faulty package rather than sabotaging setuptools.

[vbraun]

comment:35

FWCW, I just triggered it again while building Sage in a subdirectory of /tmp:

Processing webassets-0.10.1.tar.gz
Writing /tmp/easy_install-OSzZkb/webassets-0.10.1/setup.cfg
Running webassets-0.10.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-OSzZkb/webassets-0.10.1/egg-dist-tmp-H58hNo
error: SandboxViolation: chmod('/home/vbraun/.sage/.python-eggs/Pillow-2.2.2-py2.7-linux-x86_64.egg-tmp/PIL/tmpMF0VYS.$extract', 493) {}

[strogdon]

comment:36

This is curious. I did a fresh build under /tmp after cloning from the develop branch and I get on my gentoo box:

Processing webassets-0.10.1.tar.gz
Writing /tmp/easy_install-KBbcXK/webassets-0.10.1/setup.cfg
Running webassets-0.10.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-KBbcXK/webassets-0.10.1/egg-dist-tmp-7cFRfA
warning: no files found matching 'run_tests.py'
no previously-included directories found matching 'docs/_build'
warning: no previously-included files matching 'out.css' found under directory 'examples'
warning: no previously-included files matching 'out.js' found under directory 'examples'
no previously-included directories found matching 'examples/appengine-sdk'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '.gitignore' found anywhere in distribution
warning: no previously-included files matching '*.orig' found anywhere in distribution
warning: no previously-included files matching 'webassets-cache/*' found anywhere in distribution
warning: no previously-included files matching '.sass-cache/*' found anywhere in distribution
zip_safe flag not set; analyzing archive contents...
webassets.filter.__init__: module references __file__
Adding webassets 0.10.1 to easy-install.pth file
Installing webassets script to /tmp/sage_sandbox/sage/local/bin

Installed /tmp/sage_sandbox/sage/local/lib/python2.7/site-packages/webassets-0.10.1-py2.7.egg
Processing dependencies for webassets==0.10.1
Finished processing dependencies for webassets==0.10.1

I wonder what's up?

[kiwifb]

comment:37

OK guys that prompt 2 questions from me:

• Is it only happening when building in /tmp ?
• /tmp/easy_install-${random}? is it really the spkg being installed or easy_install acting on its own?

[vbraun]

comment:38

Only happens when building in /tmp. Seems that easy_install does something different in that case.

[vbraun]

comment:39

I'm getting this again with sagenb-11.4 on some but not all user accounts. No idea what the difference is. This is in my home directory, not compiling in /tmp.

Processing webassets-0.10.1.tar.gz
Writing /tmp/easy_install-AR1rXi/webassets-0.10.1/setup.cfg
Running webassets-0.10.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-AR1rXi/webassets-0.10.1/egg-dist-tmp-dpb_Mg
error: SandboxViolation: chmod('/home/vbraun/.sage/.python-eggs/Pillow-2.2.2-py2.7-linux-x86_64.egg-tmp/PIL/tmpZhc7i8.$extract', 493) {}

How about we just get rid of the unused webassets?

[vbraun]

comment:40

PS: the directory is created by setuptools just for webassets:

$ grep AR1rXi logs/pkgs/sagenb.log 
Writing /tmp/easy_install-AR1rXi/webassets-0.10.1/setup.cfg
Running webassets-0.10.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-AR1rXi/webassets-0.10.1/egg-dist-tmp-dpb_Mg

[kiwifb]

comment:41

Yes get rid of it, especially if it is such a headache. Is the problem happening on some distros and not others or something more subtle?

[vbraun]

comment:42

A different account can install webassets (again in its respective home directory):

Processing webassets-0.10.1.tar.gz
Writing /tmp/easy_install-Fd5CnH/webassets-0.10.1/setup.cfg
Running webassets-0.10.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Fd5CnH/webassets-0.10.1/egg-dist-tmp-f2qRbU
warning: no files found matching 'run_tests.py'
[...]

[vbraun]

Changed reviewer from Steve Trogdon to none

[kcrisman]

comment:44

Again, webassets is there so that people can use the "newui" branch of sagenb, which apparently people are indeed trying out and using (based on discussions there).

[strogdon]

comment:45

Just off the top of my head. Would it be possible to avoid the issue by removing webassets from sagenb and then installing it separately?

[vbraun]

comment:46

If you need webassets you can just "pip install webassets". But if its not used by default then don't ship it.

[kcrisman]

comment:47

If you need webassets you can just "pip install webassets". But if its not used by default then don't ship it.

I'd be open to a pull request, but whatever solution there is should probably have a good and easy way to tell a user to install webassets before they attempt to switch to that. It's also possible migeruhito's changes with that will have removed the need for webassets, I really know nothing about it beyond why it was added (included not knowing why that was actually necessary).

[vbraun]

comment:48

Then just add it to the instructions for trying out the newui branch, wherever those might be.

[jdemeyer]

Reviewer: Volker Braun

[jdemeyer]

comment:49

webassets is being removed at #14840.

[jdemeyer]

Changed author from Volker Braun to none