zipimport.zipimporter.get_filename used for python<2.7

[kratenko]

Just put pyramid project on webhoster server using python 2.6.6 (r266:84292, Dec 26 2010, 22:48:11) [GCC 4.4.5]. Since it has almost none of my needed packages, I was putting an egg-collection on the server as well, (containing venusian version: venusian-1.0a7-py2.6). Since I did not want to upload 4000 files, I used compressed eggs

wsgi failed with:

[Wed Jan 30 01:46:20 2013] [error] [client xxx.xxx.xxx.xxx]   File "build/bdist.linux-i686/egg/venusian/__init__.py", line 187, in scan
[Wed Jan 30 01:46:20 2013] [error] [client xxx.xxx.xxx.xxx]     fn = loader.get_filename()
[Wed Jan 30 01:46:20 2013] [error] [client xxx.xxx.xxx.xxx] AttributeError: 'zipimport.zipimporter' object has no attribute 'get_filename'

which is no surprise, since get_filename was introduced in python2.7. It look's like that code is supposed to be for python3.3, but still got executed in 2.6 there.

Update:
Just tried out venusian-1.0a6-py2.6 (since the error producing line is new to a7), I got:

[Wed Jan 30 02:15:54 2013] [error] [client xxx]   File "build/bdist.linux-i686/egg/venusian/__init__.py", line 181, in scan
[Wed Jan 30 02:15:54 2013] [error] [client xxx]     module_type = loader.etc[2]
[Wed Jan 30 02:15:54 2013] [error] [client xxx] AttributeError: 'zipimport.zipimporter' object has no attribute 'etc'

So, the expected attribute etc must be missing. Not sure why, but that's the problem in both cases, I guess.

Update 2:
I was able do get my app running, by putting all eggs not as compressed files, but as directories (since problem was related to zipping, obviously). Still I think, there is something going wrong in venusian there.

[msabramo]

I just ran into this too on Python 2.6.7. It seems to work just fine for me on Python 2.6.5, but on Python 2.6.7 the zipimporter doesn't seem to have get_filename. I think I have a fix which is to use archive instead of get_filename and I will send a PR shortly...

[tseaver]

Python 2.6.5 and 2.6.7 have no substantive differences in the zipimport.c extension module. The following are from versions just built from source:

$ ./python 
Python 2.6.5 (r265:79063, Apr 11 2013, 16:32:23) 
[GCC 4.6.3] on linux3
Type "help", "copyright", "credits" or "license" for more information.
>>> from zipimport import zipimporter
>>> zipimporter.get_filename
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: type object 'zipimport.zipimporter' has no attribute 'get_filename'
>>> zipimporter._get_filename
<method '_get_filename' of 'zipimport.zipimporter' objects>


$ ./python 
Python 2.6.7 (r267:88850, Apr 11 2013, 16:18:46) 
[GCC 4.6.3] on linux3
Type "help", "copyright", "credits" or "license" for more information.
>>> from zipimport import zipimporter
>>> zipimporter.get_filename
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: type object 'zipimport.zipimporter' has no attribute 'get_filename'
>>> zipimporter._get_filename
<method '_get_filename' of 'zipimport.zipimporter' objects>

[tseaver]

The codepath which is triggering that error isn't even supposed to be triggered on Python < 3.3:

https://github.com/Pylons/venusian/blob/master/venusian/__init__.py#L187

[msabramo]

Yeah that code path shouldn't be triggered on Python < 3.3 according to the comment. It checks for etc. For some reason, I don't have that etc attribute on Python 2.6.7 on OS X.

[msabramo]

Here's how my Python 2.6.7 zipimporter looks:

(py26.venv)marca@marca-mac:/opt/webapp/profilesvc/src/profilesvc$ cat > test.txt
test
(py26.venv)marca@marca-mac:/opt/webapp/profilesvc/src/profilesvc$ zip test.zip test.txt
  adding: test.txt (stored 0%)
(py26.venv)marca@marca-mac:/opt/webapp/profilesvc/src/profilesvc$ file test.zip
test.zip: Zip archive data, at least v1.0 to extract
(py26.venv)marca@marca-mac:/opt/webapp/profilesvc/src/profilesvc$ python
Python 2.6.7 (r267:88850, Oct 11 2012, 20:15:00)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from zipimport import zipimporter
>>> z = zipimporter('test.zip')
>>> z.etc
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'zipimport.zipimporter' object has no attribute 'etc'
>>> z.get_filename()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'zipimport.zipimporter' object has no attribute 'get_filename'
>>> z.archive
'test.zip'
>>> dir(z)
['__class__', '__delattr__', '__doc__', '__format__', '__getattribute__', '__hash__', 
'__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', 
'__sizeof__', '__str__', '__subclasshook__', '_files', '_get_filename', 'archive', 
'find_module', 'get_code', 'get_data', 'get_source', 'is_package', 'load_module', 
'prefix']

[msabramo]

Hmmm, zipimporter in 2.6 and 2.7 look alike to me. I wonder if the problem is that in this particular environment with the problem I have zipped eggs in my site-packages and in my working envs, I didn't...

[tseaver]

Looking more closely: I don't think venusian actually supports zipped eggs (although a patch to make it work would be cool).

[kratenko]

If venusian does not support zipped eggs, than imho it is the wrong tool for what pyramid uses it for. It is highly impracticle (and takes about ten times longer for the feel of it), to upload the needed packages as directories via ftp. Some 1000 files in directory structures feels much more messy than putting up a few egg files.

(sorry for closing this issue - I hit the wrong button. I'm still new to github...)

[msabramo]

I don't have much of a need to use zipped eggs, so I solved this problem by making sure that the two zipped eggs that crept into my site-packages were not zipped (e.g.: added zip_safe=False to setup.py for the offending packages). That works for me.

I don't need zipped eggs so I probably won't tackle adding that to Venusian though if someone else did, I might help test/review it if it needed more eyes.

[mcdonc]

Indeed, Venusian does not currently support zipped packages properly. Note that this isn't really a zipped eggs problem, as you can put zipfiles on the PYTHONPATH that are not eggs. I'd encourage anyone who wants to see Venusian support this to add the feature with a set of tests that makes sure it works as well as non-zipped packages.

[mcdonc]

A note for a future implementer.. on Python 2.7, you can test if the importer is a zipimporter in walk_packages by doing::

    if hasattr(importer, 'archive'):
        # it's in a zipfile

Note however that the name provided during iter_modules does not include the parent package in this circumstance. Why? You tell me.

[msabramo]

See #43

Does Venusian support zipped packages now?

[wichert]

@msabramo #45 makes zipped packages work on Python 3 for me.

[digitalresistor]

Version 1.1.0 has just been uploaded to pypi: https://pypi.python.org/pypi/venusian/1.1.0