Template files are searched for in the wrong location

[raek]

I'm trying to use pdoc on a installed-two-weeks-ago Debian Jessie machine, but I get this message:

$ pdoc --html some-module
Traceback (most recent call last):
  File "/usr/local/bin/pdoc", line 487, in <module>
    html_out(module)
  File "/usr/local/bin/pdoc", line 334, in html_out
    raise e
IOError: [Errno 2] No template at any of: /usr/share/pdoc/templates/html.mako, /usr/local/lib/python2.7/dist-packages/templates/html.mako

I installed python (2.7.6) and pip via apt-get:

sudo apt-get install python
sudo apt-get install python-pip

I installed pdoc via pip:

sudo pip install pdoc

I can see that the html.mako template file exists in the /usr/local/share/pdoc tree, but pdoc seems to look for it in /usr/share/pdoc. After a quick glance at the pdoc code, it seems to get the prefix from the sys.prefix attribute, which seems to have the value /usr on my computer. Apparently pip used another prefix when it installed the files.

I don't think I have set up things in a weird way, but please tell me if you want me to try something. I would be happy to help debugging this, but I don't know where to start.

[BurntSushi]

Thank you so much for the bug report.

This is unfortunate. I was under the impression that data_files in setup.py are guaranteed to be interpreted relative to sys.prefix. From the docs:

Each (directory, files) pair in the sequence specifies the installation directory and the files to install there. If directory is a relative path, it is interpreted relative to the installation prefix (Python’s sys.prefix for pure-Python packages, sys.exec_prefix for packages that contain extension modules).

And indeed, I'm using relative paths:

    data_files=[('share/pdoc', ['README.md', 'longdesc.rst',
                                'UNLICENSE', 'INSTALL', 'CHANGELOG']),
                ('share/pdoc/doc', ['doc/pdoc.m.html']),
                ('share/pdoc/templates', glob('templates/*')),
               ],

So it strikes me as exceedingly odd that the data files were installed with a /usr/local prefix while your sys.prefix is actually /usr. Is it possible that this prefix changes when installing it as root?

I wonder if it's possible to force the prefix. Maybe the --root option for pip install will help.

Hang tight for a work-around...

[BurntSushi]

If you copy the files in the templates directory from /usr/local/share/pdoc/templates/* to ~/.config/pdoc/templates/, then I think that should work.

[raek]

Yes, strange indeed. Looks like pip is supposed to install the file relative to sys.prefix, and your code is trying to read them relative to sys.prefix. So my conclusion is that the problem is not in pdoc...

Thanks for the workaround! I will try to reinstall pip in a different way first, though. I have a feeling that I'm not using the most common method.

[BurntSushi]

I have a feeling that I'm not using the most common method.

The method you described in your initial comment is exactly how it should be installed. :-)

But certainly, if you try other things and meet with success, I'd love to hear about them.

[raek]

This seems to be caused by Debian's packaging of Python: https://wiki.debian.org/Python (section "Deviations from upstream")

distutils setup scripts install files in /usr/local/ not
sys.prefix (which is normally /usr/). This is because /usr/ is
reserved for files installed from Debian packages. Note that
/usr/local/lib/pythonX.Y/dist-packages is in sys.path so that
modules not installed from Debian packages can still be accessed
by the system Python. Tools like debhelper pass the
--install-layout=deb option to the setup script while building a
Debian package so that its installs files into /usr/ not
/usr/local/.

So the promise that data_files are put in sys.prefix is not kept on Debian systems... :-(

[raek]

Installing pip by downloading and running get-pip.py did not make any difference (which makes sense now that we know about the Debian thing).

[BurntSushi]

Ug. There's got to be some sort of work-around, otherwise every Python package that uses data_files will break on Debian.

[BurntSushi]

Ah, what if I switch to package_data instead of data_files? The data will be stored inside the package directory wherever it's installed, which should be unambiguously available with pdoc.__path__.

Unfortunately, this requires switching from a module to a package. I don't mind doing that if it works around Debian's policies.

[raek]

package_data looks like robust solution.

Why don't you want to switch from module to package? Simplicity? Aesthetics? You could rename pdoc.py to pdoc/main.py and make a pdoc/__init__.py with content:

from .main import *

But perhaps this wasn't the issue...

[BurntSushi]

Simplicity. It's nice to have a single file instead of a package.

But this isn't a good enough reason IMO if it's breaking installs on a popular Linux distro.

And get those relative imports off my lawn! I'm fine with just slapping everything into pdoc/__init__.py. :-)

[raek]

I see. :-) I really appreciate you making these changes!

[BurntSushi]

@raek I'm hoping this fixes things. Please let me know if it doesn't and I'll re-open the issue.

(This marks the release of pdoc 0.2.0.)

[raek]

@BurntSushi: It works now. Thanks a lot!