'Document' object has no attribute 'findall' #1742

thobryan · 2014-04-22T01:02:28Z

After upgrading ubuntu 13.10 to 14.04. All packages that I am trying to install is getting the following error:

Exception: Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
  status = self.run(options, args)
File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 278, in run
  requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle) 
File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1177, in prepare_files 
  url = finder.find_requirement(req_to_install, upgrade=self.upgrade) 
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 256, in find_requirement
  page_versions.extend(self._package_versions(page.links, req.name.lower())) 
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 432, in _package_versions 
  for link in self._sort_links(links): 
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 422, in _sort_links 
  for link in links: 
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 769, in links 
  for anchor in self.parsed.findall(".//a"):
AttributeError: 'Document' object has no attribute 'findall'

Storing debug log for failure in /root/.pip/pip.log

The text was updated successfully, but these errors were encountered:

thobryan · 2014-04-24T01:58:54Z

One workaround was to execute "easy_install pip".

spinza · 2014-05-10T00:16:32Z

Getting the same trying to install a package using pip.

dstufft · 2014-05-10T00:17:08Z

Can you give the exact command you used?

spinza · 2014-05-10T00:22:23Z

sudo pip install flexget

thobryan · 2014-05-10T00:24:04Z

Ironssouw, did you upgrade Ubuntu to 14.04? (In my case I just got this error after upgrade)

spinza · 2014-05-10T00:24:31Z

Yes.

dstufft · 2014-05-10T00:25:45Z

What is the value of pip --version.

spinza · 2014-05-10T00:26:49Z

pip 1.5.4 from /usr/lib/python2.7/dist-packages (python 2.7)

dstufft · 2014-05-10T00:27:44Z

Ok, and the output of pkg -s python-pip ?

dstufft · 2014-05-10T00:27:50Z

er, dpkg -s python-pip.

spinza · 2014-05-10T00:29:58Z

$ sudo dpkg -s python-pip
Package: python-pip
Status: install ok installed
Priority: optional
Section: python
Installed-Size: 468
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Architecture: all
Version: 1.5.4-1
Depends: python (>= 2.7), python (<< 2.8), python:any (>= 2.7.1-0ubuntu2), ca-certificates, python-colorama, python-distlib, python-html5lib, python-pkg-resources, python-setuptools (>= 0.6c1), python-six, python-requests
Recommends: build-essential, python-dev-all (>= 2.6)
Description: alternative Python package installer
 pip is a replacement for easy_install, and is intended to be an improved
 Python package installer.  It integrates with virtualenv, doesn't do partial
 installs, can save package state for replaying, can install from non-egg
 sources, and can install from version control repositories.
Original-Maintainer: Debian Python Modules Team <python-modules-team@lists.alioth.debian.org>
Homepage: http://www.pip-installer.org/

spinza · 2014-05-10T00:31:30Z

I removed and reinstalled pip also. Same problem.

dstufft · 2014-05-10T00:34:34Z

Hmm, I can't reproduce on a fresh 14.04 image... Maybe try uninstalling python-html5lib and reinstalling that?

spinza · 2014-05-10T00:38:01Z

Still the same. This is an upgrade from 13.10 to 14.04.

dstufft · 2014-05-10T00:38:15Z

Ok, i'll try that then!

dstufft · 2014-05-10T01:04:25Z

Nope, still can't reproduce ;(

Ivoz · 2014-05-10T15:56:54Z

For more output that could help us narrow down the issue:

What path are you running it under? (could you give a reproduction of the shell commands you issued?)
Could you try it A) without root, by adding the --user flag instead; and B) try installing inside a virtualenv (python-virtualenv)?
Also give a paste of ~/.pip/pip.log (from under root's home and yours)?

schmidtc · 2014-05-26T15:01:55Z

I'm hitting this same problem. Seems to be triggered by installing certain packages. I can reproduce this using a fresh Amazon EC2 instance loaded with "Ubuntu Server 14.04 LTS (PV)". And running the following…

ubuntu:~$ sudo apt-get install python-pip -y
… (clipped) ...
ubuntu:~$ sudo pip install boto
Downloading/unpacking boto
  Downloading boto-2.28.0-py2.py3-none-any.whl (1.2MB): 1.2MB downloaded
Installing collected packages: boto
Successfully installed boto
Cleaning up…
ubuntu:~$ sudo pip uninstall boto -y
Uninstalling boto:
  Successfully uninstalled boto
ubuntu:~$ sudo pip install html5lib==0.90
Downloading/unpacking html5lib==0.90
  Downloading html5lib-0.90.tar.gz (86kB): 86kB downloaded
  Running setup.py (path:/tmp/pip_build_root/html5lib/setup.py) egg_info for package html5lib

Installing collected packages: html5lib
  Found existing installation: html5lib 0.999
    Uninstalling html5lib:
      Successfully uninstalled html5lib
  Running setup.py install for html5lib

Successfully installed html5lib
Cleaning up...
ubuntu:~$ sudo pip install boto
Downloading/unpacking boto
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1177, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 256, in find_requirement
    page_versions.extend(self._package_versions(page.links, req.name.lower()))
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 432, in _package_versions
    for link in self._sort_links(links):
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 422, in _sort_links
    for link in links:
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 769, in links
    for anchor in self.parsed.findall(".//a"):
AttributeError: 'Document' object has no attribute 'findall'

Storing debug log for failure in /home/ubuntu/.pip/pip.log

In summary, apt-get install python-pip. Install and uninstall boto to demonstrate pip is working. Installing html5lib breaks pip.

sudo pip uninstall html5lib seems to fix the problem.

dstufft · 2014-05-26T15:21:07Z

Ah I see.

So pip itself bundles some software, html5lib being one of those, on various Linux distros they unbundle that software and cause pip to import it normally instead of using it's own bundled copy.

What this means is that if you install an incompatible version of one of pip's dependencies into the global Python, then you'll break pip because pip will import that and attempt to use that instead of the correct one installed by apt-get.

dstufft · 2014-06-03T21:41:36Z

Going to close this. This is working as expected for the Debian/Ubuntu based systems which debundle things from inside of pip. If you still feel this should be different then the best avenue to take this up with is on the Debian or Ubuntu bug trackers.

spinza · 2014-06-04T07:35:13Z

I didn't understand the comments about what is wrong. How do I fix it?

On Monday, May 26, 2014, Donald Stufft notifications@github.com wrote:

Ah I see.

So pip itself bundles some software, html5lib being one of those, on
various Linux distros they unbundle that software and cause pip to import
it normally instead of using it's own bundled copy.

What this means is that if you install an incompatible version of one of
pip's dependencies into the global Python, then you'll break pip because
pip will import that and attempt to use that instead of the correct one
installed by apt-get.

—
Reply to this email directly or view it on GitHub
#1742 (comment).

__
Louis Rossouw
lrossouw@gmail.com | m: +27 82 498 7214

Ivoz · 2014-06-04T12:41:37Z

@lrossouw try pip installing packages in a virtualenv or using the --user flag instead of sudo installing them as system packages.

The issue seems to be caused by debian's modification of its pip, so we can't address it ourselves.

cjd · 2014-07-08T03:23:24Z

I hit this as well and "sudo pip uninstall html5lib" fixed it for me

munirhossain · 2014-08-26T21:07:13Z

Nothing helped for me after I upgraded to Ubuntu 14.04 form Ubuntu 12.04 to fix the problem. Then I completely removed all the pythons from my system and then in installed again and installed pip. Still virtualenv was not working and complaining that virtualenv==1.9.1 not found!! then I found that in the /usr/local/bin/virtualenv file there is strict requirement of 1.9.1 version. Since I removed all the python and related packages and reinstalled the version of virtualenv has been updated, and there is no more virtualenv-1.9.1. So what I did next is that removing the strictly checking of version 1.9.1 in the /usr/local/bin/virtualenv and changed as following. Then it helped. OOH-HOO it took my important 3 hours. Hope people can fix in 3 minutes.

#!/usr/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'virtualenv>=1.9.1','console_scripts','virtualenv'
__requires__ = 'virtualenv>=1.9.1'
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.exit(
        load_entry_point('virtualenv>=1.9.1', 'console_scripts', 'virtualenv')()
    )

Ivoz · 2014-08-30T09:15:39Z

@munirhossain it seems that for you the script file /usr/local/bin/virtualenv was left over after uninstallation of python-things, whereas it should have been deleted. Thereafter it took precedence on your path and was continually trying to look for a ghost installation of virtualenv 1.9. If you removed it and reinstalled virtualenv through your preferred method then a proper script should end up on your path, to use normally.

brycenesbitt · 2014-09-24T06:51:51Z

My machine has no /usr/local/bin/virtualenv

After sudo pip uninstall html5lib I get:

Downloading/unpacking six from https://pypi.python.org/packages/py2.py3/s/six/six-1.8.0-py2.py3-none-any.whl#md5=2f5f96148c68f3c1611f489678a8b445
  Downloading six-1.8.0-py2.py3-none-any.whl
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1259, in prepare_files
    )[0]
IndexError: list index out of range

Storing debug log for failure in /root/.pip/pip.log

brycenesbitt · 2014-09-24T06:56:02Z

Downstream bug report for Ubuntu 14.04:
https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1373254
If you see this also, click "affects me also".

roldandvg · 2014-10-08T21:26:01Z

Had the same problem and solvent running the following command:

sudo easy_install-U pip

For this you must have previously installed this utility with:

sudo aptitude install python-setuptools

I then proceeded to install the package with pip and all good.

snehalparmar · 2014-10-21T06:56:27Z

easy_install pip worked for me.

harrypotterismyname · 2015-03-22T07:03:47Z

@roldandvg approach works for me

sarepal · 2015-08-05T20:16:58Z

"sudo pip uninstall html5lib" fixed it for me as well

yonas · 2015-08-12T20:42:35Z

"sudo pip uninstall html5lib" worked for me too.

kuroyuki · 2016-02-25T05:51:25Z

Seems the issue was fixed in the latest Pip version but apt-get has old one.
I installed pip from pypa.io
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py
And updated to latest version with :
sudo pip install --upgrade pip
Now I able to install packages w/o problems.

dstufft closed this as completed Jun 3, 2014

msabramo mentioned this issue Sep 26, 2014

Sometimes when repushing a Python app, I get a traceback tsuru/tsuru#928

Closed

JordonPhillips mentioned this issue Feb 15, 2016

Installation fails on Debian stable aws/aws-cli#1796

Closed

lock bot added the auto-locked Outdated issues that have been locked by automation label Jun 4, 2019

lock bot locked as resolved and limited conversation to collaborators Jun 4, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

'Document' object has no attribute 'findall' #1742

'Document' object has no attribute 'findall' #1742

thobryan commented Apr 22, 2014

thobryan commented Apr 24, 2014

spinza commented May 10, 2014

dstufft commented May 10, 2014

spinza commented May 10, 2014

thobryan commented May 10, 2014

spinza commented May 10, 2014

dstufft commented May 10, 2014

spinza commented May 10, 2014

dstufft commented May 10, 2014

dstufft commented May 10, 2014

spinza commented May 10, 2014

spinza commented May 10, 2014

dstufft commented May 10, 2014

spinza commented May 10, 2014

dstufft commented May 10, 2014

dstufft commented May 10, 2014

Ivoz commented May 10, 2014

schmidtc commented May 26, 2014

dstufft commented May 26, 2014

dstufft commented Jun 3, 2014

spinza commented Jun 4, 2014

Ivoz commented Jun 4, 2014

cjd commented Jul 8, 2014

munirhossain commented Aug 26, 2014

Ivoz commented Aug 30, 2014

brycenesbitt commented Sep 24, 2014

brycenesbitt commented Sep 24, 2014

roldandvg commented Oct 8, 2014

snehalparmar commented Oct 21, 2014

harrypotterismyname commented Mar 22, 2015

sarepal commented Aug 5, 2015

yonas commented Aug 12, 2015

kuroyuki commented Feb 25, 2016

'Document' object has no attribute 'findall' #1742

'Document' object has no attribute 'findall' #1742

Comments

thobryan commented Apr 22, 2014

thobryan commented Apr 24, 2014

spinza commented May 10, 2014

dstufft commented May 10, 2014

spinza commented May 10, 2014

thobryan commented May 10, 2014

spinza commented May 10, 2014

dstufft commented May 10, 2014

spinza commented May 10, 2014

dstufft commented May 10, 2014

dstufft commented May 10, 2014

spinza commented May 10, 2014

spinza commented May 10, 2014

dstufft commented May 10, 2014

spinza commented May 10, 2014

dstufft commented May 10, 2014

dstufft commented May 10, 2014

Ivoz commented May 10, 2014

schmidtc commented May 26, 2014

dstufft commented May 26, 2014

dstufft commented Jun 3, 2014

spinza commented Jun 4, 2014

Ivoz commented Jun 4, 2014

cjd commented Jul 8, 2014

munirhossain commented Aug 26, 2014

Ivoz commented Aug 30, 2014

brycenesbitt commented Sep 24, 2014

brycenesbitt commented Sep 24, 2014

roldandvg commented Oct 8, 2014

snehalparmar commented Oct 21, 2014

harrypotterismyname commented Mar 22, 2015

sarepal commented Aug 5, 2015

yonas commented Aug 12, 2015

kuroyuki commented Feb 25, 2016