Unable to build lxml

I'm trying to use the pip install -r requirements.txt but it seems to be failing on the lxml module. Here is the output:

Collecting lxml
  Using cached lxml-3.4.2.tar.gz
    Building lxml version 3.4.2.
    Building without Cython.
    Using build configuration of libxslt 1.1.28
    /usr/lib/python3.4/distutils/dist.py:260: UserWarning: Unknown distribution option: 'bugtrack_url'
      warnings.warn(msg)
Installing collected packages: lxml
  Running setup.py install for lxml
    Building lxml version 3.4.2.
    Building without Cython.
    Using build configuration of libxslt 1.1.28
    building 'lxml.etree' extension
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -O0 -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/libxml2 -I/tmp/pip-build-d_yi2of0/lxml/src/lxml/includes -I/usr/include/python3.4m -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-3.4/src/lxml/lxml.etree.o -w
    x86_64-linux-gnu-gcc: internal compiler error: Killed (program cc1)
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-4.8/README.Bugs> for instructions.
    /usr/lib/python3.4/distutils/dist.py:260: UserWarning: Unknown distribution option: 'bugtrack_url'
      warnings.warn(msg)
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 4
    Complete output from command /usr/bin/python3.4 -c "import setuptools, tokenize;__file__='/tmp/pip-build-d_yi2of0/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-laizk7bh-record/install-record.txt --single-version-externally-managed --compile:
    Building lxml version 3.4.2.

    Building without Cython.

    Using build configuration of libxslt 1.1.28

    running install

    running build

    running build_py

    creating build

    creating build/lib.linux-x86_64-3.4

    creating build/lib.linux-x86_64-3.4/lxml

    copying src/lxml/ElementInclude.py -> build/lib.linux-x86_64-3.4/lxml

    copying src/lxml/sax.py -> build/lib.linux-x86_64-3.4/lxml

    copying src/lxml/_elementpath.py -> build/lib.linux-x86_64-3.4/lxml

    copying src/lxml/pyclasslookup.py -> build/lib.linux-x86_64-3.4/lxml

    copying src/lxml/usedoctest.py -> build/lib.linux-x86_64-3.4/lxml

    copying src/lxml/builder.py -> build/lib.linux-x86_64-3.4/lxml

    copying src/lxml/__init__.py -> build/lib.linux-x86_64-3.4/lxml

    copying src/lxml/doctestcompare.py -> build/lib.linux-x86_64-3.4/lxml

    copying src/lxml/cssselect.py -> build/lib.linux-x86_64-3.4/lxml

    creating build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/__init__.py -> build/lib.linux-x86_64-3.4/lxml/includes

    creating build/lib.linux-x86_64-3.4/lxml/html

    copying src/lxml/html/diff.py -> build/lib.linux-x86_64-3.4/lxml/html

    copying src/lxml/html/ElementSoup.py -> build/lib.linux-x86_64-3.4/lxml/html

    copying src/lxml/html/_diffcommand.py -> build/lib.linux-x86_64-3.4/lxml/html

    copying src/lxml/html/clean.py -> build/lib.linux-x86_64-3.4/lxml/html

    copying src/lxml/html/defs.py -> build/lib.linux-x86_64-3.4/lxml/html

    copying src/lxml/html/usedoctest.py -> build/lib.linux-x86_64-3.4/lxml/html

    copying src/lxml/html/formfill.py -> build/lib.linux-x86_64-3.4/lxml/html

    copying src/lxml/html/builder.py -> build/lib.linux-x86_64-3.4/lxml/html

    copying src/lxml/html/__init__.py -> build/lib.linux-x86_64-3.4/lxml/html

    copying src/lxml/html/_html5builder.py -> build/lib.linux-x86_64-3.4/lxml/html

    copying src/lxml/html/_setmixin.py -> build/lib.linux-x86_64-3.4/lxml/html

    copying src/lxml/html/html5parser.py -> build/lib.linux-x86_64-3.4/lxml/html

    copying src/lxml/html/soupparser.py -> build/lib.linux-x86_64-3.4/lxml/html

    creating build/lib.linux-x86_64-3.4/lxml/isoschematron

    copying src/lxml/isoschematron/__init__.py -> build/lib.linux-x86_64-3.4/lxml/isoschematron

    copying src/lxml/lxml.etree.h -> build/lib.linux-x86_64-3.4/lxml

    copying src/lxml/lxml.etree_api.h -> build/lib.linux-x86_64-3.4/lxml

    copying src/lxml/includes/dtdvalid.pxd -> build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/config.pxd -> build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/etreepublic.pxd -> build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/xmlschema.pxd -> build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/tree.pxd -> build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/uri.pxd -> build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/xpath.pxd -> build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/xinclude.pxd -> build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/htmlparser.pxd -> build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/schematron.pxd -> build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/xmlparser.pxd -> build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/xmlerror.pxd -> build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/relaxng.pxd -> build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/xslt.pxd -> build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/c14n.pxd -> build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/etree_defs.h -> build/lib.linux-x86_64-3.4/lxml/includes

    copying src/lxml/includes/lxml-version.h -> build/lib.linux-x86_64-3.4/lxml/includes

    creating build/lib.linux-x86_64-3.4/lxml/isoschematron/resources

    creating build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/rng

    copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/rng

    creating build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl

    copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl

    copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl

    creating build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

    running build_ext

    building 'lxml.etree' extension

    creating build/temp.linux-x86_64-3.4

    creating build/temp.linux-x86_64-3.4/src

    creating build/temp.linux-x86_64-3.4/src/lxml

    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -O0 -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/libxml2 -I/tmp/pip-build-d_yi2of0/lxml/src/lxml/includes -I/usr/include/python3.4m -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-3.4/src/lxml/lxml.etree.o -w

    x86_64-linux-gnu-gcc: internal compiler error: Killed (program cc1)

    Please submit a full bug report,

    with preprocessed source if appropriate.

    See <file:///usr/share/doc/gcc-4.8/README.Bugs> for instructions.

    /usr/lib/python3.4/distutils/dist.py:260: UserWarning: Unknown distribution option: 'bugtrack_url'

      warnings.warn(msg)

    error: command 'x86_64-linux-gnu-gcc' failed with exit status 4

    ----------------------------------------
    Command "/usr/bin/python3.4 -c "import setuptools, tokenize;__file__='/tmp/pip-build-d_yi2of0/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-laizk7bh-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-d_yi2of0/lxml

Any ideas?

[edwardslabs]

According to the install instructions here: https://github.com/CloudBotIRC/CloudBot/wiki/Installing-on-*nix Do you have the following packages installed? Especially python3.4-dev:

python3.4-dev git libenchant-dev libxml2-dev libxslt-dev zlib1g-dev

Yes. Apparently the issue is that lxml takes a large amount of memory to
compile, and my 128MB RAM VPS couldn't handle it.

On Wed, Mar 4, 2015 at 5:30 PM, Andy Edwards notifications@github.com
wrote:

According to the install instructions here:
https://github.com/CloudBotIRC/CloudBot/wiki/Installing-on-*nix Do you
have the following packages installed? Especially python3.4-dev:

python3.4-dev git libenchant-dev libxml2-dev libxslt-dev zlib1g-dev

—
Reply to this email directly or view it on GitHub
#115 (comment)
.

[dmptrluke]

Ouch. Yeah, LXML is not an easy module to build, because it links with some fairly big XML libraries. Have you tried using your OS package manager to install a prebuilt distribution of LXML? On ubuntu at least, python3-lxml seemed to work last time I tried.

[foxlet]

@hiiambo - That issue pertains to having too little RAM while using the gcc compiler. If your VPS cannot be expanded, you can try using some extra swap during the build:

Note that this procedure requires disk space on your VPS's drive

dd if=/dev/zero of=/path/to/swap.img bs=1024k count=1000       # Generates 1GB swapfile.
mkswap /path/to/swap.img
swapon /path/to/swap.img

Proceed with your build, then after you're done:

swapoff /path/to/swap.img

[daboross]

The killed message is indeed the system running out of memory and thus killing the build process. Just installing python3-lxml via apt-get or your package manager should work.

Be sure to also comment out lxml in dependencies.txt, then re-run pip install -r requirements.txt. If you don't comment out lxml, pip will try to build it again, even though python3-lxml is already installed.

CloudBot doesn't really have any particular dependency on the pip version of lxml, just that some version be installed on the system - so it is fine to just install python3-lxml and not via pip.

[jamierocks]

I get the same issue for psutil

[daboross]

If you have a python3-psutil package on your system, you can do the same thing as with lxml.

[jamierocks]

It seems I got it whenever something require Python.h :|

[daboross]

@jamierocks Yes, you will need to install python3-dev in order to install all of the dependencies.

As per https://github.com/CloudBotIRC/CloudBot/wiki/Installing-on-*nix, you need python3.4-dev git libenchant-dev libxml2-dev libxslt-dev and zlib1g-dev installed before you can run pip.

[foxlet]

Since no response from the original poster was given after a while, I will assume this was solved.