Making Carla Installable with pip install

[pcmoritz]

Dear all,

are there any plans to make it possible to install Carla with a simple

pip install carlapy

using Python wheels?

and then use it by importing it and having a carla server automatically started in the background?

That would make it easier to get started for newcomers :)

Do you foresee any challenges with this (licensing of the binary artifacts for example)?

-- Philipp.

cc @robertnishihara @pschafhalter @gy20073

[nsubiron]

I see a few issues with having the server started in the background.

• Pip installation will need to download +2GB of the binary package.
• It's quite common having the client and server in different machines.
• Right now we only have server binaries for Ubuntu, while the client alone can run anywhere.
• In my opinion, it's too performance demanding as to have it in the background.

[danielsuo]

Agree. I could imagine, however, the Carla Python client living in PyPI.

[danielsuo]

To follow up on that thought, we already turned the python client into a package for our own purposes, so we'd definitely welcome this change. Maybe worth creating a separate repository (e.g., carla-smulator/carla-client or carla-python-client) to decouple the release cycles?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[r7vme]

I think having carla client within pypi repos will simplify work for many developers and researchers.

+1 to have that.

Current approach

pip install git+https://github.com/carla-simulator/carla.git@0.8.2#"egg=carla_client&subdirectory=PythonClient"
pip install -r https://raw.githubusercontent.com/carla-simulator/carla/0.8.2/PythonClient/requirements.txt

[esteve]

@nsubiron would it make sense to spin off the Python client to a separate repository? Perhaps that could help releasing it into PyPI. I agree with @danielsuo and @r7vme that being able to install the Python API via PIP would be great. For example, the client library could easily be installed in lightweight clients without having to download the entire Carla tarball first.

[nsubiron]

@esteve We considered this in the past, but client and server are tightly coupled, a lot of code is reused on both sides. I even think at this point there is more client-side code in the repo than server-side. We are moving everything we can to the client-side for performance reasons, as we can distribute clients among several machines.

Being able to install the carla module with pip would be indeed a great feature, but we are still under heavy development, I would wait until we settle the current API (version 0.10.x maybe?), otherwise is going to be a versioning nightmare. Also we need to solve how to distribute native code to all the Python versions.

[nsubiron]

I'm going to reopen this issue actually, cause eventually we want to have this feature.

[esteve]

@nsubiron

Also we need to solve how to distribute native code to all the Python versions.

That can be done user-side when they install the client library from PyPI:

https://docs.python.org/3.7/extending/building.html#building-c-and-c-extensions-with-distutils

but if there's a lot of code in the client, yeah it'll be difficult to distribute it easily because users would end up compiling a lot of stuff. Also, is there any Unreal code in the client or is it all PyGame? If there is, that could be a problem as well.

[nsubiron]

The client code depends "only" on Boost and rpclib, but about 100 C++ files need to be compiled. We only use PyGame in the example scripts, but it's not required by the "carla" module.

We actually use setup.py (setuptools) to compile the module, but a lot of dependencies need to be there. I guess we can find a hybrid model, distributing a compiled library with everything independent on the Python version (that's our LibCarla library, but some work needs to be done encapsulating dependencies), then the user can compile the pure CPython code and link against our lib, that's only 1 file to compile.

[esteve]

@nsubiron perhaps what could be done is to provide packages only for certain platforms (e.g. Ubuntu, Windows 10, etc.) and whoever else wants to use it, they can fetch the library from PyPI and build it themselves.

[praveen-palanisamy]

Spoiler alert: pip install carla is coming!

I have built PyPI compatible (pip install-able) (binary) wheels for CARLA 0.9.3, 0.9.4, 0.9.5 each for the following Python versions: 2.7, 3.4, 3.5, 3.6, 3.7.

You can download the Linux Python wheels for these (and future releases) from here: https://github.com/praveen-palanisamy/carla/releases or use the direct links below:

Python version	CARLA 0.9.3	CARLA 0.9.4	CARLA 0.9.5
2.7	download	download	download
3.4	download	download	download
3.5	download	download	download
3.6	download	download	download
3.7	download	download	download

[praveen-palanisamy]

Currently the packages are on the test.pypi.org index.
You can pip install like this:
pip install -i https://test.pypi.org/simple/ carla==VERSION
where VERSION is the CARLA release version.

For example, to install CARLA 0.9.4, use:
pip install -i https://test.pypi.org/simple/ carla==0.9.4

I will run some final tests (help to test will be very much appreciated) to make sure that the wheels are EXACTLY matching with the tagged release versions and then release it on PyPI so that we can all just pip install carla!

[praveen-palanisamy]

@nsubiron: The releases at https://github.com/praveen-palanisamy/carla/releases also have the patch for each of the tagged release versions. A PR (for upcoming releases) will be more involved as it needs several changes (especially for the source distributions) including changes to the CI/Travis/Jenkis pipelines but, I can help when it's time to revamp.

[nsubiron]

@praveen-palanisamy thanks, that's great! we're very interested in adding this to Carla for upcoming releases, maybe you can do a PR draft and we'll start working on it too.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[germanros1987]

@praveen-palanisamy hope things are going well. Are you willing to maintain the pip effort that you started?

[praveen-palanisamy]

Thanks for checking in @germanros1987 . I will be happy to help maintain the pip packages. I ran into difficulties in isolating the Python client library from the CARLA server code with the 0.9.6 release. IIRC, the fix needed some refactoring in this repo to separate out the server & client side code in addition to new code for integrating the build and push steps for PyPI.

I wouldn't be able to maintain an up-to-date version on PyPI without CARLA's core dev team's support. I created the draft PR #1608 with my changes to work with the core dev team to take it forward by addressing the main To-Dos listed in the description here.

[germanros1987]

@melviso-osvf could you have a look at #1608 and make a version that works for 0.9.9 and newer versions?

[Gr8ayu]

Can we get an update on carla installable via Pip ?

[dsche]

It seems that the setuptools package after version 51.3.3 does not provide an easy_install script, which might affect the urgency of this issue.

[qhaas]

As @dsche alluded, egg isn't an 'official' package for python, has apparently been deprecated, and is being removed in newer versions of python/setuptools. Some of our newer python deployments can't 'install' eggs.

[xavisolesoft]

*.whl was implemented some time ago, so I close this ticket.