Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parallel download #1583

Merged
merged 1 commit into from Mar 9, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 7 additions & 0 deletions configuration/download_cache.rst
Expand Up @@ -84,3 +84,10 @@ a lot of storage. If you are using this feature, provide for a sufficiently larg
At the moment, it is only a folder. You can clean the cached artifacts just by removing that folder and its contents. You might also be able to
run scripts and jobs that remove old artifacts only. If you do such operations, please make sure that there are not other Conan processes using
it simultaneously, or they might fail.


.. note::

Installation of binaries can be accelerated setting up parallel downloads with the ``general.parallel_download``
**experimental** configuration in :ref:`conan_conf`. You might want to try combining both the parallel download and
the download cache for extra speed.
6 changes: 6 additions & 0 deletions reference/commands/consumer/install.rst
Expand Up @@ -264,3 +264,9 @@ they should match, otherwise, an error will be raised.
$ conan install . version@user/testing # OK
$ conan install . pkg/version@user/testing # OK
$ conan install pkg/version@user/testing user/channel # Error, first arg is not a path


.. note::

Installation of binaries can be accelerated setting up parallel downloads with the ``general.parallel_download``
**experimental** configuration in :ref:`conan_conf`.
5 changes: 5 additions & 0 deletions reference/commands/creator/create.rst
Expand Up @@ -152,3 +152,8 @@ The ``reference`` field can be:

In case of installing a pre-built binary, steps from 5 to 11 will be skipped. Note that ``deploy()`` method is only used in
:command:`conan install`.

.. note::

Installation of binaries can be accelerated setting up parallel downloads with the ``general.parallel_download``
**experimental** configuration in :ref:`conan_conf`.
8 changes: 8 additions & 0 deletions reference/config_files/conan.conf.rst
Expand Up @@ -20,6 +20,7 @@ The typical location of the **conan.conf** file is the directory ``~/.conan/``:
sysrequires_sudo = True # environment CONAN_SYSREQUIRES_SUDO
request_timeout = 60 # environment CONAN_REQUEST_TIMEOUT (seconds)
default_package_id_mode = semver_direct_mode # environment CONAN_DEFAULT_PACKAGE_ID_MODE
# parallel_download = 8 # experimental download binaries in parallel
# full_transitive_package_id = 0
# retry = 2 # environment CONAN_RETRY
# retry_wait = 5 # environment CONAN_RETRY_WAIT (seconds)
Expand Down Expand Up @@ -138,6 +139,13 @@ it will be ``semver_direct_mode``, but can change to any value defined in :ref:`
The ``full_transitive_package_id`` changes the way package IDs are computed regarding transitive dependencies.
By default, if not specified will be disabled (``0``). Read more about it in :ref:`full_transitive_package_id`.

The ``parallel_download`` configuration defines the number of threads to be used to do parallel downloads of
different binaries. This happens when dependencies are installed (``conan install``, ``conan create``) and when
multiple binaries for the same package are retrieved via ``conan download`` command. This is an **experimental**
feature, subject to change. It is known that the output is still not clean, and will be mangled when using multiple
threads. Please report on https://github.com/conan-io/conan/issues about performance gains, and other issues.
You might want to try this one in combination with the ``storage.download_cache`` configuration (see below.)


The ``cmake_***`` variables will declare the corresponding CMake variable when you use the
:ref:`cmake generator<cmake_generator>` and the :ref:`CMake build tool<cmake_reference>`.
Expand Down