Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs about 'profile:host' and 'profile:build' #1629

Merged
merged 18 commits into from Mar 31, 2020
Merged
2 changes: 1 addition & 1 deletion devtools.rst
Expand Up @@ -7,7 +7,7 @@ With conan it is possible to package and deploy applications.
It is also possible that these applications are also dev-tools, like compilers (e.g. MinGW), or build systems (e.g. CMake).

This section describes how to package and run executables, and also how to package dev-tools.
Also, how to apply applications like dev-tools or even libraries (like testing frameworks) to other packages to build them from sources:``build_requires``
Also, how to apply applications like dev-tools or even libraries (like testing frameworks) to other packages to build them from sources: :ref:`build_requires`

.. toctree::
:maxdepth: 2
Expand Down
106 changes: 87 additions & 19 deletions devtools/build_requires.rst
Expand Up @@ -4,7 +4,7 @@ Build requirements
==================

There are some requirements that don't feel natural to add to a package recipe. For example, imagine that you had a ``cmake/3.4`` package in
Conan. Would you add it as a requirement to the ``ZLib`` package, so it will install cmake first in order to build ``Zlib``?
Conan. Would you add it as a requirement to the ``zlib`` package, so it will install cmake first in order to build ``zlib``?

In short:

Expand All @@ -20,12 +20,13 @@ In short:

To address these needs Conan implements ``build_requires``.


Declaring build requirements
----------------------------

Build requirements can be declared in profiles, like:

.. code-block:: text
.. code-block:: ini
:caption: my_profile

[build_requires]
Expand Down Expand Up @@ -74,6 +75,57 @@ attribute, the one inside the ``build_requirements()`` method will prevail.
As a rule of thumb, downstream defined values always override upstream dependency values. If some build requirement is defined in the
profile, it will overwrite the build requirements defined in package recipes that have the same package name.


.. _build_requires_context:

Build and Host contexts
-----------------------

Conan v1.24 differentiates between the ``build`` context and the ``host`` context in the dependency graph (read more about
the meaning of ``host`` and ``build`` platforms in the :ref:`cross building <cross_building>` section) **when the user
supplies two profiles** to the command line using the ``--profile:build`` and ``--profile:host`` arguments:

* The **host context** is populated with the root package (the one specified in the :command:`conan install` or :command:`conan create` command),
all its requirements and the build requirements forced to be in the host context.
* The **build context** contains the rest of build requirements and all of them in the profiles. This category typically
includes all the :ref:`dev tools <create_installer_packages>` like CMake, compilers, linkers,...


Build requirements declared in the recipes can be forced to stay in the host context, this is needed for testing libraries that will
be linked to the generated library or other executable we want to deploy to the ``host`` platform, for example:

.. code-block:: python

class MyPkg(ConanFile):
build_requires = "nasm/2.14" # 'build' context (nasm.exe will be available)

def build_requirements(self):
self.build_requires("protobuf/3.6.1") # 'build' context (protoc.exe will be available)
self.build_requires("gtest/0.1", force_host_context=True) # 'host' context (our library will link with it)


.. image:: ../images/xbuild/conan-gtest_nasm.png
:width: 500 px
:align: center


Take into account that the same package (executable or library) can appear two times in the graph, in the ``host`` and
in the ``build`` context, with different package IDs. Conan will propagate the proper information to the consumers:

* Build requirements in the ``host`` context will propagate like any other requirement, all the ``cpp_info`` will be
available in the ``deps_cpp_info["xxx"]`` object (``env_info`` and ``user_info`` won't be propagated).
* Build requirements in the ``build`` context will propagate all the ``env_info`` and Conan will also populate the
environment variables ``DYLD_LIBRARY_PATH``, ``LD_LIBRARY_PATH`` and ``PATH`` with the corresponding information from
the ``cpp_info`` object. All these information will be available in the ``deps_cpp_info`` object.


.. important::

If no ``--profile:build`` is provided, all build requirements will belong to the one and only context and they will share
their dependencies with the libraries we are building. In this scenario all the build requirements propagate ``user_info``,
``cpp_info`` and ``env_info`` to the consumer's ``deps_user_info``, ``deps_cpp_info`` and ``deps_env_info``.


Properties of build requirements
--------------------------------

Expand All @@ -83,10 +135,8 @@ The behavior of ``build_requires`` is the same irrespective if they are defined
they will not even be checked for existence.
- Options and environment variables declared in the profile as well as in the command line will affect the build requirements for packages.
In that way, you can define, for example, for the ``cmake/3.16.3`` package which CMake version will be installed.
- Build requirements will be activated for matching packages via the ``deps_cpp_info`` and ``deps_env_info`` members. So, include
directories, library names, compile flags (CFLAGS, CXXFLAGS, LINKFLAGS), sysroot, etc. will be applied from the build requirement's
package ``self.cpp_info`` values. The same for ``self.env_info``: variables such as ``PATH``, ``PYTHONPATH``, and any other environment
variables will be applied to the matching patterns and activated as environment variables.
- Build requirements will be activated for matching packages, see the section above about :ref:`build requires context <build_requires_context>`
to know the information that this package will propagate to its consumers.
- Build requirements can also be transitive. They can declare their own requirements, both normal requirements and their own build
requirements. Normal logic for dependency graph resolution applies, such as conflict resolution and dependency overriding.
- Each matching pattern will produce a different dependency graph of build requirements. These graphs are cached so that they are only
Expand All @@ -97,21 +147,29 @@ The behavior of ``build_requires`` is the same irrespective if they are defined
- Can also use version-ranges, like ``Tool/[>0.3]@user/channel``.
- Build requirements are not listed in :command:`conan info` nor are represented in the graph (with :command:`conan info --graph`).

Testing libraries
-----------------

One example of a build requirement could be a testing framework, which is implemented as a library. Let's call it ``mytest_framework``, an
existing Conan package.
Example: testing framework and build tool
-----------------------------------------

One example of build requirement is a testing framework implemented as a library, another good example is a build tool used
in the compile process. Let's call them ``mytest_framework`` and ``cmake_turbo``, and imagine we already have a package available
for both of them.

Build requirements can be checked for existence (whether they've been applied) in the recipes, which can be useful for conditional logic in
the recipes. In this example, we could have one recipe with the following ``build()`` method:

.. code-block:: python

def build_requirements(self):
if self.options.enable_testing:
self.build_requires("mytest_framework/0.1@user/channel", force_host_context=True)

def build(self):
cmake = CMake(self)
enable_testing = "mytest_framework" in self.deps_cpp_info.deps
cmake.configure(defs={"ENABLE_TESTING": enable_testing})
# Use our own 'cmake_turbo' if it is available
use_cmake_turbo = "cmake_turbo" in self.deps_env_info.deps
cmake_executable = "cmake_turbo" if use_cmake_turbo else None
cmake = CMake(self, cmake_program=cmake_executable)
cmake.configure(defs={"ENABLE_TESTING": self.options.enable_testing})
cmake.build()
if enable_testing:
cmake.test()
Expand All @@ -135,22 +193,32 @@ And the package *CMakeLists.txt*:
COMMAND example)
endif()

This package recipe will not retrieve the ``mytest_framework`` nor build the tests, for normal installation:
This package recipe won't retrieve the ``cmake_turbo`` package for normal installation:

.. code-block:: bash

$ conan install .

But if the following profile is defined:

.. code-block:: text
:caption: mytest_profile
.. code-block:: ini
:caption: use_cmake_turbo_profile

[build_requires]
mytest_framework/0.1@user/channel
cmake_turbo/0.1@user/channel

then the install command will retrieve the ``cmake_turbo`` and use it:

.. code-block:: bash

$ conan install . --profile=use_cmake_turbo_profile


then the install command will retrieve the ``mytest_framework``, build and run the tests:
Although the previous line would work it is preferred to use the feature from Conan v1.24 and provide
two profiles to the command line, that way the build requirements in the ``build`` context won't
interfer with the ``host`` graph if they share common requirements (see :ref:`section about dev tools <create_installer_packages>`).
It can also be needed if cross compiling (see :ref:`section about cross compiling <cross_building_build_requires>`).

.. code-block:: bash

$ conan install . --profile=mytest_profile
$ conan install . --profile:host=use_cmake_turbo_profile --profile:build=build_machine