Feature/qbs build helper #8125

Psy-Kai · 2020-11-30T09:02:36Z

Changelog: Feature: Add QbsToolchain and a Qbs build helper class (currently working for Mcus, not for Android or iOS).
Docs: conan-io/docs#1978

Closes #8079

Refer to the issue that supports this Pull Request.
If the issue has missing info, explain the purpose/use case/pain/need that covers this Pull Request.
I've read the Contributing guide.
I've followed the PEP8 style guides for Python code.
I've opened another PR in the Conan docs repo to the develop branch, documenting this one.

^{Note: By default this PR will skip the slower tests and will use a limited set of python versions. Check here how to increase the testing level by writing some tags in the current PR body text.}

…re/qbs-build-helper

set when calling conan install

qbs-build-helper

PIC, cxxLanguageVersion

LDFLAGS without -Wl

memsharded

This is looking good, thanks very much.
Please have a look to the comments, and also to the broken CI, as it seems the import "toolchain.qbs.generic" is wrong

memsharded · 2020-11-30T09:58:17Z

conans/client/build/qbs.py

+from conans.errors import ConanException
+
+
+class QbsException(ConanException):


Do not redefine a class with same name again, this is problematic.
Actually, it is not necessary to define a QbsException, all the other build systems do not have a custom exception for this.

I just defined a custom exception because I wanted to prepend the origin of the exception to the message (qbs build helper and qbs toolchain). Without the custom exception I would need to add this string to every message manually. But if this is not desired I can remove the exceptions.

conans/client/build/qbs.py

memsharded · 2020-11-30T10:00:11Z

conan/tools/qbs/generic.py

+
+
+def _flags_from_env():
+    flags_from_env = {}


The way we are going to use environment variables in toolchains is still not defined, so this might be broken in the future. I am ok with adding it now this way, as long as it is clear that this integration is very experimental and subject to future breaking change.

I the environment should not be used, how do you plan to pass the compiler path or cross compiling flags? The current cross compiling goto way seems like environment variables: https://docs.conan.io/en/latest/systems_cross_building/cross_building.html#conan-v1-24-and-newer

memsharded · 2020-11-30T10:03:41Z

conan/tools/qbs/generic.py

+        self._conanfile = conanfile
+        _setup_toolchains(conanfile)
+        self._profile_values_from_setup = (
+            _read_qbs_toolchain_from_config(conanfile))


I am a bit confused about this round-trip conversion, but probably it is because I don't know Qbs enough. If the config data is already present in a qbs format file, why is it necessary to parse it, and translate it to a conan-toolchain format one? Couldn't it be used directly from that original qbs origin?

I use the qbs command qbs-setup-toolchains here because it contains some logic on how to detect compilers and create a qbs-profile out of it. I did not want to mirror the logic here. The problem is that this tool creates qbs-settings which are by default user/system wide and are stored in a .ini file containing binary Qt data. So parsing this settings file cannot be easily be done by in python. This is why I use another qbs command qbs-config to read the values in a easy to parse format.

conans/__init__.py

QbsException

Psy-Kai · 2020-11-30T13:21:42Z

@memsharded Could it be that some tests run in an environment where CC is set?

This makes the module python 3.5 compatible

Psy-Kai · 2020-12-10T09:25:14Z

@memsharded Could you help me here? I don't know how to fix the error in the python2.7 test.

memsharded · 2020-12-17T00:21:11Z

Hi @Psy-Kai

Sorry for the delay. I am having a look, and doing some changes to align to latest changes in toolchains. I will push changes soon.

Missed to change test according to change

…-review

memsharded · 2020-12-20T21:46:56Z

Hi @Psy-Kai

I did a few days ago a PR with a review in https://github.com/Psy-Kai/conan/pull/1. Please have a look when possible.

Feature/qbs build helper review

Psy-Kai · 2020-12-21T14:35:02Z

@memsharded There is still some problem on python2.7 regarding some *IO type class (like StringIO). I am not skilled enough to find the problem, sorry :/

memsharded · 2020-12-21T15:08:16Z

@memsharded There is still some problem on python2.7 regarding some *IO type class (like StringIO). I am not skilled enough to find the problem, sorry :/

Yeah, don't worry, it is not evident, I am having a look, hopefully I can fix it, I will let you know.

memsharded

I think this is a good start, as a first iteration, knowing that:

The new toolchains and helpers are all experimental. They will change in things like folders layouts, or environment management.
conan.tools.xxx should only import thing from itself, things like tools.vcvars() usage will need to be refactored. Also new msvc compiler is coming soon, will need to be taken into account.
It seems there are some implementation details/style that could be improved (more idiomatic python), but that can be done later.
At some point it will be necessary to add some integration testing that actually calls qbs

I will request some other reviews too, but lets aim to merge this for Conan 1.33

SSE4 · 2020-12-28T12:28:50Z

conan/tools/qbs/qbs.py

+    command_list = ['config:%s' % name]
+    for key, value in config_dict.items():
+        if type(value) is bool:
+            if value:


just FYI: shorter syntax:

b = 'true' if value else 'false'

(no need to change)

memsharded · 2020-12-28T12:48:10Z

Merged, this will be in Conan 1.33.

Please, try to contribute the docs and update the first comment Docs: https://github.com/conan-io/docs/pull/XXXX section, the changelog is generated automatically from this. A few hints:

Explicitly add that the toolchain, helper, etc, are experimental and subject to change
The documentation can be concise in a first iteration, and completed later, maybe just the reference section in tools.qbs.

Psy-Kai added 12 commits November 19, 2020 19:30

Implemented Qbs Build Helper

aecd29a

Merge remote-tracking branch 'refs/remotes/origin/develop' into featu…

0bed999

…re/qbs-build-helper

Added implementation of qbs toolchain

595cdaa

Merge remote-tracking branch 'refs/remotes/origin/develop' into featu…

ae0b1c6

…re/qbs-build-helper

Finished first implementation of Qbs build helper

c2cfb5b

Changed build_folder to install_folder since build_folder might not be

1f99061

set when calling conan install

Merge remote-tracking branch 'refs/remotes/origin/develop' into feature/

c3de7ac

qbs-build-helper

Added detection of buildVariant, architecture, optimization, sysroot,

3b5989b

PIC, cxxLanguageVersion

Added property use_toolchain_profile to compile with specified profile

72c82aa

Set cpp.linkerFlags for LDFLAGS with -Wl and cpp.driverLinkerFlags for

6c52917

LDFLAGS without -Wl

Made use_toolchain_profile non static

56b4c24

Removed prints

47f036d

memsharded requested changes Nov 30, 2020

View reviewed changes

Psy-Kai added 6 commits November 30, 2020 11:54

Put settings dir into quotes to support settings dir with spaces

7813894

Moved qbs to conan/tools

1d988b5

Renamed exception to be independent

d4c5c75

Use the right import paths

38e7992

Put settings-dir parameter in quotes in test

1ba7782

Modified tests to assert for QbsToolchainException instead of

d436aec

QbsException

Psy-Kai added 2 commits November 30, 2020 20:52

Use shlex.split and made LinkerFlagsParser much easier

2cf5fd6

This makes the module python 3.5 compatible

Just convert True/False to true/false with _bool

b9d4c09

memsharded added 2 commits December 17, 2020 00:41

Merge branch 'develop' into feature/qbs-build-helper

9516939

working

a705782

memsharded and others added 4 commits December 17, 2020 01:26

working

9f69914

Fix MSVC+ClangCL compatibility

84081f1

Adjusted test to last change

0ef5587

Missed to change test according to change

Merge branch 'feature/qbs-build-helper' into feature/qbs-build-helper…

85ee854

…-review

memsharded added 2 commits December 17, 2020 09:40

Merge branch 'develop' into feature/qbs-build-helper

21ac2b9

Merge branch 'feature/qbs-build-helper' into feature/qbs-build-helper…

c9f1d30

…-review

memsharded and others added 2 commits December 21, 2020 12:11

returned back to qbs namespace

ac1901c

Merge pull request #1 from memsharded/feature/qbs-build-helper-review

31cf1b3

Feature/qbs build helper review

fixing tests PY2

828a836

memsharded approved these changes Dec 22, 2020

View reviewed changes

memsharded added this to the 1.33 milestone Dec 22, 2020

memsharded requested review from danimtb and SSE4 December 22, 2020 08:50

danimtb approved these changes Dec 22, 2020

View reviewed changes

SSE4 reviewed Dec 28, 2020

View reviewed changes

SSE4 approved these changes Dec 28, 2020

View reviewed changes

memsharded merged commit 5260b65 into conan-io:develop Dec 28, 2020

Psy-Kai mentioned this pull request Jan 11, 2021

Add documentation for qbs build helper and qbs toolchain conan-io/docs#1978

Merged

dependabot bot mentioned this pull request Mar 11, 2021

Bump conan from 1.27.0 to 1.34.1 librevault/librevault#185

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/qbs build helper #8125

Feature/qbs build helper #8125

Psy-Kai commented Nov 30, 2020 •

edited by czoido

memsharded left a comment

memsharded Nov 30, 2020

Psy-Kai Nov 30, 2020

memsharded Nov 30, 2020

Psy-Kai Nov 30, 2020

memsharded Nov 30, 2020

Psy-Kai Nov 30, 2020

Psy-Kai commented Nov 30, 2020

Psy-Kai commented Dec 10, 2020

memsharded commented Dec 17, 2020

memsharded commented Dec 20, 2020

Psy-Kai commented Dec 21, 2020

memsharded commented Dec 21, 2020

memsharded left a comment

SSE4 Dec 28, 2020

memsharded commented Dec 28, 2020

		from conans.errors import ConanException


		class QbsException(ConanException):

Feature/qbs build helper #8125

Feature/qbs build helper #8125

Conversation

Psy-Kai commented Nov 30, 2020 • edited by czoido

memsharded left a comment

Choose a reason for hiding this comment

memsharded Nov 30, 2020

Choose a reason for hiding this comment

Psy-Kai Nov 30, 2020

Choose a reason for hiding this comment

memsharded Nov 30, 2020

Choose a reason for hiding this comment

Psy-Kai Nov 30, 2020

Choose a reason for hiding this comment

memsharded Nov 30, 2020

Choose a reason for hiding this comment

Psy-Kai Nov 30, 2020

Choose a reason for hiding this comment

Psy-Kai commented Nov 30, 2020

Psy-Kai commented Dec 10, 2020

memsharded commented Dec 17, 2020

memsharded commented Dec 20, 2020

Psy-Kai commented Dec 21, 2020

memsharded commented Dec 21, 2020

memsharded left a comment

Choose a reason for hiding this comment

SSE4 Dec 28, 2020

Choose a reason for hiding this comment

memsharded commented Dec 28, 2020

Psy-Kai commented Nov 30, 2020 •

edited by czoido