New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/qbs build helper #8125
Merged
memsharded
merged 31 commits into
conan-io:develop
from
Psy-Kai:feature/qbs-build-helper
Dec 28, 2020
Merged
Feature/qbs build helper #8125
Changes from all commits
Commits
Show all changes
31 commits
Select commit
Hold shift + click to select a range
aecd29a
Implemented Qbs Build Helper
Psy-Kai 0bed999
Merge remote-tracking branch 'refs/remotes/origin/develop' into featu…
Psy-Kai 595cdaa
Added implementation of qbs toolchain
Psy-Kai ae0b1c6
Merge remote-tracking branch 'refs/remotes/origin/develop' into featu…
Psy-Kai c2cfb5b
Finished first implementation of Qbs build helper
Psy-Kai 1f99061
Changed build_folder to install_folder since build_folder might not be
Psy-Kai c3de7ac
Merge remote-tracking branch 'refs/remotes/origin/develop' into feature/
Psy-Kai 3b5989b
Added detection of buildVariant, architecture, optimization, sysroot,
Psy-Kai 72c82aa
Added property `use_toolchain_profile` to compile with specified profile
Psy-Kai 6c52917
Set cpp.linkerFlags for LDFLAGS with -Wl and cpp.driverLinkerFlags for
Psy-Kai 56b4c24
Made use_toolchain_profile non static
Psy-Kai 47f036d
Removed prints
Psy-Kai 7813894
Put settings dir into quotes to support settings dir with spaces
Psy-Kai 1d988b5
Moved qbs to conan/tools
Psy-Kai d4c5c75
Renamed exception to be independent
Psy-Kai 38e7992
Use the right import paths
Psy-Kai 1ba7782
Put settings-dir parameter in quotes in test
Psy-Kai d436aec
Modified tests to assert for QbsToolchainException instead of
Psy-Kai 2cf5fd6
Use shlex.split and made LinkerFlagsParser much easier
Psy-Kai b9d4c09
Just convert True/False to true/false with _bool
Psy-Kai 9516939
Merge branch 'develop' into feature/qbs-build-helper
memsharded a705782
working
memsharded 9f69914
working
memsharded 84081f1
Fix MSVC+ClangCL compatibility
Psy-Kai 0ef5587
Adjusted test to last change
Psy-Kai 85ee854
Merge branch 'feature/qbs-build-helper' into feature/qbs-build-helper…
memsharded 21ac2b9
Merge branch 'develop' into feature/qbs-build-helper
memsharded c9f1d30
Merge branch 'feature/qbs-build-helper' into feature/qbs-build-helper…
memsharded ac1901c
returned back to qbs namespace
memsharded 31cf1b3
Merge pull request #1 from memsharded/feature/qbs-build-helper-review
Psy-Kai 828a836
fixing tests PY2
memsharded File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
from conan.tools.qbs.qbstoolchain import QbsToolchain | ||
from conan.tools.qbs.qbs import Qbs |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,97 @@ | ||
import os | ||
|
||
from conans import tools | ||
from conans.errors import ConanException | ||
|
||
|
||
def _configuration_dict_to_commandlist(name, config_dict): | ||
command_list = ['config:%s' % name] | ||
for key, value in config_dict.items(): | ||
if type(value) is bool: | ||
if value: | ||
b = 'true' | ||
else: | ||
b = 'false' | ||
command_list.append('%s:%s' % (key, b)) | ||
else: | ||
command_list.append('%s:%s' % (key, value)) | ||
return command_list | ||
|
||
|
||
class Qbs(object): | ||
def __init__(self, conanfile, project_file=None): | ||
# hardcoded name, see qbs toolchain | ||
self.use_toolchain_profile = 'conan_toolchain_profile' | ||
self._conanfile = conanfile | ||
self._set_project_file(project_file) | ||
self.jobs = tools.cpu_count() | ||
self._configuration = dict() | ||
|
||
def _set_project_file(self, project_file): | ||
if not project_file: | ||
self._project_file = self._conanfile.source_folder | ||
else: | ||
self._project_file = project_file | ||
|
||
if not os.path.exists(self._project_file): | ||
raise ConanException('Qbs: could not find project file %s' % self._project_file) | ||
|
||
def add_configuration(self, name, values): | ||
self._configuration[name] = values | ||
|
||
def build(self, products=None): | ||
products = products or [] | ||
args = [ | ||
'--no-install', | ||
'--build-directory', self._conanfile.build_folder, | ||
'--file', self._project_file, | ||
] | ||
|
||
if products: | ||
args.extend(['--products', ','.join(products)]) | ||
|
||
args.extend(['--jobs', '%s' % self.jobs]) | ||
|
||
if self.use_toolchain_profile: | ||
args.append('profile:%s' % self.use_toolchain_profile) | ||
|
||
for name in self._configuration: | ||
config = self._configuration[name] | ||
args.extend(_configuration_dict_to_commandlist(name, config)) | ||
|
||
cmd = 'qbs build %s' % (' '.join(args)) | ||
self._conanfile.run(cmd) | ||
|
||
def build_all(self): | ||
args = [ | ||
'--no-install', | ||
'--build-directory', self._conanfile.build_folder, | ||
'--file', self._project_file, | ||
'--all-products' | ||
] | ||
|
||
args.extend(['--jobs', '%s' % self.jobs]) | ||
|
||
if self.use_toolchain_profile: | ||
args.append('profile:%s' % self.use_toolchain_profile) | ||
|
||
for name in self._configuration: | ||
config = self._configuration[name] | ||
args.extend(_configuration_dict_to_commandlist(name, config)) | ||
|
||
cmd = 'qbs build %s' % (' '.join(args)) | ||
self._conanfile.run(cmd) | ||
|
||
def install(self): | ||
args = [ | ||
'--no-build', | ||
'--clean-install-root', | ||
'--install-root', self._conanfile.install_folder, | ||
'--file', self._project_file | ||
] | ||
|
||
for name in self._configuration: | ||
args.append('config:%s' % name) | ||
|
||
cmd = 'qbs install %s' % (' '.join(args)) | ||
self._conanfile.run(cmd) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,264 @@ | ||
import shlex | ||
import platform | ||
import textwrap | ||
|
||
from io import StringIO | ||
from jinja2 import Template | ||
from conans import tools | ||
from conans.errors import ConanException | ||
from conans.util.files import save | ||
|
||
_profile_name = 'conan' | ||
_profiles_prefix_in_config = 'profiles.%s' % _profile_name | ||
|
||
_architecture = { | ||
'x86': 'x86', | ||
'x86_64': 'x86_64', | ||
'ppc32be': 'ppc', | ||
'ppc32': 'ppc', | ||
'ppc64le': 'ppc64', | ||
'ppc64': 'ppc64', | ||
'armv4': 'arm', | ||
'armv4i': 'arm', | ||
'armv5el': 'arm', | ||
'armv5hf': 'arm', | ||
'armv6': 'arm', | ||
'armv7': 'arm', | ||
'armv7hf': 'arm', | ||
'armv7s': 'arm', | ||
'armv7k': 'arm', | ||
'armv8': 'arm64', | ||
'armv8_32': 'arm64', | ||
'armv8.3': 'arm64', | ||
'sparc': 'sparc', | ||
'sparcv9': 'sparc64', | ||
'mips': 'mips', | ||
'mips64': 'mips64', | ||
'avr': 'avr', | ||
's390': 's390x', | ||
's390x': 's390x', | ||
'asm.js': None, | ||
'wasm': None, | ||
'sh4le': 'sh' | ||
} | ||
_build_variant = { | ||
'Debug': 'debug', | ||
'Release': 'release', | ||
'RelWithDebInfo': 'profiling', | ||
'MinSizeRel': 'release' | ||
} | ||
_optimization = { | ||
'MinSizeRel': 'small' | ||
} | ||
_cxx_language_version = { | ||
'98': 'c++98', | ||
'gnu98': 'c++98', | ||
'11': 'c++11', | ||
'gnu11': 'c++11', | ||
'14': 'c++14', | ||
'gnu14': 'c++14', | ||
'17': 'c++17', | ||
'gnu17': 'c++17', | ||
'20': 'c++20', | ||
'gnu20': 'c++20' | ||
} | ||
|
||
|
||
def _bool(b): | ||
if b is None: | ||
return None | ||
return str(b).lower() | ||
|
||
|
||
def _env_var_to_list(var): | ||
return shlex.split(var) | ||
|
||
|
||
def _check_for_compiler(conanfile): | ||
compiler = conanfile.settings.get_safe('compiler') | ||
if not compiler: | ||
raise ConanException('Qbs: need compiler to be set in settings') | ||
|
||
if compiler not in ['Visual Studio', 'gcc', 'clang']: | ||
raise ConanException('Qbs: compiler {} not supported'.format(compiler)) | ||
|
||
|
||
def _default_compiler_name(conanfile): | ||
# needs more work since currently only windows and linux is supported | ||
compiler = conanfile.settings.get_safe('compiler') | ||
the_os = conanfile.settings.get_safe('os') | ||
if the_os == 'Windows': | ||
if compiler == 'gcc': | ||
return 'mingw' | ||
if compiler == 'Visual Studio': | ||
if tools.msvs_toolset(conanfile) == 'ClangCL': | ||
return 'clang-cl' | ||
return 'cl' | ||
if compiler == 'clang': | ||
return 'clang-cl' | ||
raise ConanException('unknown windows compiler') | ||
|
||
return compiler | ||
|
||
|
||
def _settings_dir(conanfile): | ||
return '%s/conan_qbs_toolchain_settings_dir' % conanfile.install_folder | ||
|
||
|
||
def _setup_toolchains(conanfile): | ||
if tools.get_env('CC'): | ||
compiler = tools.get_env('CC') | ||
else: | ||
compiler = _default_compiler_name(conanfile) | ||
|
||
env_context = tools.no_op() | ||
if platform.system() == 'Windows': | ||
if compiler in ['cl', 'clang-cl']: | ||
env_context = tools.vcvars(conanfile) | ||
|
||
with env_context: | ||
cmd = 'qbs-setup-toolchains --settings-dir "%s" %s %s' % ( | ||
_settings_dir(conanfile), compiler, _profile_name) | ||
conanfile.run(cmd) | ||
|
||
|
||
def _read_qbs_toolchain_from_config(conanfile): | ||
s = StringIO() | ||
conanfile.run('qbs-config --settings-dir "%s" --list' % ( | ||
_settings_dir(conanfile)), output=s) | ||
config = {} | ||
s.seek(0) | ||
for line in s: | ||
colon = line.index(':') | ||
if 0 < colon and not line.startswith('#'): | ||
full_key = line[:colon] | ||
if full_key.startswith(_profiles_prefix_in_config): | ||
key = full_key[len(_profiles_prefix_in_config)+1:] | ||
value = line[colon+1:].strip() | ||
if value.startswith('"') and value.endswith('"'): | ||
temp_value = value[1:-1] | ||
if (temp_value.isnumeric() or | ||
temp_value in ['true', 'false', 'undefined']): | ||
value = temp_value | ||
config[key] = value | ||
return config | ||
|
||
|
||
class LinkerFlagsParser(object): | ||
def __init__(self, ld_flags): | ||
self.driver_linker_flags = [] | ||
self.linker_flags = [] | ||
|
||
for item in ld_flags: | ||
if item.startswith('-Wl'): | ||
self.linker_flags.extend(item.split(',')[1:]) | ||
else: | ||
self.driver_linker_flags.append(item) | ||
|
||
|
||
def _flags_from_env(): | ||
flags_from_env = {} | ||
if tools.get_env('ASFLAGS'): | ||
flags_from_env['cpp.assemblerFlags'] = '%s' % ( | ||
_env_var_to_list(tools.get_env('ASFLAGS'))) | ||
if tools.get_env('CFLAGS'): | ||
flags_from_env['cpp.cFlags'] = '%s' % ( | ||
_env_var_to_list(tools.get_env('CFLAGS'))) | ||
if tools.get_env('CPPFLAGS'): | ||
flags_from_env['cpp.cppFlags'] = '%s' % ( | ||
_env_var_to_list(tools.get_env('CPPFLAGS'))) | ||
if tools.get_env('CXXFLAGS'): | ||
flags_from_env['cpp.cxxFlags'] = '%s' % ( | ||
_env_var_to_list(tools.get_env('CXXFLAGS'))) | ||
if tools.get_env('LDFLAGS'): | ||
parser = LinkerFlagsParser(_env_var_to_list(tools.get_env('LDFLAGS'))) | ||
flags_from_env['cpp.linkerFlags'] = str(parser.linker_flags) | ||
flags_from_env['cpp.driverLinkerFlags'] = str( | ||
parser.driver_linker_flags) | ||
return flags_from_env | ||
|
||
|
||
class QbsToolchain(object): | ||
filename = 'conan_toolchain.qbs' | ||
|
||
_template_toolchain = textwrap.dedent('''\ | ||
import qbs | ||
|
||
Project { | ||
Profile { | ||
name: "conan_toolchain_profile" | ||
|
||
/* detected via qbs-setup-toolchains */ | ||
{%- for key, value in _profile_values_from_setup.items() %} | ||
{{ key }}: {{ value }} | ||
{%- endfor %} | ||
|
||
/* deduced from environment */ | ||
{%- for key, value in _profile_values_from_env.items() %} | ||
{{ key }}: {{ value }} | ||
{%- endfor %} | ||
{%- if sysroot %} | ||
qbs.sysroot: "{{ sysroot }}" | ||
{%- endif %} | ||
|
||
/* conan settings */ | ||
{%- if build_variant %} | ||
qbs.buildVariant: "{{ build_variant }}" | ||
{%- endif %} | ||
{%- if architecture %} | ||
qbs.architecture: "{{ architecture }}" | ||
{%- endif %} | ||
{%- if optimization %} | ||
qbs.optimization: "{{ optimization }}" | ||
{%- endif %} | ||
{%- if cxx_language_version %} | ||
cpp.cxxLanguageVersion: "{{ cxx_language_version }}" | ||
{%- endif %} | ||
|
||
/* package options */ | ||
{%- if position_independent_code %} | ||
cpp.positionIndependentCode: {{ position_independent_code }} | ||
{%- endif %} | ||
} | ||
} | ||
''') | ||
|
||
def __init__(self, conanfile): | ||
_check_for_compiler(conanfile) | ||
self._conanfile = conanfile | ||
_setup_toolchains(conanfile) | ||
self._profile_values_from_setup = ( | ||
_read_qbs_toolchain_from_config(conanfile)) | ||
self._profile_values_from_env = _flags_from_env() | ||
tools.rmdir(_settings_dir(conanfile)) | ||
|
||
self._architecture = _architecture.get( | ||
conanfile.settings.get_safe('arch')) | ||
self._build_variant = _build_variant.get( | ||
conanfile.settings.get_safe('build_type')) | ||
self._optimization = _optimization.get( | ||
conanfile.settings.get_safe('build_type')) | ||
self._cxx_language_version = _cxx_language_version.get( | ||
str(conanfile.settings.get_safe('compiler.cppstd'))) | ||
self._sysroot = tools.get_env('SYSROOT') | ||
self._position_independent_code = _bool( | ||
conanfile.options.get_safe('fPIC')) | ||
|
||
def generate(self): | ||
save(self.filename, self.content) | ||
|
||
@property | ||
def content(self): | ||
context = { | ||
'_profile_values_from_setup': self._profile_values_from_setup, | ||
'_profile_values_from_env': self._profile_values_from_env, | ||
'build_variant': self._build_variant, | ||
'architecture': self._architecture, | ||
'optimization': self._optimization, | ||
'sysroot': self._sysroot, | ||
'position_independent_code': self._position_independent_code, | ||
'cxx_language_version': self._cxx_language_version | ||
} | ||
t = Template(self._template_toolchain) | ||
content = t.render(**context) | ||
return content |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just FYI: shorter syntax:
(no need to change)