[feature] Support visual studio backend for meson

[Makogan]

Reading the docs https://docs.conan.io/en/latest/reference/build_helpers/meson.html

It would seem conan assumes that meson will always use ninja as the build backend. However meson allows you to build using msvc through the flan --backend.

This means that the following meson command:

meson setup build_vs --backend=vs2019 -Dpkg_config_path=build_vs

Cannot be assembled from the conan.py script.

If you try something like this:

        args = [f'-Dpkg_config_path={os.path.abspath("../" + str(self.folders.generators))}']
        if self.settings.compiler == "gcc":
            self.output.error("gcc")
            args += ["-Db_sanitize=undefined", "-Dcpp_std=c++20"]
        if self.settings.compiler == "Visual Studio":
            self.output.error("VS")
            args += ["--backend=vs2019", "-Dcpp_std=c++20"]

        meson = Meson(self)
        meson.configure(build_folder='.', args=args)
        meson.build(args=['-j2'])

You will get this error:

ERROR: conanfile.py: Error in build() method, line 123
        meson.build(args=['-j2'])
        ConanException: Error 1 while executing ninja -C "C:\Users\Makogan\Documents\neverengine\build_visual_studio_debug\." -j2 

• [ x ] I've read the CONTRIBUTING guide.

[franramirez688]

Hi @Makogan

To change the backend you only have to pass it through the constructor:

    def build(self):
        meson = Meson(self, backend="vs2019")
        meson.configure()
        meson.build()

If you don't pass anything, by default, it's going to use the ninja backend.

Anyway, I invite you to have a look at the new MesonToolchain/Meson generator/build-helper documentation.
These new ones are going to be used for the next Conan 2.0 major release as well, so it would be great to get more feedback about them. For instance:

from conan.tools.meson import Meson, MesonToolchain

class App(ConanFile):
    # ....

    def generate(self):
        tc = MesonToolchain(self, backend="vs2019")
        tc.generate()

    def build(self):
        meson = Meson(self)
        meson.configure()
        meson.build()

[memsharded]

Actually, it is not really necessary to modify the default in the recipe. You can let the Ninja default, and define tools.meson.mesontoolchain:backend=xxx in your [conf] profile or command line to change it.

[Makogan]

Sorry if I try this:

 def build(self):
        meson = Meson(self, backend="vs2019")
        meson.configure()
        meson.build()

I get
ConanException: Build only supported with 'ninja' backend

   Is there any further setup that needs to be done?

I also tried doing things with the generate syntax and i get this error:

ERROR: conanfile.py: Error in build() method, line 130
        meson.build()
        ConanException: Error 1 while executing ninja -C "C:\Users\Makogan\Documents\neverengine\build_visual_studio_debug\."
        

Looking here:
https://github.com/conan-io/conan/blob/develop/conans/client/build/meson.py

There does not seem to be a call path for the vs backend.

[memsharded]

It seems you are using the wrong Meson. Make sure to from conan.tools.meson import Meson, MesonToolchain

[Makogan]

This is the start of my recipe:

from distutils.command.build_clib import build_clib
import os
from conans import ConanFile, tools, Meson
from conan.tools.meson import MesonToolchain
from conans.tools import OSInfo
from conans.tools import os_info, SystemPackageTool
from conan.tools.gnu import PkgConfigDeps


class ConanFileNeverEngine(ConanFile):
    generators = "PkgConfigDeps"

If I modify the code to match the suggestion i.e. change my imports

from distutils.command.build_clib import build_clib
import os
from conans import ConanFile, tools
from conan.tools.meson import Meson, MesonToolchain
from conans.tools import OSInfo
from conans.tools import os_info, SystemPackageTool
from conan.tools.gnu import PkgConfigDeps

I start getting a series of errors about unrecognized keywords, meaning I can no longer do any of the following:

        meson = Meson(self, backend=backend)
        meson.configure(build_folder='.', args=args)
        meson.build()

This class seems to expect all work to be done in the generate step. I tried setting up everything there:

def generate(self):
        pc = PkgConfigDeps(self)
        pc.generate()

        args = [f'-Dpkg_config_path={os.path.abspath("../" + str(self.folders.generators))}']

        if self.settings.compiler == "gcc":
            self.output.error("gcc")
            args += ["-Db_sanitize=undefined", "-Dcpp_std=c++20"]
            backend = "ninja"
        if self.settings.compiler == "Visual Studio":
            self.output.error("VS")
            args += ["-Dcpp_std=c++20"]
            backend="vs2019"
        tc = MesonToolchain(self, backend=backend, args=args, build_folder='.')
        tc.generate()

I get:

ERROR: Cannot find specified native file: conan_meson_native.ini
ERROR: conanfile.py: Error in build() method, line 127
        meson.configure()
        ConanException: Error 1 while executing meson setup --native-file "conan_meson_native.ini" "C:\Users\Makogan\Documents\neverengine\build_visual_studio_debug\build" "C:\Users\Makogan\Documents\neverengine\." -Dprefix="C:\Users\Makogan\Documents\neverengine"

I read the docs:
https://docs.conan.io/en/latest/reference/conanfile/tools/meson.html

I am not 100% what I am doing wrong (thank you for the help and sorry to be lost)

[memsharded]

This is wrong:

from conans import ConanFile, tools, Meson

You need to:

from conans import ConanFile, tools
from conan.tools.meson import MesonToolchain, Meson

Then, yes, the new integration Meson doesn't defined the build_folder as argument, you need to remove it and rely on the layout() method.

[Makogan]

I did, I explained on the comment I changed from one to the other:

This is at the header of the recipe:

import os
from conans import ConanFile, tools
from conan.tools.meson import Meson, MesonToolchain
from conans.tools import OSInfo

[Makogan]

I am reading this issue and it seems the feature is pretty new. Can I ask what was the rationale to not simply modify the _run_ninja_backend command?

It seems trivial to modify it to accept other backends.

#9922

[memsharded]

We have a few tests that cover the new integrations, let me for example paste one of them as an example:

from conans import ConanFile, tools
        from conan.tools.meson import Meson, MesonToolchain


        class App(ConanFile):
            settings = "os", "arch", "compiler", "build_type"
            options = {"shared": [True, False], "fPIC": [True, False]}
            default_options = {"shared": False, "fPIC": True}

            def config_options(self):
                if self.settings.os == "Windows":
                    del self.options.fPIC

            def layout(self):
                self.folders.build = "build"

            def generate(self):
                tc = MesonToolchain(self, backend='vs')
                tc.generate()

            def build(self):
                meson = Meson(self)
                meson.configure()
                meson.build()
        """)

that is missing the package() and package_info() it is mostly a consumer, but could be a good start.

[Makogan]

If it is not clear, a recipe like this:

from distutils.command.build_clib import build_clib
import os
from conans import ConanFile, tools
from conan.tools.meson import Meson, MesonToolchain
from conans.tools import OSInfo
from conans.tools import os_info, SystemPackageTool
from conan.tools.gnu import PkgConfigDeps


class ConanFileNeverEngine(ConanFile):
    generators = "PkgConfigDeps"

    requires = [
    ]

    settings = {
        "os": None,
        "compiler" : None,
        "cppstd": None,
        "build_type" : None,
        "arch" : None}

    def generate(self):
        pc = PkgConfigDeps(self)
        pc.generate()

        args = [f'-Dpkg_config_path={os.path.abspath("../" + str(self.folders.generators))}']

        if self.settings.compiler == "gcc":
            self.output.error("gcc")
            args += ["-Db_sanitize=undefined", "-Dcpp_std=c++20"]
            backend = "ninja"
        if self.settings.compiler == "Visual Studio":
            self.output.error("VS")
            args += ["-Dcpp_std=c++20"]
            backend="vs2019"
        tc = MesonToolchain(self, backend=backend, args=args, build_folder='.')
        tc.generate()

    def layout(self):
        build_dir = '_'.join([str(self.settings.compiler).lower().replace(' ', '_'), str(self.settings.build_type).lower()])
        self.folders.build = f'build_{build_dir}'
        self.folders.generators = self.folders.build + '/dependencies'
        self.folders.imports = self.folders.build
        self.folders.source = "."
        self.folders.install = self.folders.build
        
    def build(self):
        meson = Meson(self)
        meson.configure()
        meson.build()

Produces this error:

**********************************************************************
** Visual Studio 2022 Developer Command Prompt v17.0.4
** Copyright (c) 2021 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'
Could not find any valid candidate for native files: conan_meson_native.ini

ERROR: Cannot find specified native file: conan_meson_native.ini
ERROR: conanfile.py: Error in build() method, line 127
       meson.configure()
       ConanException: Error 1 while executing meson setup --native-file "conan_meson_native.ini" "C:\Users\Makogan\Documents\neverengine\build_visual_studio_debug\build" "C:\Users\Makogan\Documents\neverengine\." -Dprefix="C:\Users\Makogan\Documents\neverengine"

[memsharded]

I am reading this issue and it seems the feature is pretty new. Can I ask what was the rationale to not simply modify the _run_ninja_backend command?

The previous Meson integration is legacy, it has a few issues, but cannot be modified without breaking. So all the build system integrations are going to be fully replaced with the new ones, in the from conan.tools.meson, from conan.tools.cmake space. So we are not touching the previous old code, but replacing it with a new alternative.

[memsharded]

Also, it seems you are referencing to the latest docs, but the latest Conan does not have the constructor arguments you are using, check: https://docs.conan.io/en/latest/reference/conanfile/tools/meson.html#constructor:

def __init__(self, conanfile, backend=None):

So the MesonToolchain does not have a build_folder or args argument.

[Makogan]

from conans import ConanFile, tools
        from conan.tools.meson import Meson, MesonToolchain


        class App(ConanFile):
            settings = "os", "arch", "compiler", "build_type"
            options = {"shared": [True, False], "fPIC": [True, False]}
            default_options = {"shared": False, "fPIC": True}

            def config_options(self):
                if self.settings.os == "Windows":
                    del self.options.fPIC

            def layout(self):
                self.folders.build = "build"

            def generate(self):
                tc = MesonToolchain(self, backend='vs')
                tc.generate()

            def build(self):
                meson = Meson(self)
                meson.configure()
                meson.build()
        """)

For your reference I copy pasted this onto my computer and ran the same set of commands I normally use to install and compile with meson, I get this error:

ERROR: Cannot find specified native file: conan_meson_native.ini
ERROR: conanfile.py: Error in build() method, line 23
        meson.configure()
        ConanException: Error 1 while executing meson setup --native-file "conan_meson_native.ini" "C:\Users\Makogan\Documents\neverengine\build\build" "C:\Users\Makogan\Documents\neverengine" -Dprefix="C:\Users\Makogan\Documents\neverengine"

So something seems odd.

conan version is 1.44.0

Commands to run:

$ conan install . -if build_visual_studio_debug/ -s build_type=Debug -s compiler.runtime=MDd -s compiler="Visual Studio" -s compiler.version=17 --build=missing

$ conan build . -if build_visual_studio_debug/

[memsharded]

Oh, which Meson version are your using? Maybe it could be your Meson not supporting this?

[Makogan]

0.60.3

[Makogan]

I am on a git bash terminal, is the command expecing a cmd maybe?

It happens inside a cmd as well

[Makogan]

@franramirez688

In my honest opinion this new design seems to suffer from taking on too much responsibility. The old design was better in that it was a thin wrapper that just let you pass what you needed straight through to meson. Conan's value proposition is to be a package manager, not a build system. In my opinion it should delegate as much as possible to the build system.

The legacy code seemed, to me, near perfect, except for an explicit check that was unnecessary. In other words, conan was almost perfect and the only time it wasn't was because it took more responsibility than it needed.

so it would be great to get more feedback about them. For instance:

My honest feedback, conan should just pass the user arguments straight to meson and be as transparent as possible. It is less burden on the tool, the conan developers and conan users. Users know how to use meson and it;s less frustrating and less error prone to just let them use the tool directly, that way they can setup the compilation commands the way that their program needs.

As a user I am not sure what:

ERROR: Cannot find specified native file: conan_meson_native.ini
ERROR: conanfile.py: Error in build() method, line 23
        meson.configure()
        ConanException: Error 1 while executing meson setup --native-file "conan_meson_native.ini" "C:\Users\Makogan\Documents\neverengine\build\build" "C:\Users\Makogan\Documents\neverengine" -Dprefix="C:\Users\Makogan\Documents\neverengine"

Is trying to do, that's already a problem as the way I use meson and the way conan wants to use it iffer, complicating my build process. Moreover this new tool seems to be setting up a bunch of scary files:

I see these files being generated by the new command

C:\Users\Makogan\Documents\neverengine\conan_meson_native.ini
C:\Users\Makogan\Documents\neverengine\conanbuild.bat
C:\Users\Makogan\Documents\neverengine\conanvcvars.bat
C:\Users\Makogan\Documents\neverengine\deactivate_conanbuild.bat

They dont seem to be generated by the old tool'. The fact that conan is moving in a direction where it needs to autogen bat files is spooky. They don;t do much but imho it;s symptomatic of overengineering.

[franramirez688]

Hi @Makogan

Sorry for all the failing points, I know that's a bit frustrating 😞

Let me put here all the steps to try to have something simple working fine from your side.

Let's imagine that we have this project structure in another empty folder to be sure that it's clean:

| - main.cpp
| - conanfile.py
| - meson.build

Where the main.cpp is a simple Hello world! and the meson.build looks like:

project('tutorial', 'cpp')
executable('demo', 'main.cpp')

Then, the conanfile.py could be something like:

from conans import ConanFile, tools
from conan.tools.meson import Meson, MesonToolchain


class App(ConanFile):
    settings = "os", "arch", "compiler", "build_type"
    options = {"shared": [True, False], "fPIC": [True, False]}
    default_options = {"shared": False, "fPIC": True}

    def layout(self):
        self.folders.build = "build"

    def generate(self):
        tc = MesonToolchain(self, backend="vs2019")
        tc.generate()

    def build(self):
        meson = Meson(self)
        meson.configure()
        meson.build()

Now, if you run the commands that you already put in your previous comment:

$ conan install . -if build_visual_studio_debug/ -s build_type=Debug -s compiler.runtime=MDd -s compiler="Visual Studio" -s compiler.version=17 --build=missing

$ conan build . -if build_visual_studio_debug/

Let's try this simple approach at first to see if it's working without any kind of odd problem.

[memsharded]

@franramirez688

I have wanted to do a conan new --template=meson_lib for a while, to help with this. Would it make sense to try to contribute this asap?

[Makogan]

Sorry if my reply was too brusque I did mean what I say but I was trying to give honest feedback, it might have sounded a little ranty. On any case I did make a repository as suggested with exactly 3 files:

PS C:\Users\Makogan\Documents\test> ls


    Directory: C:\Users\Makogan\Documents\test


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----         1/27/2022   8:26 PM            578 conanfile.py
-a----         1/27/2022   8:22 PM            113 main.cpp
-a----         1/27/2022   8:24 PM            333 meson.build

The conan file is almost the same as the one you suggested except the build directory is different to match the one in my commands. This is the meson:

project(
    'NeverEngine',
    'cpp',
    version: '0.0.1',
    license: 'GPL',
    default_options : [
        'cpp_std=c++20',
        'default_library=static',
        'optimization=3',
        'buildtype=debug'])

test = executable(
    'test',
    'main.cpp',
    include_directories : [],
    dependencies : [])

And the main.cpp is your typical hello world. I run the commands and I get the same error:

ERROR: Cannot find specified native file: conan_meson_native.ini
ERROR: conanfile.py: Error in build() method, line 19
      meson.configure()
      ConanException: Error 1 while executing meson setup --native-file "conan_meson_native.ini" "C:\Users\Makogan\Documents\test\build_visual_studio_debug\build" "C:\Users\Makogan\Documents\test" -Dprefix="C:\Users\Makogan\Documents\test"

I really appreciate the help and I think conan is an awesome tool and I am grateful for it. Please understand, my prior feedback is not just frustration, it is my legitimate belief that tools should not do my work for me, they should just make it easier to do my work. When tools start to take power away from me to try to make my life easier all they accomplish is to prevent me from designing solutions that solve my needs. I really do mean, from a sober perspective, the legacy design was better. I don't like that conan needs to create all of these intermediary bat files and this conan_meson_native.ini file. This is not how I use meson, and not how conan used to use meson this adds multiple layers of complexity and more complexity means more bugs, errors and edge cases.

Please, I am saying this as a legitimate concern, as a user, as someone whose life has been made better by the tool, please consider returning to the old design. Please keep conan transparent. There is no need for this additional functionality. I used to handle my dependencies through a python script I wrote myself, it wasn't fun, conan made my life better by handling the package management for me, I don't want to go back to writing my own tool. But the reality is, if conan moves in a direction where it opaques what it's doing and tries to solve my problems for me then my own technical constraints may force me to not use the tool :C. If I cannot understand how conan delivers commands to meson or if conan assembles itself in a way that is too dissimilar to how meson should be run without the wrapper then the tool would no longer be able to serve the needs of the project I work on.

Once more thank you so much for your patience and help, please consider my feedback if possible.

[Makogan]

To stress how close to working the prior design was I hacked conan just a little bit this is what the "_run_ninja_commands" looks like:

def _run_ninja_targets(self, args=None, build_dir=None, targets=None):
        # if self.backend != "ninja":
        #     raise ConanException("Build only supported with 'ninja' backend")

        args = args or []
        build_dir = build_dir or self.build_dir or self._conanfile.build_folder

        arg_list = join_arguments([
            '-C "%s"' % build_dir,
            args_to_string(args),
            args_to_string(targets)
        ])
        self._run("meson compile %s" % arg_list)

This works just fine, it's such a minor change. In order to presserve backwards compatibility all it;s needed is to add a new command and an if statment checking the backend

[Makogan]

I made a pull request, the change is so small this should not break compatibility, it handles a strict superset of cases:

#10447

[franramirez688]

Hi @Makogan

Thanks a lot for your feedback and the PR (we have to review it), they are really appreciated for sure.

After figuring it out, I just realized what was really going on. I did not get why you were receiving this output:

ERROR: Cannot find specified native file: conan_meson_native.ini
ERROR: conanfile.py: Error in build() method, line 19
      meson.configure()
      ConanException: Error 1 while executing meson setup --native-file "conan_meson_native.ini" "C:\Users\Makogan\Documents\test\build_visual_studio_debug\build" "C:\Users\Makogan\Documents\test" -Dprefix="C:\Users\Makogan\Documents\test"

It was not happening on my side because I was using the develop branch, i.e., the next Conan 1.45 release 😅
In summary, it's a current bug (that we did not know yet) but it's luckily solved in 1.45 because of a big refactor done in the MesonToolchain generator.
For the record, the current bug is this param --native-file "conan_meson_native.ini" because it should be --native-file "C:\Users\Makogan\Documents\test\conan_meson_native.ini". Anyway, it's now fixed for the next release 😉

[Makogan]

Hi @Makogan

Thanks a lot for your feedback and the PR (we have to review it), they are really appreciated for sure.

After figuring it out, I just realized what was really going on. I did not get why you were receiving this output:

ERROR: Cannot find specified native file: conan_meson_native.ini
ERROR: conanfile.py: Error in build() method, line 19
      meson.configure()
      ConanException: Error 1 while executing meson setup --native-file "conan_meson_native.ini" "C:\Users\Makogan\Documents\test\build_visual_studio_debug\build" "C:\Users\Makogan\Documents\test" -Dprefix="C:\Users\Makogan\Documents\test"

It was not happening on my side because I was using the develop branch, i.e., the next Conan 1.45 release sweat_smile In summary, it's a current bug (that we did not know yet) but it's luckily solved in 1.45 because of a big refactor done in the MesonToolchain generator. For the record, the current bug is this param --native-file "conan_meson_native.ini" because it should be --native-file "C:\Users\Makogan\Documents\test\conan_meson_native.ini". Anyway, it's now fixed for the next release wink

That's nice to hear but I am still a little worried that the file needs to be made in the first place. As mentioned, I really think the "legacy" implementation was the right design.

[memsharded]

That's nice to hear but I am still a little worried that the file needs to be made in the first place. As mentioned, I really think the "legacy" implementation was the right design.

No, it is not. We have tons of evidence that the previous design was harming many developers. The main reason is that it couples the build to Conan completely, unless you call conan build, the binaries that you built locally are different to the ones that are built by Conan in the cache. This is unacceptable. The only possibility to allow developers a local flow calling meson ... that achieves that same build is conan generating the necessary files that can be passed to the meson invocation, so that guarantees everything is there, nothing depends on Conan magically injecting environment vars on the fly.

This approach is exactly the same that has been already deployed to other build systems, like CMake, and so far the feedback is very solid it is a step in the right direction.

We apologize for the inconvenience and the frustration of the migration attempt, it is true, that MesonToolchain is behind other integrations, but we will get there soon too, and we are positive that it will be smoother and a better integration than the current one.

[franramirez688]

Thanks for contributing @Makogan 😄

It will be released in the next Conan 1.45.