Can't use cmake generator "NMake Makefiles" with conan

[db4]

If I set environment variable CONAN_CMAKE_GENERATOR to"NMake Makefiles", conan correctly passes -G "NMake Makefiles" to cmake during configure but build fails with

NMAKE : fatal error U1065: invalid option 'j'                                                     
Stop.                                                                                             
ERROR: PROJECT: Error in build() method, line 30                                                  
        cmake.build()                                                                             
        ConanException: Error 2 while executing cmake --build C:\Work\conan-test\build -- -j4

the problem is here:
https://github.com/conan-io/conan/blob/develop/conans/client/build/cmake.py

            if "Makefiles" in self.generator:
                if "--" not in args:
                    args.append("--")
                args.append("-j%i" % cpu_count())

You always add -j flag for Makefiles generators but NMake does not understand that.

[memsharded]

I am going to submit a fix for this for 1.0.4, but if you want to try now to patch your cmake.py with:

        if "Makefiles" in self.generator and "NMake" not in self.generator:

your feedback will be welcomed.

[db4]

Yes, this way it works for me, thanks. BTW, there is another NMake-related problem: to use "NMake Makefiles" generator one have to set vcvars environment before calling cmake (otherwise cmake won't find cl compiler). It's not set by default when I run conan build ..., is there any option to work around that?

[memsharded]

Does it work when you do the conan create flow?

[db4]

Well, I'm experimenting with Use conanfile.py for consumers and trying to configure & build my own cmake project that uses NMake Makefiles generator. So I run conan install and then conan build. If VS environment is not set before, conan build fails to detect cl compiler.

[memsharded]

Just for curiosity and willing to learn, what is the reason to generate NMake files instead of VS solution if you want to actually use VS compiler? What are the advantages of using NMake, I haven't used it yet.

Maybe you can try to use the vcvars helper: http://docs.conan.io/en/latest/reference/tools.html#tools-vcvars in your build() method.

[db4]

Just for curiosity and willing to learn, what is the reason to generate NMake files instead of VS solution

I can't use if(CMAKE_BUILD_TYPE STREQUAL Debug) with cmake Visual Studio generator. The only alternative is generator expressions, but they are introduced relatively recently and have many limitations.

Maybe you can try to use the vcvars helper

Thanks. But it also seems not working correctly - it replaces PATH with VS paths, not appends them. Looks like the problem is that original Windows variable is Path while vcvarsall.bat sets PATH (note the case), but I need to investigate more.

[memsharded]

Yes, Visual Studio generator is multi-config, and CMAKE_BUILD_TYPE is not defined like in single-config environments. But later switch in IDE or with cmake --build . --config Release works well. I guess you have some limitations in your project, but what about Ninja? Have you tried it? Feel free to reach us privately (email, slack) if you want to discuss private details.

[db4]

Regarding PATH update. Just sitting in python debugger. Case was not a problem - python itself converts Path to uppercase. But I believe I've found the bug. In
https://github.com/conan-io/conan/blob/develop/conans/client/tools/env.py

def environment_append(env_vars):
    """
    :param env_vars: List of simple environment vars. {name: value, name2: value2} => e.j: MYVAR=1
                     The values can also be lists of appendable environment vars. {name: [value, value2]}
                      => e.j. PATH=/path/1:/path/2
    :return: None
    """
    old_env = dict(os.environ)
    for name, value in env_vars.items():
        if isinstance(value, list):
            env_vars[name] = os.pathsep.join(value)
            if name in old_env:
                env_vars[name] += os.pathsep + old_env[name]
    os.environ.update(env_vars)

if isinstance(value, list) is False because env_vars['PATH'] is a string, so env_vars[name] += os.pathsep + old_env[name] never happens.

environment_append() is called from

def vcvars(*args, **kwargs):
    new_env = vcvars_dict(*args, **kwargs)
    with environment_append(new_env):
        yield

[lasote]

env_vars['PATH'] can be a list if you call:

environment_append({"PATH": ["one", "two"]})

if you don't pass a list it doesn't have to do anything special, just update the environ with the string.

[memsharded]

You can try with:

def build(self):
        with tools.vcvars(self.settings, filter_known_paths=False):
            cmake = CMake(self, generator="NMake Makefiles")
            cmake.configure(source_folder="src")
            cmake.build()

This is possibly a bug (the default value of filter_known_paths to True), we are considering to change.

I am a bit surprised that CMake is not able to find VS cl.exe while using NMake Makefiles generator, but with the above you are able to inject the VS path to cl.exe before executing CMake.

In any case, I still feel it like a small workaround, as we are talking mainly about user consumption, I feel that going with Visual Studio generator or maybe Ninja could be more beneficial, I guess you are coding something in your CMakeLists that is exclusively single-config at CMake configure time (not at build time), but in my experience that can usually be improved in some way.

[db4]

@lasote,

if you don't pass a list it doesn't have to do anything special, just update the environ with the string.

Yes, so the problem is probably in vcvars_dict(), that always assigns a string value to PATH key , no matter if filter_known_paths=True is set or not: I think it should split values into a list for PATH and maybe for some other keys like INCLUDE and LIB

[db4]

@memsharded,

I am a bit surprised that CMake is not able to find VS cl.exe while using NMake Makefiles generator

That's quite obvious. For Visual Studio generator cmake knows the target toolchain and architecture from its name (like Visual Studio 15 2017 Win64). That's not the case for NMake Makefiles. So cmake should rely on cl that it finds in the path. I believe it also applies to Ninja.

[memsharded]

Yes, that makes sense, thanks for the clarification :)

Did you try the above? Did it work? Please tell me.

[memsharded]

@db4 with the change in #2392, it won't be necessary to explicitly set filter_known_paths=False. Please, if you could try the above and confirm it works, it would be a great feedback before we release 1.0.4 (intended for tomorrow). Thanks!

[db4]

Yes, it works for me. Thanks a lot!

[memsharded]

Released in 1.0.4

[aissat]

C:\build>conan -v
Conan version 1.4.3

C:\build>gcc --version
gcc (MinGW.org GCC-6.3.0-1) 6.3.0
Copyright (C) 2016 Free Software Foundation, Inc.

C:\build>cmake --version
cmake version 3.11.3

C:\build>conan install .. --build=missing
...........
Makefile:128: recipe for target 'all' failed
mingw32-make.exe: *** [all] Error 2
zlib/1.2.11@conan/stable:
zlib/1.2.11@conan/stable: ERROR: Package '7bc8c2c85db7a618e5320dc997f27fc33e1df074' build failed
zlib/1.2.11@conan/stable: WARN: Build folder C:\Users\ING\.conan\data\zlib\1.2.11\conan\stable\build\7bc8c2c85db7a618e5320dc997f27fc33e1df074
ERROR: zlib/1.2.11@conan/stable: Error in build() method, line 56
        cmake.build(build_dir=".")
        ConanException: Error 2 while executing cmake --build . -- -j4

[memsharded]

Hi @aissat !

We'd need more information, for example what is the profile you are using to build that. Because by default in Windows, latest Visual Studio is used. Also, any environment variable that you might be using, like CONAN_CMAKE_GENERATOR, as env-var or defined in conan.conf. Thanks!

[aissat]

hi @memsharded
conan.conf

[log]
run_to_output = True        # environment CONAN_LOG_RUN_TO_OUTPUT
run_to_file = False         # environment CONAN_LOG_RUN_TO_FILE
level = 50                  # environment CONAN_LOGGING_LEVEL
print_run_commands = False  # environment CONAN_PRINT_RUN_COMMANDS

[general]
default_profile = default
compression_level = 9                 # environment CONAN_COMPRESSION_LEVEL
sysrequires_sudo = True               # environment CONAN_SYSREQUIRES_SUDO
request_timeout = 60                  # environment CONAN_REQUEST_TIMEOUT (seconds)
config_install = zlib/1.2.11@conan/stable

[storage]
path = ~/.conan/data

[proxies]

my profile

[settings]
os=Windows
os_build=Windows
arch=x86_64
arch_build=x86_64
compiler=gcc
compiler.version=6.3
compiler.libcxx=libstdc++11
build_type=Release
[env]
CFLAGS=-fpic
CXXFLAGS=-fpic -std=c++11
[options]
[build_requires]

[memsharded]

Could you please check what is the line in your conan.conf?:

config_install = zlib/1.2.11@conan/stable

Seems like a wrong conan config install command, could be?

Note also that the flags:

[env]
CFLAGS=-fpic
CXXFLAGS=-fpic -std=c++11

won't be used in Windows. In Windows, it is building with CMake (the CMake build helper will automatically handle fPIC and std flags). Other arbitrary flags that you could provide here won't necessarily be passed to CMake, as it doesn't honor such environment variables. There are some ongoing ideas trying to provide such functionality that CMake doesn't implement out of the box.

[memsharded]

Hi again @aissat

I am failing to see how your issue is related to NMake, which is the title of this issue.

I have been able to build zlib with my profile with:

conan install zlib/1.2.11@conan/stable --build=zlib -pr=myprofile
zlib/1.2.11@conan/stable: Not found in local cache, looking in remotes...
zlib/1.2.11@conan/stable: Trying with 'conan-center'...
Downloading conanmanifest.txt
[==================================================] 121B/121B
Downloading conanfile.py
[==================================================] 5.9KB/5.9KB
zlib/1.2.11@conan/stable: Installing package
Requirements
    zlib/1.2.11@conan/stable from 'conan-center'
Packages
    zlib/1.2.11@conan/stable:7bc8c2c85db7a618e5320dc997f27fc33e1df074

zlib/1.2.11@conan/stable: WARN: Forced build from source
Downloading conan_sources.tgz
[==================================================] 281B/281B
zlib/1.2.11@conan/stable: Building your package in C:\...i\.conan\data\zlib\1.2.11\conan\stable\build\7bc8c2c85db7a618e5320dc997f27fc33e1df074
zlib/1.2.11@conan/stable: Configuring sources in C:\...\.conan\data\zlib\1.2.11\conan\stable\source
[==================================================] 607.7KB/607.7KB
zlib/1.2.11@conan/stable: Copying sources to build folder
zlib/1.2.11@conan/stable: Generator cmake created conanbuildinfo.cmake
zlib/1.2.11@conan/stable: Calling build()
-- The C compiler identification is GNU 4.9.3
-- The CXX compiler identification is GNU 4.9.3
...

it doesn't use NMake at all.

Please open a new issue, make sure that you provide:

• The exact commands you are running
• The files necessary for those commands to work
• The output of running the commands.

Thanks!

[aissat]

hi @memsharded
thnx for reply
can get your config profile

[aissat]

when i removed
config_install = zlib/1.2.11@conan/stable
get

$ conan config install
ERROR: Called config install without arguments and 'general.config_install' not defined in conan.conf

[memsharded]

What configuration are you trying to install? conan config install is mostly for teams, probably you don't need it if you are just starting with conan. It should point to a git repo or a zip file containing the configuration that your team or company is using, and that git repo or zip file is provided by yourself.

Note the difference between conan install, which is the command I used:

$ conan install zlib/1.2.11@conan/stable --build=zlib -pr=myprofile

and conan config install that is for installing your team configuration

[aissat]

I think the problem was solved by adding the following

[build_requires]
	*:mingw_installer/1.0@conan/stable 

But another problem appeared that every time he downloaded