Windows & castxml, compiler not found

[ethanhs]

I just started with the following very basic example:

from pygccxml import utils, parser, declarations

generator_path, generator_name = utils.find_xml_generator()
path= parser.config.create_compiler_path(generator_name,generator_path)

xml_generator_config = parser.xml_generator_configuration_t(
    xml_generator_path=generator_path,
    xml_generator=generator_name)

filename = "C:\\Qt\\Qt5.6.0\\5.6\\msvc2015\\include\\QtCore\\qobject.h"


decls = parser.parse([filename], xml_generator_config)

And it fails saying that compiler_path is not defined, when it checks if I am using mingw or not (I'm not).

# Platform specific options
        if platform.system() == 'Windows':

            if "mingw" in self.__config.compiler_path.lower(): # HERE self.__config.compiler_path is None, so .lower() doesn't work
                # Look at the compiler path. This is a bad way
                # to find out if we are using mingw; but it
                # should probably work in most of the cases
                cmd.append('--castxml-cc-gnu ' + self.__config.compiler_path)
            else:
                # We are using msvc
                cmd.append('--castxml-cc-msvc cl')
                if 'msvc9' == self.__config.compiler:
                    cmd.append('-D"_HAS_TR1=0"')
        else:
                #more code

In the docs, it says that creating a compiler path is only needed on Mac and Linux. Are the docs in error, or is this an issue with the code finding my Visual Studio compiler?

I am on Windows 10, Python 3.5, VS 14

[iMichka]

Hi. Thank you for reporting this.

There are multiple reasons why all this is not so clear. Pygccxml was written in a time where there the compilers where mostly only gcc and msvc. Since I added the support of CastXML, the documentation has gone out of sync in some places.

I pushed a patch on the develop branch, which will be in the next 1.8.0. You can see the changes here: 4a5c391.

The guessing of the compiler is now improved. First it will look for msvc, and then for mingw. That should work now for you. I would be interested to know if my change help finding msvc automatically (I did not test it, but could give it a try this weekend). I am not often under windows, so this part is less taken care of. Also there are no regular unit tests running under windows, so errors are often unnoticed until I make a test before a release.

Feel free to open a pull request if there are some wordings errors or if you want to extend the documentation at some places.

[praetorian20]

I don't think a typical MSVC installation modifies your PATH, so even with your change where cl may not be able to find the compiler. The right way to get this to work on Windows is probably to open the Visual Studio xxxx Command Prompt (or VSxxxx Command Prompt depending on the version of VS you're using) and use pygccxml from within that command window. If you do that, the correct cl executable is on PATH, and the original code should've worked too (I've never run pygccxml on Windows, so this is all speculation).

Anyway, I don't think your change hurts anything, but even with the change, you'll still need to open the appropriate VSxxxx Command Prompt (or modify PATH) to get things to work.

Also, since clang on Windows is an officially supported thing now, it might not be a bad idea to check for clang++ on all platforms (assuming CastXML will work with clang on Windows).

[ethanhs]

@praetorian20 Ideally, I too would like to see castxml and pygccxml use clang on all platforms. However:

The names a reference compiler with which the given command is compatible. It must be one of:

• gnu: GNU Compiler Collection C++ (g++)
• gnu-c: GNU Compiler Collection C (gcc)
• msvc: Microsoft Visual C++ (cl)
• msvc-c: Microsoft Visual C (cl)

Source

So clang++ doesn't seem to be supported officially on any platform.

Also, thank you @iMichka so much for your quick response and fix. There is an issue with a type error (the RuntimeError here is failing saying that Errors must be based on BaseException, so I'll try to fix what isn't working and I'll make a PR when I have a fix.

Edit: also seems to be an issue when I run from the VS command prompt (so it can find cl) as Windows has issues handling paths with spaces. I will look into this too.

EDIT2: Fixed the issue with the exception. Python wasn't dealing with the multiline very well.

[praetorian20]

@ethanhs I think the id field is gnu for both g++ and clang++.

As for the problem with the MSVC command, there are a few things you could try:

• On this line, I think there should be a space at the end of the string '--castxml-cc-msvc', so it should be '--castxml-cc-msvc '.
• If a path with spaces still doesn't work, try putting the compiler path within quotes - cmd.append('--castxml-cc-msvc ' + '"' + self.__config.compiler_path '"')
• If that doesn't work, then try putting the compiler path within a pair of quoted parentheses - cmd.append('--castxml-cc-msvc ' + '"(" ' + self.__config.compiler_path + ' ")"')
• You can also try a combination of the previous two, quoted parentheses and quoting the path within.
• Finally, if none of these work, just try cmd.append('--castxml-cc-msvc cl') (the original version), but invoke pygccxml from within a VS Command Prompt.

[ethanhs]

@praetorian20 On the clang id, I have no idea. If you want to figure that out, please go ahead. :)

On this line, I think there should be a space at the end of the string '--castxml-cc-msvc', so it should be '--castxml-cc-msvc '.

Good catch! I didn't get to that yet. I will add that in my PR.

On your last point, it doesn't work outside of the VS command prompt as you suspected, as cl is not in the PATH. So really I think just having people run it from a VS command prompt may be the best idea. Perhaps require people to specify the msvc compiler they wish to use on Windows. The paths to cl are standard essentially.

C:\Program Files (x86)\Microsoft Visual Studio %version%.0\VC\bin\cl.exe, where version is a number 9-14 (atm).

[ethanhs]

@iMichka do you want me to add something before searching for cl by calling it by searching in the path I mentioned? Essentially if __config.compiler is defined as eg msvc10, it would search for cl in C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\cl.exe? People can install in another area, but that will cover most VS installs. And we can get the VS version to use based on the Python version.

[iMichka]

I fixed the missing space already.

I do not want the guessing function to get too complex. Ideally people should define the path themselves, because they know where their compiler is located (hopefully :)).

I think looking in the PATH is the most obvious thing to do: it's the first place I would expect to find a compiler. And this is why the which/where calls are the less worse strategy.

I do not think hard-coding the paths to msvc with version numbers is a good idea. The compiler could be installed elsewhere, and I will need to make a new release each time to add new compiler versions. Too much work. Here again; setting the path manually is probably the best thing to do.

About version numbers: looking at compiler versions is difficult. It may work for windows, but other compilers have been known to be lying about their version numbers. Example: Apple Clang is not the same as the same clang installed from source, because Apple patches their clang. Gcc has also been known to be lying in some preprocessor macros, so this is also not a way to find out which gcc you where using. You basically never know what you end up with, so I would like to not have to rely on compiler version numbers.

Also: if you install gcc with homebrew on mac, the compiler call will be g++-6. This is another example I do not want to support: here people should set their path manually.

There is now an exception if the compiler is not found. So if on Windows msvc is not in the PATH, people will know what to do.

I would be really happy if you could make a pull request with fixes/improvements.

+1 on looking for clang on windows.