-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
C++20 modules support in MSVC, take 2 #333
Comments
First, in the notes linked above, I added (last year) a note with a link to this document: https://lists.isocpp.org/sg15/att-1346/C___20_Modules_Build_in_Visual_Studio.pdf which summarizes how MSBuild builds modules and find dependent modules. It's short and gives you all the clues for the basic interface I believe. For the compilation flags, here is an article that summarize in a table all the flags we need (I think): Taking this info and the ones I gathered in the past, and trying to summarize quickly, and removing information about header-units: To build module files (a):
To consume module files (b):
That's the gist of it (ignoring also how to find the dependencies, this is explained in the first link of this post but Note that I did not find much documentation for Also note that these flags should only work with
I looked at all the environment variables set in "Developer" presets from Terminal when installing recent VS2022 and preview. Basically this is vs command prompt tools just in the Terminal app.
I'll investigate. |
About I asked on MSVC's STL discord if there is anything special to know about building these modules, for now there is no direct answer but it will probably come soon. I highly suspect that nothing special is needed. Meanwhile, I proceeded as follow to have a basic check of how MSBuild does it:
Source
Keep in mind that these flags are augmented with other flags which are "default".
This is very verbose but the lower output level will not give the command lines. First appearance of
then
Which looks to me like it's normal module build of these std files. |
I got confirmation from STL (the maintainer) that there is nothing special to do to build |
Building modules As for |
No environment variables. I have been discouraging that for modules. If you have an MSVC toolset and you can locate its |
Filling in some blanks in the MSVC documentation on implementation partitions support (referred to as "internal partitions"):
Tested with 17.6. @GabrielDosReis None of this is document, AFAICS. @mathstuf FYI. |
I am trying to decide if we should use
@GabrielDosReis Are there any plans to enable the new preprocessor by default at some point? |
Checked with VS 2022 17.8.2 and 17.9.0-preview 1: no Looks like for now it's more targeted at C users. |
Interesting; I've not seen this in CMake. Maybe this case isn't being tested? I'll check today.
This sounds odd as well; I'll also test this. |
Using
|
Manually mucking with response files, this seems true, but I feel like I can make this matter… |
Yes, if you use a type from an internal partition in an exported API, the internal partition needs resolved. So, in general, you do need the internal partition references. See the changes in this MR: https://gitlab.kitware.com/cmake/cmake/-/merge_requests/9039 |
Yes, you are right, I just retested and specifying the full partition name also works.
Interesting, thanks! Looks like the same requirement in Clang. |
@Klaim Thanks for re-checking!
And C++ header unit users. We could try to build our 400+ packages with this flag and see if there are any regressions. |
Capturing more information as I discover it: MSVC replaces the |
For new IFC-related, note |
There is no environment variable. The |
I would recommend using |
No, MSVC does not implement or guarantee any "transitive |
Thanks. I will work with the doc team to fix what is missing. |
As usual, Windows SDKs are the main blockers here. Please could you file a bug/request on DevCom and send me the link - that makes it easier for me to work with internally. |
MSVC supports named modules with the old non-conforming preprocessor. However, it requires |
An unfortunate editing oversight when the C conformance work was done and the doc was update. |
Yes, you do. There is no transitive |
The standards text say that |
One thing I would encourage you to provide is ability for a user to specify (at the build definition level) their own IFC mapping for a given header unit reference or a module name reference. For instance, a user may need to specify that the IFC for the header unit |
@GabrielDosReis Thanks for the feedback, please see me replies below:
I would really like to provide the mapping in a file but the way In contrast, Let me know if you would be amenable to supporting something similar to the GCC's format and I would be happy to file an issue.
Here you go: https://developercommunity.visualstudio.com/t/Please-clarify-ifwhen-Zc:preprocessor/10537317
We may do this later if there is strong demand but I am really not a fan of this approach. It gives the false (IMO) sense that the user can manually arrange for this to work reliably. For example, the user can already specify the I am hoping that the need for header units will greatly diminish with support for |
I agree that users specifying this is probably a bit far. It's something far more under the direction of the header unit provider than the consumer. As such, I expect CMake's |
Dual-personality header unit/named module BMIs, how hard can that be? ;-) But, yes, interesting idea to automatically translate standard library header inclusion to |
Overall? Sounds easier than implementing header units… Moves a lot of the work from build systems to compilers though. |
Probably, but it's a solution most likely to be limited to |
Headers doing this were already at risk of: #include <vector>
#include "header_using_std_namespace.h" // assuming it uses `vector` somewhere Source TUs might have been more isolated before, but were also at risk of a transitive include coming in over time. |
No. The expectation isn't that the IFC reference map is manually authored. Rather, what experience has consistently shown is that deploying header units at scale is an iterative process and ability to quickly look at the mapping and do a local edits of a file that might have been automatically generated is most useful. Summary: you can generate the IFC reference map by a tool; there is no expectation that is it manually written by the user. |
Response files work too! I was pointing out the IFC reference map as a more scalable way (from the CL processing perspective), but there is no obligation to take advantage of it if you find it not appropriate for your purposes. |
At this point, that is not something the team is prioritizing. |
Thanks! |
I am not sure I understand. The idea isn't that the Anyway, the mechanism is there, if you ever finds it useful for |
|
Exactly! And it is beyond just standard library modules. |
Actually, it is not really about replacing an unnamed module by a named module. It is about subsuming a large collection of small BMIs with a single large BMI. |
FYI, this has apparently now been fixed and is pending release (above issue was marked as duplicate of the following): |
Note that bugs related with this fix were found in 17.10-previews too and has been fixed but not released yet. |
Ok, I believe named modules support in MSVC (including |
We've been blocked by MSVC issue 10029154 for a year and half now and I don't believe it makes sense to continue waiting. So we need an alternative plan. Supporting header units without a module mapper or other form of compiler support (like the subject of the above issue) will essentially require us to implement a C++ preprocessor that matches the exact semantics of MSVC, which is definitely not something we would like to take on. As a result, it seems the only alternative available to us is to add support for named modules and leave header units for later (or never; I am getting the feeling that the C++ community is starting to give up on header units and perhaps this is for the best).
So the proposed plan is as follows:
Update
build2
to use the new modules-related options in MSVC. We will continue using our existing module dependency extraction machinery, which works fine for named modules (and is used for supporting modules in Clang and in older versions of MSVC).We will want to support building
std
andstd.compat
modules. It feels like the best approach here is to build them on the side similar to how we already build module interfaces of installed libraries. Locating the corresponding module interface files (std.ixx
andstd.compat.ixx
) should be doable: they appear to be installed into themodules\
subdirectory of the MSVC toolchain directory (i.e., it's next to theinclude\
andlib\
subdirectories). I wonder if there is an environment variable (similar toINCLUDE
) that is set by the Visual Studio command prompts?The first practical step we would need to take in executing this plan is to get a sense of the new command line options that need to be passed when (a) building a module interface and (b) building a translation unit that imports this modules interface. We would also need to figure out if anything special needs to be done to compile
std.ixx
. Finally, there are also partitions (interface and implementation) and it would be good to understand how they fit in.@Klaim has already done some work on that: https://github.com/Klaim/cxx20-modules-examples/blob/visualstudio/visualstudio-projects/build2-modules-msvc-notes.md
@GabrielDosReis FYI
The text was updated successfully, but these errors were encountered: