Cygwin build is broken again #3799
Comments
patacongo
added
backport/10.0
backport/10.1
|
I took the time to do the git bisect to find the comment that caused this build failure. On each step of the bisection, I did the following: The final step of the bisection reported: |
|
I will take a look and try to infer the failure, as I'm really not familiar with CYGWIN. |
|
Now that we have the minimal builds for macOS I can look at bringing back the Cygwin CI, previously it was super slow and I never got to the bottom of that. |
|
That would be super useful since it would also allow me to validate the cmake changes |
|
I'm taking a break for the US holiday weekend l, but I'll push this up the priority after along with the nxflat changes from last week that seem to be good and should be added to CI. |
|
@patacongo did you continue the bisection to the end? If I look at the diff of that commit I only see changes to the depend stage, which should not affect the build itself. Or is that during depend target? |
No, the delimiters were there but disappeared. For quoted values like \, one of the backslashes will disappear from the Bash command line each time it is processed. You can see the good values a little clearer with V=1: |
Yes: |
|
I'm guessing it is due to CFLAGS not being inside quotes here: |
Cygwin is slower than Linux for two reasons: First, the real time virus checking that you get with Windows. When I am am doing lots of builds, I normally turn of realtime virus checking during the build. The speeds thing up by maybe 40% The other reason, which we can't do anything about, is due to the layer of simulated POSIX interfaces. In particular, fork(). All of the GCC build tools do lots of forks() but Windows does not support fork() and my understanding is that the processing to emulate fork() behavior is slow. You can see this if you make a tight loop in a Bash script and run some do-almost-nothing process as fast as you can. You will find that this is limited to something like 7 forks per second. |
|
As a workaround, is there a way to limit parallelization to 1 core, effectively turning off parallelization? |
Yes, but that will have to be tomorrow morning. |
That, of course, will fail if the CFLAGS from the Make.defs file already contains quotes. |
Unfortunately, that does not fix the problem. Here is the change I made: And here is the result: |
|
I think you can see the failure pretty clearly with make V=1 (see above): The dependency file for libkc.a is create with the following in libs/libc/Makefile: Where: We can see this with make V=1. Here the CFLAGS are fine. BUT since this is passed on the Bash command line, the backslashes will be eliminated. Thanks Bash: So when mkwindesp.sh sees the paths, the backslashes have been eliminated: And the error follows: |
Is this how the executed command actually looks like? If so, the problem are the quotes: note that the quote starts after the |
Right, I see you already thought about it. So indeed the backslashes need to be escaped. How is this usually handled? I'm not quite sure why this is breaking here and not in other places (ie: other places where make is recursively called passing flags) |
If is breaking on the very first invocation of 'make makekdepfile' That is the culprit because the CFLAGS are passed on the command line here in libs/libc/Makefile. Then the very first file that is processes fails because the backslashes in CFLAGS have been lost: That is also where the outermost quotes are picked up as you asked earlier. So this is source of the problem on both accounts. It is Bash command line processing that mucks with the backslashes. Quoting of various forms sometimes works but is not robust. This issue only occurs when you are using a POSIX environment with a Windows native toolchain. In my environment, I use the ARM embedded toolchain for Windows. In this case, we have to do special path handling based on CONFIG_CYGWIN_WINTOOL=y. If we used a GCC toolchain such as the NuttX buildroot built under Cygwin, then there would be no problem (because there would be no Windows paths). We could really simplify the build system is we removed support for Windows native toolchains under Cygwin. But people really like their Windows tools. In the past, Windows was the predominant NuttX build environment. Sourceforge used to keep the statistics and 60% of the downloads were from Windows machines, 30% from Linux, and the remaining 10% from macOS and "Other". But I suspect that that has changed; that fact that this build failure has been in the last two releases says that no one out there is using the Cygwin + Windows native toolchain. |
|
Here are the download stats: https://sourceforge.net/projects/nuttx/files/stats/os?dates=2007-02-17+to+2021-05-29 63% of the downloads are to Windows machines. People are still downloading old releases from Sourceforge! 9 downloads this week! Linux is less than 25% and macOS is a thin sliver in the pie chart. Hackers see the world differently than it really is. It is not all Linux and GCC and ARM and GIT. |
|
I'm surprised by those stats, since as you mention we rarely get issue reports for cygwin. Also, I wonder if those downloads are from actual users since they would be downloading quite an old release by now. |
I am sure that there are a percentage of people in all platforms that are downloading just out of curiosity without actually being users. But I have no reason to believe that that percentages would be different for Windows vs. Linux downloaders.
Maybe we could get some kind of semi-professional poll to determine who NuttX users really are and what is really important to them. My experience is the engineers usually don't really know their customers. |
Yes, and also some Windows users may use Linux VMs to build.
Yes, that is a good idea. We could send it on the mailing list and via other social networks to get good reach. |
|
A temporary work-around is to use the NuttX buildroot tools built under Cygwin. That builds and runs just fine (although I could not build GDB or the NxFLAT tools from the buildroot package). Most people prefer to use industry standard Windows-based tools, however, no DIY tools. |
|
Quoted from the Bash manual:
So this error is expected in Cygwin w/ native ARM tools due to "\" chars inside double quotes. A simple fix would be as follows. |
Failure occurred on first compilation of stm32f4discovery:kostest configuration. The problem probably does not depend on configuration. Most likely all Cygwin builds that use a Windows native toolchain are broken. But perhaps this is limited to Cygwin 2 pass builds with a Windows native toolchain only?
The text was updated successfully, but these errors were encountered: