Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Work-around for our windows CI system. #2086

Closed
wants to merge 8 commits into from

Conversation

slaff
Copy link
Contributor

@slaff slaff commented Jun 16, 2020

The compilation ends with "C:\MinGW\msys\1.0\bin\make.exe: *** couldn't commit memory for cygwin heap, Win32 error 0"

This change tries to work around this issue on appveyor.

…'t commit memory for cygwin heap, Win32 error 0"

This change tries to work around this issue on appveyor.
@slaff slaff added this to the 4.2.0 milestone Jun 16, 2020
@mikee47
Copy link
Contributor

mikee47 commented Jun 16, 2020

I've failed to reproduce the problem in my own appveyor builds, but the issue looks like it's related to CMake / MinGW interaction - https://help.appveyor.com/discussions/problems/3193-cmake-building-for-mingw-issue-with-git-shexe

@mikee47
Copy link
Contributor

mikee47 commented Jun 16, 2020

@slaff Sorry, didn't intend to push to your repo!

@slaff
Copy link
Contributor Author

slaff commented Jun 16, 2020

Sorry, didn't intend to push to your repo!

@mikee47 It's ok. But the compilation now fails even earlier.

@mikee47 mikee47 force-pushed the fix/appveyor-out-of-memory branch from 3766ed9 to 1b0e772 Compare June 16, 2020 11:52
@mikee47
Copy link
Contributor

mikee47 commented Jun 16, 2020

OK, so CMake not involved here so ignore that. This error has been reported quite recently (https://osdn.net/projects/mingw/ticket/38356) so updating MinGW (more precisely, MSYS) might fix it. Doing that causes the error with socklen_t so that's now fixed. It caused a collision with npcap (used for Windows network support) so I've updated that to the latest as well.

@slaff All builds fine when I push to appveyor, so have pushed to your branch to check if it fixes things. I don't understand why appveyor doesn't fail in my account though - perhaps there are additional resource limitations when running for SmingHub?

@slaff
Copy link
Contributor Author

slaff commented Jun 16, 2020

I don't understand why appveyor doesn't fail in my account though - perhaps there are additional resource limitations when running for SmingHub?

It does not fail in my one either... I should probably open a ticket in their system and see what they can tell us.

@mikee47
Copy link
Contributor

mikee47 commented Jun 16, 2020

This breaks linux build, easy fix but I'll wait to see if it's sorted the appveyor before pushing anything else.

@mikee47
Copy link
Contributor

mikee47 commented Jun 16, 2020

OK, so still broken...

@mikee47
Copy link
Contributor

mikee47 commented Jun 16, 2020

I'm now checking out this approach https://stackoverflow.com/questions/9932874/win-8-cygwin-heap-failure using a rebased msys-1.0.dll

@mikee47 mikee47 force-pushed the fix/appveyor-out-of-memory branch from 1b0e772 to a395c42 Compare June 16, 2020 12:34
@mikee47
Copy link
Contributor

mikee47 commented Jun 16, 2020

Rebased DLL didn't work.

@mikee47
Copy link
Contributor

mikee47 commented Jun 16, 2020

The failure is with ar.exe so trying a patched version to increase default heap https://cygwin.com/cygwin-ug-net/setup-maxmem.html

@mikee47 mikee47 force-pushed the fix/appveyor-out-of-memory branch 4 times, most recently from 623d27e to 9bf3ada Compare June 16, 2020 19:53
@mikee47
Copy link
Contributor

mikee47 commented Jun 17, 2020

Fixed by reducing PATH to the minimum:

c:\mingw\bin
c:\mingw\msys\1.0\bin
C:\Windows\system32
C:\Windows
C:\Windows\System32\Wbem
C:\Windows\System32\WindowsPowerShell\v1.0\
C:\Program Files\7-Zip
C:\Program Files (x86)\CMake\bin
C:\Python27
C:\Python27\Scripts
C:\Tools\curl\bin
C:\Program Files\PowerShell\6\
C:\Program Files\Git\cmd
C:\Program Files\Git\usr\bin

Still looking for the actual culprit and a formal fix.

@mikee47 mikee47 force-pushed the fix/appveyor-out-of-memory branch 4 times, most recently from 3cd389b to 7a55d98 Compare June 17, 2020 13:13
@mikee47
Copy link
Contributor

mikee47 commented Jun 17, 2020

I've bisected the PATH directories and it looks like reducing the PATH environment variable to below 512 characters fixes the problem. If there's internal path truncation (or buffer overruns) going on then this would explain the unpredictable behaviour.

@mikee47 mikee47 force-pushed the fix/appveyor-out-of-memory branch from 7a55d98 to b9da5fe Compare June 17, 2020 19:20
@mikee47
Copy link
Contributor

mikee47 commented Jun 18, 2020

Problem not yet fixed but here's a rundown of what I've tried. Remoting into the appveyor session after build failure, I can reproduce the error like this:

cd c:\projects\tests\HostTests
set SMING_HOME=c:\projects\sming-xxx\Sming
make bearssl-esp8266-clean
make -j2

Note that running make again completes the build successfully.

As noted above, if I reduce the length of the PATH environment variable to less than 512 characters then the above sequence succeeds without error. However, setting the same path in build.cmd doesn't fix the problem.

Note also that rebasing as noted above doesn't have any effect, nor does updating mingw to latest version.

Something else to try is using make tests instead of the above. I'm assuming that the error occurs within ar but it could also be within make itself.

@mikee47
Copy link
Contributor

mikee47 commented Jun 18, 2020

I'm happy to keep pursuing this but need a checklist of thing to try as it's quite tedious. At present, the queue time for the SmingHub appveyor builds is quite long (1 hour+) and then I get an hour at most to try things out.

@mikee47
Copy link
Contributor

mikee47 commented Jun 18, 2020

Something that would really help is to be able to reproduce the error on my own appveyor account. As yet I have no clue as to why the two should differ.

@slaff
Copy link
Contributor Author

slaff commented Jun 18, 2020

I'm happy to keep pursuing this but need a checklist

@mikee47 Better leave it for now. It would be better to spend time on ESP32 architecture instead of the Appveyor build which runs fine on yours and mine VM but fails on the official Sming VM.

@slaff slaff removed this from the 4.2.0 milestone Jun 18, 2020
@slaff slaff force-pushed the fix/appveyor-out-of-memory branch from 476965a to 39ee211 Compare June 19, 2020 08:55
@slaff slaff force-pushed the fix/appveyor-out-of-memory branch from 39ee211 to d131aee Compare June 19, 2020 09:08
@mikee47
Copy link
Contributor

mikee47 commented Oct 24, 2020

@slaff I've found the actual cause of this, bear with...

@slaff
Copy link
Contributor Author

slaff commented Oct 24, 2020

Closing in favour of #2124.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants