New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sporadic "Command exited with code -1" on some Visual Studio 2017 builds #1490

Closed
FeodorFitsner opened this Issue Apr 11, 2017 · 23 comments

Comments

Projects
None yet
4 participants

@FeodorFitsner FeodorFitsner added this to the 16-Apr-2017 milestone Apr 11, 2017

@jagerman

This comment has been minimized.

Show comment
Hide comment
@jagerman

jagerman Apr 17, 2017

Still failing. I have no idea how to debug this. I've tried enabling RDP access on failure, re-running the build on the exact same appveyor image where it just failed, out of the same build directory, and it succeeds every time.

I tried the build in a loop on a local VM (with Windows 10 + VS2017), and 100/100 builds passed.

jagerman commented Apr 17, 2017

Still failing. I have no idea how to debug this. I've tried enabling RDP access on failure, re-running the build on the exact same appveyor image where it just failed, out of the same build directory, and it succeeds every time.

I tried the build in a loop on a local VM (with Windows 10 + VS2017), and 100/100 builds passed.

@jagerman

This comment has been minimized.

Show comment
Hide comment
@jagerman

jagerman Apr 17, 2017

(For now, I'm just ignoring the failures and waiting for the next image update).

jagerman commented Apr 17, 2017

(For now, I'm just ignoring the failures and waiting for the next image update).

@FeodorFitsner

This comment has been minimized.

Show comment
Hide comment
@FeodorFitsner

FeodorFitsner Apr 17, 2017

Member

Can you do I repro solution in a public repository? I understand it's not happening with any project, but with only specific ones.

Member

FeodorFitsner commented Apr 17, 2017

Can you do I repro solution in a public repository? I understand it's not happening with any project, but with only specific ones.

@jagerman

This comment has been minimized.

Show comment
Hide comment
@jagerman

jagerman Apr 17, 2017

Sorry, I don't follow: "I repro solution"

jagerman commented Apr 17, 2017

Sorry, I don't follow: "I repro solution"

@jagerman

This comment has been minimized.

Show comment
Hide comment
@jagerman

jagerman Apr 17, 2017

More details/investigation/tracking in pybind/pybind11#792.

jagerman commented Apr 17, 2017

More details/investigation/tracking in pybind/pybind11#792.

@FeodorFitsner

This comment has been minimized.

Show comment
Hide comment
@FeodorFitsner

FeodorFitsner Apr 17, 2017

Member

Sorry, I meant do you have a public repository/build reproducing the issue.

Member

FeodorFitsner commented Apr 17, 2017

Sorry, I meant do you have a public repository/build reproducing the issue.

@jagerman

This comment has been minimized.

Show comment
Hide comment
@jagerman

jagerman Apr 17, 2017

Ah, yes, the https://github.com/pybind/pybind11 repository exhibits it.

jagerman commented Apr 17, 2017

Ah, yes, the https://github.com/pybind/pybind11 repository exhibits it.

@jagerman

This comment has been minimized.

Show comment
Hide comment
@jagerman

jagerman Apr 17, 2017

Here's an interesting failure that didn't even make it to the pybind code before failing (cmake's initial does-the-compiler-work test failed): https://ci.appveyor.com/project/wjakob/pybind11/build/1.0.1691/job/etguctmxjas78jn2

jagerman commented Apr 17, 2017

Here's an interesting failure that didn't even make it to the pybind code before failing (cmake's initial does-the-compiler-work test failed): https://ci.appveyor.com/project/wjakob/pybind11/build/1.0.1691/job/etguctmxjas78jn2

@ThirteenAG

This comment has been minimized.

Show comment
Hide comment
@ThirteenAG

ThirteenAG Apr 18, 2017

I have this too, for example: https://ci.appveyor.com/project/ThirteenAG/widescreenfixespack/build/218
Usually rebuilding the same commit works, but "Command exited with code -1" definitely appears every once in a while.

ThirteenAG commented Apr 18, 2017

I have this too, for example: https://ci.appveyor.com/project/ThirteenAG/widescreenfixespack/build/218
Usually rebuilding the same commit works, but "Command exited with code -1" definitely appears every once in a while.

@Neproify

This comment has been minimized.

Show comment
Hide comment

Neproify commented Apr 18, 2017

@FeodorFitsner

This comment has been minimized.

Show comment
Hide comment
@FeodorFitsner

FeodorFitsner Apr 19, 2017

Member

We've deployed Visual Studio 2017 update - version 15.1 (26403.07) to Visual Studio 2017 image. It looks like it fixed the issue with exit code -1.

Could you guys please check and let me know how that worked?

Member

FeodorFitsner commented Apr 19, 2017

We've deployed Visual Studio 2017 update - version 15.1 (26403.07) to Visual Studio 2017 image. It looks like it fixed the issue with exit code -1.

Could you guys please check and let me know how that worked?

@jagerman

This comment has been minimized.

Show comment
Hide comment
@jagerman

jagerman Apr 19, 2017

When did the updated version get deployed? The last -1 failure I saw was around 3 hours ago (several builds since haven't had a failure).

jagerman commented Apr 19, 2017

When did the updated version get deployed? The last -1 failure I saw was around 3 hours ago (several builds since haven't had a failure).

@FeodorFitsner

This comment has been minimized.

Show comment
Hide comment
@FeodorFitsner

FeodorFitsner Apr 19, 2017

Member

@jagerman about an hour ago. Actually, we were using your project for verifying the fix: https://ci.appveyor.com/project/FeodorFitsner/pybind11/history 😄

Member

FeodorFitsner commented Apr 19, 2017

@jagerman about an hour ago. Actually, we were using your project for verifying the fix: https://ci.appveyor.com/project/FeodorFitsner/pybind11/history 😄

@jagerman

This comment has been minimized.

Show comment
Hide comment
@jagerman

jagerman Apr 19, 2017

Cool! :) I'll keep an eye on it and report back if I encounter any more.

jagerman commented Apr 19, 2017

Cool! :) I'll keep an eye on it and report back if I encounter any more.

@jagerman

This comment has been minimized.

Show comment
Hide comment
@jagerman

jagerman Apr 29, 2017

I just encountered another one:

https://ci.appveyor.com/project/wjakob/pybind11/build/1.0.1751/job/8ua0rcj9a8ie2ubr

This is the first one I've seen, so it seems greatly reduced but perhaps not entirely solved.

jagerman commented Apr 29, 2017

I just encountered another one:

https://ci.appveyor.com/project/wjakob/pybind11/build/1.0.1751/job/8ua0rcj9a8ie2ubr

This is the first one I've seen, so it seems greatly reduced but perhaps not entirely solved.

@FeodorFitsner

This comment has been minimized.

Show comment
Hide comment
@FeodorFitsner

FeodorFitsner Apr 29, 2017

Member

OK. Let's try removing /logger:%MSBuildLogger% and see if that might be affecting the build.
Second, is there any way to get more detailed logging in case of error? If it exists with code -1 I bet there should be something in the detailed log - some error or warning.

Member

FeodorFitsner commented Apr 29, 2017

OK. Let's try removing /logger:%MSBuildLogger% and see if that might be affecting the build.
Second, is there any way to get more detailed logging in case of error? If it exists with code -1 I bet there should be something in the detailed log - some error or warning.

@jagerman

This comment has been minimized.

Show comment
Hide comment
@jagerman

jagerman Apr 29, 2017

Back when this was occurring often—it is very rare now: this was the first failure I've seen since the update 11 days ago—I tried some builds without /logger and still got failures (this one, for example—the log is really long as I also had MSBuild verbosity set to diagnostic mode, but there doesn't appear to be anything useful in that diagnostic output).

I also haven't been able to find anything useful in logs; though I admit Windows is a bit foreign to me, so I'm not entirely sure where to look for this (or even which program's logs I should be looking for). Something in the Event Viewer? Is there anything helpful I could add to the build script to grab any sort of system errors?

I also don't know what exit code -1 means: is that a process being killed by the system for some reason? Or some internal MSBuild error? (If I try killing MSBuild through task manager, I get a 1 as exit code). And when an individual compilation fails, MSBuild recognizes that and compiler output appears.

jagerman commented Apr 29, 2017

Back when this was occurring often—it is very rare now: this was the first failure I've seen since the update 11 days ago—I tried some builds without /logger and still got failures (this one, for example—the log is really long as I also had MSBuild verbosity set to diagnostic mode, but there doesn't appear to be anything useful in that diagnostic output).

I also haven't been able to find anything useful in logs; though I admit Windows is a bit foreign to me, so I'm not entirely sure where to look for this (or even which program's logs I should be looking for). Something in the Event Viewer? Is there anything helpful I could add to the build script to grab any sort of system errors?

I also don't know what exit code -1 means: is that a process being killed by the system for some reason? Or some internal MSBuild error? (If I try killing MSBuild through task manager, I get a 1 as exit code). And when an individual compilation fails, MSBuild recognizes that and compiler output appears.

@jagerman

This comment has been minimized.

Show comment
Hide comment
@jagerman

jagerman Apr 29, 2017

And a couple more: https://ci.appveyor.com/project/wjakob/pybind11/build/1.0.1760/job/bym09yatp2f7xb6e
https://ci.appveyor.com/project/jagerman/pybind11/build/1.0.683/job/tkqgq0c8rnfqvxwd

Until today, there hadn't been any such failures since this issue was closed AFAIK. Did something change in the VS 2017?

jagerman commented Apr 29, 2017

And a couple more: https://ci.appveyor.com/project/wjakob/pybind11/build/1.0.1760/job/bym09yatp2f7xb6e
https://ci.appveyor.com/project/jagerman/pybind11/build/1.0.683/job/tkqgq0c8rnfqvxwd

Until today, there hadn't been any such failures since this issue was closed AFAIK. Did something change in the VS 2017?

@FeodorFitsner

This comment has been minimized.

Show comment
Hide comment
@FeodorFitsner

FeodorFitsner May 1, 2017

Member

Will take a look. This maybe some scheduling task or some built-in check for updates (Windows updates are disabled) interfering with the build process.

Member

FeodorFitsner commented May 1, 2017

Will take a look. This maybe some scheduling task or some built-in check for updates (Windows updates are disabled) interfering with the build process.

@Neproify

This comment has been minimized.

Show comment
Hide comment

Neproify commented May 12, 2017

@FeodorFitsner

This comment has been minimized.

Show comment
Hide comment
@FeodorFitsner

FeodorFitsner May 14, 2017

Member

Hopefully, that will be fixed with the next update as we disabled most of VS-related scheduled tasks.

Member

FeodorFitsner commented May 14, 2017

Hopefully, that will be fixed with the next update as we disabled most of VS-related scheduled tasks.

@jagerman

This comment has been minimized.

Show comment
Hide comment
@jagerman

jagerman May 20, 2017

Seems to be working so far; all builds for the last couple days have been good.

jagerman commented May 20, 2017

Seems to be working so far; all builds for the last couple days have been good.

@FeodorFitsner

This comment has been minimized.

Show comment
Hide comment
@FeodorFitsner

FeodorFitsner May 20, 2017

Member

Cool, thanks for the update!

Member

FeodorFitsner commented May 20, 2017

Cool, thanks for the update!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment