New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All *nix (cygwin) tools crash as of 15082014 #272

Closed
MartinSGill opened this Issue Aug 17, 2015 · 96 comments

Comments

Projects
None yet
6 participants
@MartinSGill

MartinSGill commented Aug 17, 2015

As of this latest update for every unix-like tool I use (e.g. ssh included with git) I get crash reports popping up.

The programs themselves do appear to work, so I'm guessing this is somehow related to the new hook module.

bash.sh
(crash), when I click away the report I get the prompt
ls
directory is listed, (crash) when I click away the report I get the prompt back
when I call something like git pull (which call many unix commands I end up with a desktop full of errors.

2015-08-17_11h25_34

This only happens in ConEmu, regardless of which shell (e.g. powershell, cmd etc) I use.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Aug 17, 2015

I do not see any crashes. So, you need to either describe step-by-step instruction, or provide crash dump.
https://conemu.github.io/en/MemoryDump.html

@MartinSGill

This comment has been minimized.

MartinSGill commented Aug 17, 2015

How I reproduce it:

  1. Start ConEmu (x86, or x64 both crash), with powershell (admin) console
  2. Change to a git repository folder
  3. run "git pull" (git/bin is in path)

Crash dump: https://www.dropbox.com/s/44ygkgqh6nvk4uw/WER3F44.tmp.mdmp?dl=0

Everything was working fine last Friday, and ConEmu is the only thing I've updated.

@nileflowers

This comment has been minimized.

nileflowers commented Aug 17, 2015

+1

Seeing it all git-bash commands on Windows 7 on 150816 alpha. None of these tools are updated except ConEmu.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Aug 17, 2015

@MartinSGill
I see two other detouring apps: NVIDIA CoProcManager and Sophos Anti-Virus. I don't have any of them and can't check. But one of them may raise this crash.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Aug 17, 2015

@nileflowers
Same here. Crash dump.

@nileflowers

This comment has been minimized.

nileflowers commented Aug 17, 2015

@MartinSGill

This comment has been minimized.

MartinSGill commented Aug 18, 2015

I've been reverting my install. Build 150813 is the last build that works for me. Whatever the issue is it seems to have been introduced in 150814.

@MartinSGill MartinSGill changed the title from All *nix (cygwin) tools crash as of 15082015 to All *nix (cygwin) tools crash as of 15082014 Aug 18, 2015

@kraml

This comment has been minimized.

kraml commented Aug 18, 2015

I have the same problem here. And it does not only affect cygwin tools... The tools so far I know are affected include:

  • msysgit shell
  • rustc compiler - rustc itself runs OK but when it calls the gcc.exe, gcc.exe will fail
  • HP RESTful API tool - a command line tool provided by HP for their server management

Uncheck Inject ConEmuHK in Features can workaround this issue.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Aug 18, 2015

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Aug 18, 2015

  1. Both supplied crash dumps has either C:\Program Files (x86)\NVIDIA Corporation\CoProcManager\_etoured.dll or C:\Program Files (x86)\NVIDIA Corporation\CoProcManager\detoured.dll. As I've said before, this detours may harm normal behavior of applications.
  2. Even I have installed NVIDIA drivers this CoProcManager is not installed on my PC. So I can't check anything. What is that CoProcManager??? Also, google saw a lot of issues with it.
  3. Both crash dumps do not describe the first crash. Pleas follow these steps: http://conemu.github.io/en/CrashDump.html#manual-dumps And I'm insterested in full memory dumps of crashed process and its direct parent process.
@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Aug 18, 2015

One more thing. Because I don't have CoProcManager installed on my PC, I need to know exact Video Card models and drivers (version, installation file name, url) you are using. And, because I'm not a end-point user of this product, seems like I can't file an issue to NVIDIA.

@kraml

This comment has been minimized.

kraml commented Aug 18, 2015

@Maximus5 here is the dump for the hprest.exe command as a HP tool.
https://dl.dropboxusercontent.com/u/16148499/CEDump-150816-56060-150818180008.7z

I do have a nVIDIA card, Quadro K2100m, on my laptop. The driver version is 352.86. DL link:
http://www.nvidia.com/download/driverResults.aspx/85053/en-us

I'm also downloading the latest video driver and will feedback whether it changes anything.

Update regarding video driver:
Since my laptop support hybrid graphics, I just tested uninstall nVIDIA driver completely and leave only the Intel integrated video. The problem was gone. After installed back the latest nVIDIA driver, it's back. So it definitely is related to nVIDIA driver, but only happens after a certain ConEmu revision...

@Maximus5 Maximus5 referenced this issue Aug 18, 2015

Closed

Protoc crash #278

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Aug 18, 2015

There are a lot of page found by google by simple query "coprocmanager detoured.dll problem".
www.google.com/search?...

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Aug 18, 2015

And the expected result.
2015-08-19_01-19-22

So, at least for tests, I need:

  • This mysterios CoProcManager folder (well, I belive the files are somewhere in the distro folder, but I need exact list of them)
  • Exact command which initialize detours. I believe it somewhere in the startup items and most probably it infiltrates explorer.exe.

Till than I can't tell who exactly is responsible for these crashes. And may be that is NVIDIA problem, especially because a lot of another compains exist on the internet. ConEmu sources are open, but NVIDIA sources seems like to be closed. So I even can't check or fix the bug, if the bug is inside CoProcManager.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Aug 18, 2015

And I think this simple app will crash too on reported workstations. Please check.
https://github.com/Maximus5/test-callexit/releases/tag/v1.0

@kraml

This comment has been minimized.

kraml commented Aug 19, 2015

Here are the files under the coprocomanager folder:
C:\Program Files\NVIDIA Corporation\coprocmanager\detoured.dll
C:\Program Files\NVIDIA Corporation\coprocmanager\nvd3d9wrapx.dll
C:\Program Files\NVIDIA Corporation\coprocmanager\nvdxgiwrapx.dll
C:\Program Files (x86)\NVIDIA Corporation\coprocmanager\detoured.dll
C:\Program Files (x86)\NVIDIA Corporation\coprocmanager\nvd3d9wrap.dll
C:\Program Files (x86)\NVIDIA Corporation\coprocmanager\nvdxgiwrap.dll

The two detoured.dll are in the driver distro, the two nv*.dll fiiles are as .dl_ so need to be uncompressed, 7-zip can do it.

Checked by sysinternal autorun tool none of these files are referenced in common startup location. However they are referenced in the registry.

I've uploaded an archive of the above dll files and the registry export.
https://dl.dropboxusercontent.com/u/16148499/ConEmu%20Crash%20nVIDIA%20files.7z

I can also confirm that after renamed the two coprocmanager folders the problem was gone.

I would like to try the simple callexit app but lacks of two dlls on my system. Although downloaded them from opendll.com still no luck as the dependencies have dependencies. Could you upload a static linked version?

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Aug 19, 2015

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Aug 19, 2015

Is detoured.dll loaded in explorer.exe? Check with ProcessExplorer.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Aug 19, 2015

Try to filter by nvidia and show what autorun finds. Here is mine:
2015-08-19_04-16-19

@kraml

This comment has been minimized.

kraml commented Aug 19, 2015

No binary download for the CallExit 1.1 release...

detoured.dll is not loaded in any process. The other two nv*.dll files under the same folder are loaded in many processes but not explorer.exe. detoured.dll are their dependencies though.

nvidia autorun

@Maximus5

This comment has been minimized.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Aug 19, 2015

In the latest dump from this comment I see a lot of another detours: AVG, MacrType, Sizer, HP ProtectTools.
And only removing NVIDIA's CoProcManager fixes the crash? From my point of view, this fact side by side with this search query is an evidence that problems must be fixed by NVIDIA.

@kraml

This comment has been minimized.

kraml commented Aug 19, 2015

CallExit 1.1 crashed as expected, if Inject ConEmuHK is checked and nVIDIA coprocmanager is in place.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Aug 19, 2015

It would be interesting to find minimal reproduction code. It may help to find a workaround or to submit bugreport to NVIDIA. I prefer last option of course...

So, can anyone from this topic build ConEmuHk with VisualStudio or MinGW? To avoid a lot of test iterations...

@javahaxxor

This comment has been minimized.

javahaxxor commented Aug 19, 2015

happens to me too. Uncheck Inject ConEmuHK works as a workaround, thanks kraml

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Aug 19, 2015

Remove CoProcManager as workaround... Without ConEmuHk you may face another crashes and lack of features.

@kraml

This comment has been minimized.

kraml commented Aug 19, 2015

@Maximus5 I can build with MinGW, if you can provide instructions.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Aug 19, 2015

Take latest daily branch.
Cd into ./src/ConEmuHk
Supposing you have 32bit mingw toolchain:

mingw32-make -f makefile_gcc DIRBIT=32

Take binary from ./Release folder and update your test location (it may be this ./Release folder).
Run ConEmu

ConEmu.exe -basic -cmd {cmd-32}

Post here when you have success or crash after that.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Sep 7, 2015

Build 150908 was releases. Most probably, crash will repeat... If so, I need new dumps with this version.

@kraml

This comment has been minimized.

kraml commented Sep 7, 2015

Yes it crashed as expected. Here are the dumps from the same process, similar to my previous comment #272 (comment)

https://dl.dropboxusercontent.com/u/16148499/CEDump-20150907%201359.7z

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Sep 7, 2015

Please check the following patch and create new dumps. Two dumps is quite enough.
https://sourceforge.net/projects/conemu/files/temp/ConEmuHk.150908b.dll.7z/download

@kraml

This comment has been minimized.

@Maximus5

This comment has been minimized.

@kraml

This comment has been minimized.

kraml commented Sep 7, 2015

Crash the same. Need dump?

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Sep 7, 2015

Yes, please

@kraml

This comment has been minimized.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Sep 7, 2015

@kraml

This comment has been minimized.

@Maximus5

This comment has been minimized.

@kraml

This comment has been minimized.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Sep 9, 2015

Can you, please, run CallExit.exe outside of ConEmu and create full dump at the moment of Press Enter to continue appears? Please, ensure that after Enter it will not crash (outside of ConEmu).
You may create dump with ProcessExplorer.

@Maximus5

This comment has been minimized.

@kraml

This comment has been minimized.

kraml commented Sep 9, 2015

No crash with ConEmuHk.150908f.dll.7z now. I've test callexit.exe, rustc, hprest and some msys2 commands, with both 64-bit and 32-bit ConEmu, no more crash now.

As for the dump you requested in this comment #272 (comment) , I would like to provide one but when I create dump from PE, I got a dialog that says "Error configuring dump resources: system cannot found the designated file", and the dump file is 0 byte. I never used PE to dump before so not sure whether I missed any required configuration on my system.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Sep 9, 2015

Well, you may use ConEmuC to create dump if running process.

@kraml

This comment has been minimized.

kraml commented Sep 9, 2015

Here is the dump for callexit from cmd.exe instead of ConEmu.

https://dl.dropboxusercontent.com/u/16148499/CEDump-150908-46504-150909214707.7z

I created it by start callexit, find its PID from PE, then run comemuc /debugpid= and press ctrl+break to generate a full dump when the press enter message is on the screen.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Sep 10, 2015

Weird... Last dump has no CoProcManager dll's loaded. What do NVIDIA think about?

OK, this is last test before next release. I'm not happy with workaround, but it's working at least. Anyway, I still believe the problem is inside NVIDIA driver.

https://sourceforge.net/projects/conemu/files/temp/ConEmuHk.150908g.dll.7z/download

@kraml

This comment has been minimized.

kraml commented Sep 10, 2015

The last g dll works ok, no crash.

I do notice that when started from cmd.exe there are no NVIDIA coprocmanager DLLs attach with callexit.exe. But if start it directly from explorer windows by double click, NVIDIA is there. Here is the dump in such case, if you would like to look into it.

https://dl.dropboxusercontent.com/u/16148499/CEDump-150908-71028-150910202156.7z

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Sep 10, 2015

I do notice that when started from cmd.e­xe there are no NVIDIA coprocmanager DLL­s attach with callexit.exe.
That's interesting... How did you start cmd.exe and does that cmd.exe have NVIDIA dlls loaded?

@kraml

This comment has been minimized.

kraml commented Sep 10, 2015

I started cmd.exe by click the start menu button, type cmd.exe in the search box and press enter. I just tried find the cmd.exe file in explorer window under C:\Windows\System32 and double click it. In neither case no NVIDIA dll was loaded.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Sep 10, 2015

Weird logic...

@nileflowers

This comment has been minimized.

nileflowers commented Sep 11, 2015

This issue is no longer occurring after updating to 150910. I have the Settings | Features | In-console options | Inject ConEmuHk checkbox enabled, so no need for the previous semi-workaround.

@kraml

This comment has been minimized.

kraml commented Sep 11, 2015

yep 150910 works.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Sep 26, 2015

Can you please create debug log files for Shell mode during execution of CallExit.exe?

http://conemu.github.io/en/SettingsDebug.html

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Oct 7, 2015

How about build 151006?

Related: gh-341, gh-323

@MartinSGill

This comment has been minimized.

MartinSGill commented Oct 8, 2015

I've not noticed any issues since latest 151006.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment