OS X Process.threads can crash a process #514

Closed
mrjefftang opened this Issue Jun 18, 2014 · 7 comments

Comments

Projects
None yet
3 participants
@mrjefftang
Collaborator

mrjefftang commented Jun 18, 2014

OS: OS X 10.9.3
Python: 3.4.0
psutil: 2.1.1
Code: psutil.Process(target_pid).get_memory_maps()

I'm not quite sure why the code causes the target process to crash, it's rare on my laptop, impossible on a VM, but occurs often on other laptops.

Crash logs always indicate:
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x.....

From what I can tell, the culprit is https://github.com/giampaolo/psutil/blob/master/psutil/_psutil_osx.c#L408

The next virtual memory region isn't necessarily a size offset from the current. The start of the next memory address can be page aligned.

Process:         mds [62]
Path:            /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Support/mds
Identifier:      mds
Version:         800.28
Code Type:       X86-64 (Native)
Parent Process:  launchd [1]
Responsible:     mds [62]
User ID:         0

Date/Time:       2014-06-18 12:02:58.262 -0400
OS Version:      Mac OS X 10.9.3 (13D65)
Report Version:  11
Anonymous UUID:  1660732C-C663-F139-B0C8-BDC5FC7D465F

Sleep/Wake UUID: 5E002B5D-86DB-4A1D-A353-E61483E5C138

Crashed Thread:  7  Dispatch queue: MDSImporter

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000104874000

VM Regions Near 0x104874000:
    VM_ALLOCATE            000000010485b000-0000000104874000 [  100K] rw-/rwx SM=ZER  
--> 
    VM_ALLOCATE            0000000104875000-000000010487a000 [   20K] rw-/rwx SM=ZER  
Process:         mds_stores [116]
Path:            /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Support/mds_stores
Identifier:      mds_stores
Version:         800.28
Code Type:       X86-64 (Native)
Parent Process:  launchd [1]
Responsible:     mds_stores [116]
User ID:         0

Date/Time:       2014-06-18 12:02:57.112 -0400
OS Version:      Mac OS X 10.9.3 (13D65)
Report Version:  11
Anonymous UUID:  1660732C-C663-F139-B0C8-BDC5FC7D465F

Sleep/Wake UUID: 5E002B5D-86DB-4A1D-A353-E61483E5C138

Crashed Thread:  5  Dispatch queue: com.apple.metadata.spotlightindex

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x000000010674a6c0

VM Regions Near 0x10674a6c0:
    MALLOC_LARGE           00000001066f9000-000000010674a000 [  324K] rw-/rwx SM=ZER  
--> 
    MALLOC_LARGE           000000010674b000-000000010677a000 [  188K] rw-/rwx SM=ZER  
@wesyoung

This comment has been minimized.

Show comment
Hide comment
@wesyoung

wesyoung Sep 11, 2014

does what you're experiencing cause just the target process to crash? or the host itself to crash? i'm experiencing a similar issue (python 2.7) but in random cases it causes the host itself to crash.. (10.8 and 10.9)

i have a feeling we're having similar but different issues; wanting to verify...

does what you're experiencing cause just the target process to crash? or the host itself to crash? i'm experiencing a similar issue (python 2.7) but in random cases it causes the host itself to crash.. (10.8 and 10.9)

i have a feeling we're having similar but different issues; wanting to verify...

@mrjefftang

This comment has been minimized.

Show comment
Hide comment
@mrjefftang

mrjefftang Sep 11, 2014

Collaborator

It's almost impossible to reproduce in a deterministic manner. I loop through all processes calling get_memory_maps and eventually stuff starts crashing. Sometimes it'll get the Dock, Finder, or even the login process causing me to get logged out.

Collaborator

mrjefftang commented Sep 11, 2014

It's almost impossible to reproduce in a deterministic manner. I loop through all processes calling get_memory_maps and eventually stuff starts crashing. Sometimes it'll get the Dock, Finder, or even the login process causing me to get logged out.

@wesyoung

This comment has been minimized.

Show comment
Hide comment
@wesyoung

wesyoung Sep 11, 2014

yea; but does the host ever crash? (ie: reboot?)

yea; but does the host ever crash? (ie: reboot?)

@mrjefftang

This comment has been minimized.

Show comment
Hide comment
@mrjefftang

mrjefftang Sep 11, 2014

Collaborator

I've never seen it reboot, only testing with python3 and 10.9.

Collaborator

mrjefftang commented Sep 11, 2014

I've never seen it reboot, only testing with python3 and 10.9.

@wesyoung

This comment has been minimized.

Show comment
Hide comment
@wesyoung

wesyoung Sep 11, 2014

k; then either i'm seeing slightly diff behavior with py2.7 or a diff bug; gonna spend some time trying to re-produce... thx for the insight =)

k; then either i'm seeing slightly diff behavior with py2.7 or a diff bug; gonna spend some time trying to re-produce... thx for the insight =)

@mrjefftang mrjefftang changed the title from OS X get_memory_maps can crash a process to OS X Process.threads can crash a process Jun 4, 2015

@mrjefftang

This comment has been minimized.

Show comment
Hide comment
@mrjefftang

mrjefftang Jun 4, 2015

Collaborator

I've done some additional testing and I've come to the conclusion that it is not get_memory_maps but instead get_threads. I'm still testing to see if other attributes will cause failure.

If you crash launchd (OS X's version of init), the kernel will panic causing an auto reboot which @wesyoung may have seen.

Collaborator

mrjefftang commented Jun 4, 2015

I've done some additional testing and I've come to the conclusion that it is not get_memory_maps but instead get_threads. I'm still testing to see if other attributes will cause failure.

If you crash launchd (OS X's version of init), the kernel will panic causing an auto reboot which @wesyoung may have seen.

giampaolo added a commit that referenced this issue Oct 5, 2016

check return value of proc_regionfilename(); this possibly addresses #…
…514: [OSX] Process.memory_maps() segfault (critical!).D
@giampaolo

This comment has been minimized.

Show comment
Hide comment
@giampaolo

giampaolo Oct 5, 2016

Owner

I know this is a long standing issue but I think I may have found the cause of this. We forgot to check the return value of proc_regionfilename().
This is now fixed here:
833e70a
I am gonna close this for now but if you have the chance to test this and confirm the progress is gone that would be great.

Owner

giampaolo commented Oct 5, 2016

I know this is a long standing issue but I think I may have found the cause of this. We forgot to check the return value of proc_regionfilename().
This is now fixed here:
833e70a
I am gonna close this for now but if you have the chance to test this and confirm the progress is gone that would be great.

@giampaolo giampaolo closed this Oct 5, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment