-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OSX: memory_maps() segfaults or raise EINVAL #1291
Comments
Please paste the full trace back
On Sun, 10 Jun 2018 at 06:59, wrishel ***@***.***> wrote:
I get Exception: OSError(22, 'Invalid argument') on OSX 10.13.4 with
Python 2.7
Adding this to the loop makes it go away:
for proc in psutil.process_iter():
if proc.pid == os.getpid(): continue
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#1291>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAplLAXf0fJCzEADUoq4gBchk7p8xiSSks5t7KeegaJpZM4UhlYK>
.
--
Giampaolo - http://grodola.blogspot.com
|
Giampaolo, here is the full traceback. I'm embarrassed that I didn't think to do this in the first place. As always I appreciate how promptly you reply. Wes
|
Duplicate of #1209. |
@giampaolo I'm still getting this with the latest code. #1209 seems to deal with zombie processes and
Please consider reopening this issue. This fails multiple tests when running
I wrote a small program that goes over the memory mapping to see where the problem is here. I'm also getting "errno 22" on the last regions. |
Mmm segfault is not good. I'm not familiar with that code: |
What's even worse this time is that travis nor I can reproduce it. :-\ |
I'm still getting the same thing: Please see program I wrote here - it seems that Output from test program above program (snipped to where it starts failing with "errno 22"):
And snipped output from
|
I see. Yes, I tried your program and I can't get errno=22 (EINVAL) on OSX 10.11.6 . |
@wiggin15 I've found this: if (info.max_protection!=0 && !contiguous) {
...
int ret = proc_regionfilename (dbg->pid, address,
module_name, sizeof (module_name)); On my machine struct vm_region_submap_info_64 {
vm_prot_t protection; /* present access protection */
vm_prot_t max_protection; /* max avail through vm_prot */
vm_inherit_t inheritance;/* behavior of map/obj on fork */
memory_object_offset_t offset; /* offset into object/map */
unsigned int user_tag; /* user tag on map entry */
unsigned int pages_resident; /* only valid for objects */
unsigned int pages_shared_now_private; /* only for objects */
unsigned int pages_swapped_out; /* only for objects */
unsigned int pages_dirtied; /* only for objects */
unsigned int ref_count; /* obj/map mappers, etc */
unsigned short shadow_depth; /* only for obj */
unsigned char external_pager; /* only for obj */
unsigned char share_mode; /* see enumeration */
boolean_t is_submap; /* submap vs obj */
vm_behavior_t behavior; /* access behavior hint */
vm32_object_id_t object_id; /* obj/map name, not a handle */
unsigned short user_wired_count;
unsigned int pages_reusable;
}; Basically what I am suggesting is trying to reverse engineer this thing in order to understand under what conditions we should avoid calling |
I'm afraid that doesn't work either - This doesn't reproduce on our continuous-integration mac machine, which runs macOS 10.12.1, but it does reproduce on more development mac stations in my team, running different types of mac machines and macOS 10.14.3. |
I see. Thanks for looking into this. This reminds me of |
We have 2 issues here: the segfault and the compatibility breakage. I think a reasonable way to approach this is the following:
|
Done in b31f3bf. Closing this out. |
On a second thought also raising AccessDenied for the current process is backward incompatible. Returning an empty list is too. It looks like the only way forward is to remove this API in 5.6.0 (which is a major version BTW). |
OK, [1] |
Call to .memory_maps() fails on Azure pipelines with: Traceback (most recent call last): ... return cext.proc_memory_maps(self.pid) OSError: [Errno 22] Invalid argument Also, support for memory_maps() has been removed for macOS starting psutil-v5.6.x xref: giampaolo/psutil#1291
Call to .memory_maps() fails on Azure pipelines with: Traceback (most recent call last): ... return cext.proc_memory_maps(self.pid) OSError: [Errno 22] Invalid argument Also, support for memory_maps() has been removed for macOS starting psutil-v5.6.x xref: giampaolo/psutil#1291
* origin/master: (150 commits) Linux / CPU freq, fixes giampaolo#1481 improve pmap.py script reuse ps.py script in psutil.test() move get_terminal_size() in _compat.py improve ps.py script improve ps.py script move bytes2human() into psutil._common and reused it from scripts dir fix windows failure re. py 2 vs. 3 fix linux tests fix giampaolo#1474: fix formatting of psutil.tests() which mimicks 'ps aux' output give CREDITS for giampaolo#1480 remove outdated tests Fix read access violation in psutil.cpu_count(logical=False) (giampaolo#1480) update doc update doc [Win] Process IO priority constants + high priority (giampaolo#1479 / giampaolo#1476) don't fail if there are not prev failed tests fix giampaolo#1478: add make command to re-run tests failed on last run [Win] return value is not properly handled for undocumented NT* Windows APIs. (giampaolo#1477) fix error on py 2.7 where OSError doesn't always have winerror attribute update HISTORY for giampaolo#1475 properly check OSError.winerror refactor ionice() on Linux refactor ionice() on Linux ionice test refactoring giampaolo#1404: fix regression not returning CPUs > 9 give CREDITS to Daniel Beer for giampaolo#1471 Fix spurious exception when iterating processes on Solaris (giampaolo#1471) give CREDITS for giampaolo#1470 Fix corner case when /etc/mtab doesn't exist and procfs=/proc (giampaolo#1470) update DEVNOTES Typo fixed (giampaolo#1469) giampaolo#1458: implement colors on Windows update HISTORY / CREDITS, fix some C warnings Make uptime type consitent to fix boot time error. (giampaolo#1225) fix giampaolo#1463: cpu_distribution.py script is broken update issue template issue giampaolo#1404 / linux / phys CPUs count Big docfix (giampaolo#1464) Big docfix (giampaolo#1464) Make tests invariant to LANG setting (giampaolo#1462) test runner: show errors on KeyboardInterrupt test runner refactoring (avoid code duplication) Coloured tests (giampaolo#1459) pre-release [Windows] calculate USS memory by using NtQueryVirtualMemory (giampaolo#1453) fix version highlighting in docs/index (giampaolo#1455) fix version highlighting in README (giampaolo#1454) run win specific tests twice as fast test refactoring test: avoid failing at import time mention how to run tests in INSTALL guide giampaolo#1448: fix Wine support due to missing rtlIpv6AddressToStringA update HISTORY Fix giampaolo#1329: [AIX] disable some functions based on availability in libperfstat (giampaolo#1349) bump up version, fix some doc issues fix ResourceWarning pre-release fix giampaolo#1447: we weren't use @wrap_exceptions around oneshot() (doh\!) update doc + change git hook location update doc make pre-release checks/install src dist in a venv add new make command to check tar.gz sanity move doc; rephrase it a bit add issue templates for 'bug' and 'enhancement' types remove issue template commited by accident Update issue templates giampaolo#1291: (BACKWARD-INCOMPATIBLE) remove memory_maps() on OSX Restore Win-7 support on GIT master (5.5.1 was OK) (giampaolo#1446) try to fix ntext.h restore previous def fix compiler warning fix compiler warning fix compiler warning fix compiler warning take defs from PH set proper SYSTEM_PROCESS_INFORMATION struct from PH fix compilation warnings giampaolo#1398 / win / cmdline: call NtQueryInformationProcess twice, the first time to get the right buf size (ProcessHacker does this) update doc better print formatting for print scripts fix giampaolo#1442: use python3 as Makefile default appveyor: run print scripts after tests highlight top 6 slowest calls add printerr() and exit() to shared utils module add arg parser for ad script introduce a new scriptsutils.py private module shared by all internal utils + refactor print_access_speed.py script giampaolo#1291 / OSX: mark memory_maps() as deprecated and make it alwats raise AccessDenied OSX memory_maps() - add error handling add script for to benchmark API calls move access_denied script _assert_alive() refactor (linux) refactor fix NetBSD: Process.connections() may return incomplete results if using oneshot() giampaolo#1439 add win tests related to send_signal(CTRL_C_EVENT) giampaolo#1227 fix win test fix win tests fix backslash warnings refactor README a bit #fix 1438: do not return any parent() for PID 0 + update doc ...
I'm able to get around this by Below is the info yielded by parsing lsof of Python's PID.
Online you may see speed concerns regarding |
Hello. I'd rather not, sorry. In general I prefer to refrain from parsing |
I use it. It's the closest I've found to a cross platform way to test shared library dependencies at run time. And I've already written the parser and it's not too ugly 😛. |
I have an example of how to iterate over the memory maps in Austin. It certainly isn't documentation, but perhaps can be of help? https://github.com/P403n1x87/austin/blob/master/src/mac/py_proc.h It would be nice to have memory_maps back into psutil. I don't have easy access to macOS so I can't really try to do this myself at the moment, unfortunately. |
it would be great to be able to rely on this function across all platforms. |
I'm in a situation where I need to get a list of memory mappings- specifically, I need the file-backed mappings (e.g. the various dynamic libraries and other resources, like Java jar files- I need the full path to each on a per-process basis) I currently do this on Linux with I understand that shelling out to It's probably clear, but this is the kind of data I need:
I was hoping that I'm bummed because I switched away from a custom Python solution that married Anyway, just a long-winded way to say "+1", I guess ;) Thanks for the work you all do on this |
I get Exception: OSError(22, 'Invalid argument') on OSX 10.13.4 with Python 2.7
Adding this to the loop makes it go away:
The text was updated successfully, but these errors were encountered: