Please support GNU/kFreeBSD #202

giampaolo · 2014-05-23T14:48:54Z

From matrixhasu on August 29, 2011 01:22:41

Hi,
in debian we have 2 flavors of GNU/kFreeBSD (amd64 and i386) and currently 
psutil is unable to detect those OSes since the check is too restrictive.

With the attached patch I'm able to build psutil and start the test suite, but 
I currently have 2 hangs:

- test_psutil.py - it hangs at "test_cmdline (__main__.TestCase) ..."
- test/test_memory_leaks.py - it hangs at "test_get_pid_list 
(__main__.TestModuleFunctionsLeaks) ..."

I'll see if I'll be able to debug it further in the coming days.

Regards,
Sandro

Original issue: http://code.google.com/p/psutil/issues/detail?id=202

The text was updated successfully, but these errors were encountered:

giampaolo · 2014-05-23T14:48:56Z

From matrixhasu on August 28, 2011 16:23:09

now really attaching the patch

Attachment: psutil_kfreebsd.diff

giampaolo · 2014-05-23T14:48:57Z

From matrixhasu on August 29, 2011 14:24:54

For today I've come up to the attached diff: it seems like several tests are 
hanging due to get_pid_list() . Currently it's implemented using a sysctl, but 
maybe the parameter passed are not the right ones for kFreeBSD. I've asked on 
the support IRC channel and they suggest to use a different approach: 
kvm_getprocs . the the kvm library interface is provided by most (of not all?) 
BSD descendants (f.e. FreeBSD, NetBSD, OpenBSD, DragonFlyBSD, Darwin, 
GNU/kFreeBSD too of course :) and Solaris too) - what do you think about it?

After skipping the hanging tests, there are still a lot of them failing (I'll 
try to nail them down in the coming days, if you don't beat me first ;)):

    ======================================================================
    ERROR: test_connection_fromfd (__main__.TestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "test/test_psutil.py", line 147, in inner
        return fun(self, *args, **kwargs)
      File "test/test_psutil.py", line 1009, in test_connection_fromfd
        for conn in p.get_connections():
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/__init__.py",
 line 402, in get_connections
        return self._platform_impl.get_connections()
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psbsd.py",
 line 198, in get_connections
        return lsof.get_process_connections()
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py",
 line 184, in get_process_connections
        stdout = self.runcmd(cmd)
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py",
 line 253, in runcmd
        raise NotImplementedError(msg)
    NotImplementedError: this functionnality requires lsof command line utility 
to be installed on the system

    ======================================================================
    ERROR: test_get_open_files (__main__.TestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "test/test_psutil.py", line 911, in test_get_open_files
        files = p.get_open_files()
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/__init__.py",
 line 394, in get_open_files
        return self._platform_impl.get_open_files()
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psbsd.py",
 line 191, in get_open_files
        return lsof.get_process_open_files()
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py",
 line 142, in get_process_open_files
        stdout = self.runcmd(cmd)
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py",
 line 253, in runcmd
        raise NotImplementedError(msg)
    NotImplementedError: this functionnality requires lsof command line utility 
to be installed on the system

    ======================================================================
    ERROR: test_get_open_files2 (__main__.TestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "test/test_psutil.py", line 950, in test_get_open_files2
        for path, fd in p.get_open_files():
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/__init__.py",
 line 394, in get_open_files
        return self._platform_impl.get_open_files()
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psbsd.py",
 line 191, in get_open_files
        return lsof.get_process_open_files()
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py",
 line 142, in get_process_open_files
        stdout = self.runcmd(cmd)
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py",
 line 253, in runcmd
        raise NotImplementedError(msg)
    NotImplementedError: this functionnality requires lsof command line utility 
to be installed on the system

    ======================================================================
    ERROR: test_nice (__main__.TestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "test/test_psutil.py", line 851, in test_nice
        p.nice = first_nice
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/__init__.py",
 line 210, in nice
        return self._platform_impl.set_process_nice(value)
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psbsd.py",
 line 90, in wrapper
        raise AccessDenied(self.pid, self._process_name)
    AccessDenied: (pid=3099)

    ======================================================================
    ERROR: test_zombie_process (__main__.TestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "test/test_psutil.py", line 1233, in test_zombie_process
        meth()
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/__init__.py",
 line 402, in get_connections
        return self._platform_impl.get_connections()
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psbsd.py",
 line 198, in get_connections
        return lsof.get_process_connections()
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py",
 line 184, in get_process_connections
        stdout = self.runcmd(cmd)
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py",
 line 253, in runcmd
        raise NotImplementedError(msg)
    NotImplementedError: this functionnality requires lsof command line utility 
to be installed on the system

    ======================================================================
    ERROR: test_get_pids (_posix.PosixSpecificTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/morph/psutil-read-only/test/_posix.py", line 113, in 
test_get_pids
        if BSD: return
    NameError: global name 'BSD' is not defined

    ======================================================================
    ERROR: test_avail_virtmem (_bsd.BSDSpecificTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/morph/psutil-read-only/test/_bsd.py", line 110, in 
test_avail_virtmem
        _, sysctl_avail_virtmem = parse_sysctl_vmtotal(result)
      File "/home/morph/psutil-read-only/test/_bsd.py", line 33, in 
parse_sysctl_vmtotal
        total, active = mobj.groups()
    AttributeError: 'NoneType' object has no attribute 'groups'

    ======================================================================
    ERROR: test_disks (_bsd.BSDSpecificTestCase)...

giampaolo · 2014-05-23T14:48:58Z

From matrixhasu on August 29, 2011 14:24:54

    ...
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/morph/psutil-read-only/test/_bsd.py", line 146, in test_disks
        usage = psutil.disk_usage(part.mountpoint)
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/__init__.py",
 line 667, in disk_usage
        return _psplatform.get_disk_usage(path)
      File 
"/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py",
 line 93, in get_disk_usage
        st = os.statvfs(path)
    OSError: [Errno 2] No such file or directory: '/srv/chroot/sid/home'

    ======================================================================
    ERROR: test_total_virtmem (_bsd.BSDSpecificTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/morph/psutil-read-only/test/_bsd.py", line 89, in 
test_total_virtmem
        sysctl_total_virtmem, _ = parse_sysctl_vmtotal(result)
      File "/home/morph/psutil-read-only/test/_bsd.py", line 33, in 
parse_sysctl_vmtotal
        total, active = mobj.groups()
    AttributeError: 'NoneType' object has no attribute 'groups'

    ======================================================================
    FAIL: test_disk_partitions (__main__.TestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "test/test_psutil.py", line 412, in test_disk_partitions
        self.assertTrue(os.path.isdir(disk.mountpoint))
    AssertionError

    ======================================================================
    FAIL: test_get_num_threads (__main__.TestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "test/test_psutil.py", line 693, in test_get_num_threads
        self.assertEqual(step2, step1 + 1)
    AssertionError: 1 != 2

    ======================================================================
    FAIL: test_get_threads (__main__.TestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "test/test_psutil.py", line 708, in test_get_threads
        self.assertEqual(len(step2), len(step1) + 1)
    AssertionError: 1 != 2

    ======================================================================
    FAIL: test_sys_cpu_times2 (__main__.TestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "test/test_psutil.py", line 331, in test_sys_cpu_times2
        self.fail("difference %s" % difference)
    AssertionError: difference 1.29999999956e-05

    ======================================================================
    FAIL: test_sys_per_cpu_times2 (__main__.TestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "test/test_psutil.py", line 356, in test_sys_per_cpu_times2
        self.fail()
    AssertionError

    ======================================================================
    FAIL: test_BOOT_TIME (_bsd.BSDSpecificTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/morph/psutil-read-only/test/_bsd.py", line 58, in 
test_BOOT_TIME
        self.assertEqual(btime, psutil.BOOT_TIME)
    AssertionError: 1314001403 != 1314001408.0

    ======================================================================
    FAIL: test_process_create_time (_bsd.BSDSpecificTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/morph/psutil-read-only/test/_bsd.py", line 127, in 
test_process_create_time
        self.assertEqual(start_ps, start_psutil)
    AssertionError: 'Mon Aug 29 21:01:47 2011' != 'Mon Aug 29 21:01:48 2011'

    ----------------------------------------------------------------------
    Ran 82 tests in 2.652s

    FAILED (failures=7, errors=9)
    /usr/lib/python2.6/atexit.py:24: RuntimeWarning: 
TestCase.test_get_connections_all was skipped
      func(*targs, **kargs)
    /usr/lib/python2.6/atexit.py:24: RuntimeWarning: 
TestCase.test_get_connections was skipped
      func(*targs, **kargs)
    /usr/lib/python2.6/atexit.py:24: RuntimeWarning: Couldn't run limited user 
tests (super-user privileges are required)
      func(*targs, **kargs)



Additionally, after the test suite completes, the terminal control is not 
returned, and not even Ctrl-C works (Ctrl-Z + kill %x does the trick).

Attachment: psutil_kfreebsd2.diff

giampaolo · 2014-05-23T14:48:58Z

From g.rodola on August 29, 2011 14:53:12

I'll try to reimplement get_pid_list() by using this kvm_getprocs() to fix 
get_pid_list() related failures and let you know once done.

As for test_get_num_threads and test_get_threads failures you might try to add 
a time.sleep(.1) call here: 
https://code.google.com/p/psutil/source/browse/trunk/test/test_psutil.py#698 
...and here: 
https://code.google.com/p/psutil/source/browse/trunk/test/test_psutil.py#712 
Open files/connections failures require lsof to be installed (hopefully I'll 
manage to re-implement that part in C and finally get rid of the stinky lsof 
python parser).

Other BSD-specific tests such as test_avail_virtmem and others just need to be 
adapted a bit or simply be skipped if there's a reliable way to detect we're on 
GNU/kFreeBSD (sys.platform?).

As for test_sys_cpu_times* I honestly have no clue. I have the code in front of 
me and I don't understand why it fails (it shouldn't).

Generally speaking, these failures are not that terrible and suggest that 
adding support for these arcane system I've never heard of might worth the effort. =)

>     st = os.statvfs(path)
> OSError: [Errno 2] No such file or directory: '/srv/chroot/sid/home'

This is interesting. Maybe it worths opening a report against bugs.python.org.

giampaolo · 2014-05-23T14:48:59Z

From matrixhasu on August 30, 2011 15:15:22

awesome! If you need some help or so I can handle you think ml:
debian-bsd@lists.debian.org (it's actually for Debian porting, but
they can help you on the bsd side of psutil).

even with the sleep I got:

    FAIL: test_get_num_threads (__main__.TestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "test/test_psutil.py", line 711, in test_get_num_threads
        self.assertEqual(step2, step1 + 1)
    AssertionError: 1 != 2

    ======================================================================
    FAIL: test_get_threads (__main__.TestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "test/test_psutil.py", line 727, in test_get_threads
        self.assertEqual(len(step2), len(step1) + 1)
    AssertionError: 1 != 2

maybe you can check the existance of lsof and skip the test if not there?

    $ python -c "import sys; print sys.platform"
    gnukfreebsd8

I'll give it a look on the machine, maybe I'll be lucky ;)

yeah, kfreebsd is very interesting since it's the only open source
solution with native zfs support (on linux is available under fuse,
which is kinda... slow...). Oh did I tell you we also have hurd (and
psutil will need work for that too, but we'll get to that later)?

It may just be a problem on with the chroot I was using or so - need
to investigate futher.

giampaolo · 2014-05-23T14:49:00Z

From jlo...@gmail.com on September 16, 2011 11:54:43

Labels: -Type-Defect Type-Enhancement

giampaolo · 2014-05-23T14:49:00Z

From m...@akamit.com on January 29, 2013 06:52:23

    python -c "import sys; print sys.platform"
    openbsd5

        Complete output from command python setup.py egg_info:
        platform openbsd5 is not supported

giampaolo · 2014-05-23T14:49:01Z

From g.rodola on January 29, 2013 06:53:41

Yes, OpenBSD is currently not supported.

giampaolo · 2014-05-23T14:49:02Z

From landry.b...@gmail.com on June 02, 2013 13:12:30

Mozilla imported psutil as part of the buildsystem & mach, so other real bsds 
(and debian/kfreebsd) ought to be supported. What should be the way to go, 
implement a backend for each of them, or try to stash everything in the 
existing bsd backend, with if/switches wrt the family ?

Some work was done a while ago for an older version (0.4.1) in 
https://github.com/jasperla/openbsd-wip/blob/master/sysutils/py-psutil/patches/patch-psutil__psbsd_py
 , i'll try to reapply that on top of master.

giampaolo · 2014-05-23T14:49:02Z

From g.rodola on June 02, 2013 13:24:15

Glad to hear Mozilla adopted psutil! 

In order to add support for a new OS/architecture one should usually file an 
issue on the bug tracker so that we can keep track of the progress and provide a patch.

Alternatively I can give you commit privileges against the Mercurial repository 
and create a separate branch where you can develop and once everything is ready 
merge it back into trunk.

This issue is for GNU/kFreeBSD so if you're willkng to work on adding OpenBSD 
support please file a separate issue and we'll keep discussing there.

Thanks,

- Giampaolo

giampaolo · 2016-11-13T12:14:32Z

From https://en.wikipedia.org/wiki/Debian_GNU/kFreeBSD and https://lists.debian.org/debian-devel-announce/2014/11/msg00005.html:

Debian GNU/kFreeBSD was discontinued as an officially supported platform as of Debian 8.0.

We discussed kfreebsd at length, but are not satisfied that a release with Jessie will be of sufficient quality. We are dropping it as an official release architecture, though we do hope that the porters will be able to make a simultaneous unofficial release.

As such, I'm closing this out as outdated.

mr-c · 2019-09-24T09:51:41Z

There is a new patchset for kfreebsd at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=918884#5

giampaolo closed this as completed Nov 13, 2016

giampaolo added the freebsd label Nov 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Please support GNU/kFreeBSD #202

Please support GNU/kFreeBSD #202

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented Nov 13, 2016

mr-c commented Sep 24, 2019

Please support GNU/kFreeBSD #202

Please support GNU/kFreeBSD #202

Comments

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented Nov 13, 2016

mr-c commented Sep 24, 2019