Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows / speeup: dynamically load libraries on startup and never again #1422

Merged
merged 48 commits into from
Feb 21, 2019

Conversation

giampaolo
Copy link
Owner

@giampaolo giampaolo commented Feb 20, 2019

This is kinda big. It encapsulates the logic for dynamically loading libraries at runtime instead of on each function call. Undocumented / private Windows APIs are now loaded globally on import. In doing so I refactored a lot of stuff and didn't test this on Win XP, which I am tempted to keep alive. I also removed MinGW related cruft, so this probably deserves a major version bump.

EDIT: benchmark script including timings and the APIs involved:

API                                      BEFORE     AFTER   DIFF
psutil.boot_time()                       0.018      0.008   +76%
psutil.cpu_count(logical=False)          0.055      0.033   +50%
psutil.cpu_count(logical=True)           0.041      0.025   +48%
psutil.cpu_stats()                       0.379      0.157   +82%
psutil.cpu_times(percpu=True)            0.341      0.066   +135%
psutil.disk_io_counters()                0.945      0.921   +2%
psutil.users()                           5.671      5.466   +3%
proc.ionice()                            0.056      0.047   +17%
proc.ionice(0)                           0.061      0.051   +18%
proc.open_files()   (too slow)
proc.connections()  (too slow)

Benchmark script:

from __future__ import print_function
import sys
import timeit
import psutil

ITERATIONS = 10000
apis = [
    'psutil.boot_time()',
    'psutil.disk_io_counters()',
    'psutil.cpu_count(logical=False)',
    'psutil.cpu_count(logical=True)',
    'psutil.cpu_times(percpu=True)',
    'psutil.users()',
    'psutil.cpu_stats()',
    # 'psutil.net_connections(kind="inet4")',  # slow
    'proc.ionice()',
    'proc.ionice(0)',
    # 'proc.open_files()',  # slow
]
apis = sorted(set(apis))
setup = "import psutil; proc = psutil.Process()"

def main():
    if not psutil.WINDOWS:
        sys.exit("Windows only")
    for api in apis:
        elapsed = timeit.timeit(api, setup=setup, number=ITERATIONS)
        print("%-40s %.3f" % (api, elapsed))


if __name__ == '__main__':
    main()

@giampaolo
Copy link
Owner Author

Attaching benchmarks in the first message.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant