Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEV: Add ability to run memory benchmarks on macOS #18686

Merged
merged 1 commit into from Jun 16, 2023

Conversation

alugowski
Copy link
Contributor

Some benchmarks such as io_matlab can only run on Linux because:

  • setrlimit may throw on macOS
  • memory total/available stats are read from /proc
  • a subprocess' peak memory usage is a max of polled RSS size in /proc

This PR addresses all three such that benchmarks that measure memory usage also work on macOS.

  • handle a common exception setrlimit raises on macOS
  • memory total/available stats are fetched from psutil
  • subprocess' peak memory usage is read from the rusage struct returned by os.wait4()

The key contribution is that final bit. os.wait4() waits on a process and returns its rusage struct. This struct contains the peak RSS usage by the child process.

os.wait4() is available on Unix systems, including Linux and macOS. So this method adds macOS support. The other upside is that this is a true max, not a max of polling samples.

Add new method of finding peak memory usage using `os.wait4()`.
This also gives the true peak usage, not a max of samples.
@j-bowhay j-bowhay added the DX Everything related to making the experience of working on SciPy more pleasant label Jun 16, 2023
@rgommers rgommers added the Benchmarks Running, verifying or documenting benchmarks for SciPy label Jun 16, 2023
@rgommers rgommers added this to the 1.12.0 milestone Jun 16, 2023
Copy link
Member

@rgommers rgommers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thanks a lot @alugowski!

I tested on macOS, and indeed it fixes the loadmat/savemat memory benchmarks (results report peak_mem / size, so the smaller numbers for larger size make sense):

[ 50.00%] ··· ====== ==================== ====================
              --                     compressed               
              ------ -----------------------------------------
               size          True                False        
              ====== ==================== ====================
                1M        51.658752             51.11808      
               10M        5.9834368            5.9342848      
               100M       1.50798336           1.5015936      
               300M   1.1664861866666667   1.1606971733333333 
              ====== ==================== ====================

[100.00%] ··· io_matlab.MemUsage.track_savemat                                                                         ok
[100.00%] ··· ====== =================== ====================
              --                    compressed               
              ------ ----------------------------------------
               size          True               False        
              ====== =================== ====================
                1M        51.953664           50.085888      
               10M        9.4420992           6.8354048      
               100M       4.42466304          2.49004032     
               300M   4.048431786666667   2.1637802666666666 
              ====== =================== ====================

All LGTM, so in it goes.

@rgommers rgommers merged commit c415f15 into scipy:main Jun 16, 2023
24 checks passed
@alugowski alugowski deleted the macbench branch June 24, 2023 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Benchmarks Running, verifying or documenting benchmarks for SciPy DX Everything related to making the experience of working on SciPy more pleasant
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants