Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Too many open files #2

Closed
GoogleCodeExporter opened this issue May 7, 2015 · 8 comments
Closed

Too many open files #2

GoogleCodeExporter opened this issue May 7, 2015 · 8 comments

Comments

@GoogleCodeExporter
Copy link

What steps will reproduce the problem?
1.  Running a honggfuzz session on OS X 10.6
2.
3.

What is the expected output? What do you see instead?

[WARNING] Couldn't open() 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-79818c280a08c94bf7fd2a54b1
e2567d.tif' file in R/O mode: Too many open files
[ERROR] Couldn't open and map 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-79818c280a08c94bf7fd2a54b1
e2567d.tif' in R/O mode: Too many open files
[WARNING] Couldn't open() 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-93456679a773921d30efafd08f
3ad542.tif' file in R/O mode: Too many open files
[ERROR] Couldn't open and map 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-93456679a773921d30efafd08f
3ad542.tif' in R/O mode: Too many open files
[INFO] Launched new process, pid: 86649, (8/8)
[INFO] Launched new process, pid: 86650, (8/8)
[WARNING] Couldn't open() 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-96292a1bd64fec83bb6cdd2480
a755b6.tif' file in R/O mode: Too many open files
[ERROR] Couldn't open and map 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-96292a1bd64fec83bb6cdd2480
a755b6.tif' in R/O mode: Too many open files
[WARNING] Couldn't open() 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-76c43508fc007bcf5902b6a28e
8055a5.tif' file in R/O mode: Too many open files
[ERROR] Couldn't open and map 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-76c43508fc007bcf5902b6a28e
8055a5.tif' in R/O mode: Too many open files
[INFO] Launched new process, pid: 86651, (8/8)
[INFO] Launched new process, pid: 86652, (8/8)
[WARNING] Couldn't open() 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-84da94dc7e5469f7849b0a7efd
ff5462.tif' file in R/O mode: Too many open files
[ERROR] Couldn't open and map 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-84da94dc7e5469f7849b0a7efd
ff5462.tif' in R/O mode: Too many open files
[WARNING] Couldn't open() 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-93456679a773921d30efafd08f
3ad542.tif' file in R/O mode: Too many open files
[ERROR] Couldn't open and map 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-93456679a773921d30efafd08f
3ad542.tif' in R/O mode: Too many open files


What version of the product are you using? On what operating system?

OS X 10.6.7
honggfuzz, version 0.1 Robert Swiecki <swiecki@google.com>, Copyright 2010 by 
Google Inc. All Rights Reserved.

Please provide any additional information below.

An lsof of the honggfuzz process is attached.


Original issue reported on code.google.com by p...@pizza.org on 11 Apr 2011 at 9:35

Attachments:

@GoogleCodeExporter
Copy link
Author

Hi Felix, would you mind looking into this. 

The problem seems to be caused by rlimit on MacOS (at a first glance). Maybe 
the default, 'soft' limit is low, and we need to set them to a higher level.

The main process is not opening any files and leaving any FDs, the child 
process simply opens and prepares file and then executes the fuzzer process, so 
there should be no problems here as well.

Paul, could you provide the command-line you're using. Maybe it's something 
related to external fuzzers.. dunno.

Original comment by robert.s...@gmail.com on 11 Apr 2011 at 11:57

@GoogleCodeExporter
Copy link
Author

Ah.. missed the lsof output, it seems it's something related to the MacOS crash 
reporting. Gladly assigning to Felix :)

Original comment by robert.s...@gmail.com on 11 Apr 2011 at 11:59

@GoogleCodeExporter
Copy link
Author

I'll look into it

Original comment by groeb...@google.com on 12 Apr 2011 at 9:30

@GoogleCodeExporter
Copy link
Author

Looks like .plist files are left in DiagnosticReports — we'll fix this by 
just copying .crash files from that directory.

Paul, please let us know whether SVN revision 24 solves this, or provide 
honggfuzz and qlmanage command line arguments for further investigation.

Original comment by groeb...@google.com on 14 Apr 2011 at 2:10

@GoogleCodeExporter
Copy link
Author

Hi

So far so good, not seen any issues.

It does look like it is sometimes failing to kill the processes, and I
have to kill -9 them in order to get things running again.

It would be nice if the honggfuzz output gave details of the number of
fuzz cases it has completed.

Original comment by p...@pizza.org on 21 Apr 2011 at 2:58

@GoogleCodeExporter
Copy link
Author

> It does look like it is sometimes failing to kill the processes,
> and I have to kill -9 them in order to get things running again.

Do you use timeouts (-t)?

Original comment by robert.s...@gmail.com on 22 Apr 2011 at 3:43

@GoogleCodeExporter
Copy link
Author

Hm.. there are a few scenarios in which timeouts will not work; 1st, the 
process catches SIGPROF/SIGALRM and sleeps - the last resort solutions, i.e. 
rlimits on CPU time will not help here. 

2nd, your process spawns another process via some strange system means, like it 
asks some other component to do it. For example when you fuzz firefox the first 
process works with timeout, but you cannot spawn 2nd task, it will simply ask 
the first one to open new window.

Well, in any case operating fuzzers requires some knowledge on OS internals.

Original comment by robert.s...@gmail.com on 22 Apr 2011 at 3:45

@GoogleCodeExporter
Copy link
Author

Seems it's been fixed.

Original comment by robert.s...@gmail.com on 2 Jul 2011 at 6:01

  • Changed state: Fixed

anestisb added a commit to anestisb/honggfuzz that referenced this issue Oct 12, 2015
* Replace LOGMSG macros with new xLOG_x macros
* Rename DEBUG flag to DEBUG_BUILD flag since it
conflicts with LOG debug definition

Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
anestisb added a commit to anestisb/honggfuzz that referenced this issue Nov 20, 2015
* debug_enhancements:
  LINUX: Option to save maps
  Indent
  Log refactor updates google#2
  Log refactor updates

Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>

# Conflicts:
#	Makefile
#	cmdline.c
#	files.c
#	files.h
#	linux/ptrace_utils.c
anestisb added a commit to anestisb/honggfuzz that referenced this issue Apr 27, 2016
* debug_enhancements:
  LINUX: Option to save maps
  Indent
  Log refactor updates google#2
  Log refactor updates
  linux: Save proc maps if debug build
  mac: Enable crashdump for debug builds
  Export defined DEBUG flag

Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>

# Conflicts:
#	cmdline.c
#	common.h
anestisb added a commit to anestisb/honggfuzz that referenced this issue Feb 10, 2017
* master: (22 commits)
  Display: erase line before using it
  Display: erase line before using it
  Display: erase line before using it
  libhfuzz: make write function taking const buf
  display: Set _HF_DISPLAY_LINES to 14 (cannot be a macro)
  display: Set _HF_DISPLAY_LINES to 14 (number of display lines before logs)
  sanitizers: w/o -S just set some reasonable values inside the process
  fuzzer: Move fuzzer->origFileName generation to specific functions
  sanitizers: use strdup instead of callod/strcpy
  Set up some basic *san envvars, even with sanitizers support disabled
  display: remote_cmd reformatting
  display: With non -v put the cursor at the bottom of the screen
  display: alternative way of disable scroll regions
  diusplay: experimental no-scroll for logs
  Add pid logging with the initial msg
  files: Warn about empty files
  display: print number of input files
  display: print number of input files
  display: compactize the coverage output google#2
  display: compactize the coverage output
  ...
anestisb added a commit to anestisb/honggfuzz that referenced this issue Feb 10, 2017
* master: (22 commits)
  Display: erase line before using it
  Display: erase line before using it
  Display: erase line before using it
  libhfuzz: make write function taking const buf
  display: Set _HF_DISPLAY_LINES to 14 (cannot be a macro)
  display: Set _HF_DISPLAY_LINES to 14 (number of display lines before logs)
  sanitizers: w/o -S just set some reasonable values inside the process
  fuzzer: Move fuzzer->origFileName generation to specific functions
  sanitizers: use strdup instead of callod/strcpy
  Set up some basic *san envvars, even with sanitizers support disabled
  display: remote_cmd reformatting
  display: With non -v put the cursor at the bottom of the screen
  display: alternative way of disable scroll regions
  diusplay: experimental no-scroll for logs
  Add pid logging with the initial msg
  files: Warn about empty files
  display: print number of input files
  display: print number of input files
  display: compactize the coverage output google#2
  display: compactize the coverage output
  ...
anestisb added a commit to anestisb/honggfuzz that referenced this issue Feb 10, 2017
* master: (22 commits)
  Display: erase line before using it
  Display: erase line before using it
  Display: erase line before using it
  libhfuzz: make write function taking const buf
  display: Set _HF_DISPLAY_LINES to 14 (cannot be a macro)
  display: Set _HF_DISPLAY_LINES to 14 (number of display lines before logs)
  sanitizers: w/o -S just set some reasonable values inside the process
  fuzzer: Move fuzzer->origFileName generation to specific functions
  sanitizers: use strdup instead of callod/strcpy
  Set up some basic *san envvars, even with sanitizers support disabled
  display: remote_cmd reformatting
  display: With non -v put the cursor at the bottom of the screen
  display: alternative way of disable scroll regions
  diusplay: experimental no-scroll for logs
  Add pid logging with the initial msg
  files: Warn about empty files
  display: print number of input files
  display: print number of input files
  display: compactize the coverage output google#2
  display: compactize the coverage output
  ...
anestisb added a commit to anestisb/honggfuzz that referenced this issue Feb 10, 2017
* master: (22 commits)
  Display: erase line before using it
  Display: erase line before using it
  Display: erase line before using it
  libhfuzz: make write function taking const buf
  display: Set _HF_DISPLAY_LINES to 14 (cannot be a macro)
  display: Set _HF_DISPLAY_LINES to 14 (number of display lines before logs)
  sanitizers: w/o -S just set some reasonable values inside the process
  fuzzer: Move fuzzer->origFileName generation to specific functions
  sanitizers: use strdup instead of callod/strcpy
  Set up some basic *san envvars, even with sanitizers support disabled
  display: remote_cmd reformatting
  display: With non -v put the cursor at the bottom of the screen
  display: alternative way of disable scroll regions
  diusplay: experimental no-scroll for logs
  Add pid logging with the initial msg
  files: Warn about empty files
  display: print number of input files
  display: print number of input files
  display: compactize the coverage output google#2
  display: compactize the coverage output
  ...
anestisb added a commit to anestisb/honggfuzz that referenced this issue Feb 10, 2017
* master: (22 commits)
  Display: erase line before using it
  Display: erase line before using it
  Display: erase line before using it
  libhfuzz: make write function taking const buf
  display: Set _HF_DISPLAY_LINES to 14 (cannot be a macro)
  display: Set _HF_DISPLAY_LINES to 14 (number of display lines before logs)
  sanitizers: w/o -S just set some reasonable values inside the process
  fuzzer: Move fuzzer->origFileName generation to specific functions
  sanitizers: use strdup instead of callod/strcpy
  Set up some basic *san envvars, even with sanitizers support disabled
  display: remote_cmd reformatting
  display: With non -v put the cursor at the bottom of the screen
  display: alternative way of disable scroll regions
  diusplay: experimental no-scroll for logs
  Add pid logging with the initial msg
  files: Warn about empty files
  display: print number of input files
  display: print number of input files
  display: compactize the coverage output google#2
  display: compactize the coverage output
  ...
anestisb added a commit to anestisb/honggfuzz that referenced this issue Apr 8, 2017
* master: (99 commits)
  examples/linux_ip: Remove data from tun before using it
  examples/linux_ip: send multiple packets from one input
  examples/ip: udp_lite support
  examples/ip: ip offload
  examples/ip: +sctp
  BTS: Save high PC addresses when kernelOnly is enabled
  ipt: ioctl reset after use
  Linux kernel IP fuzzing with Intel PT/BTS: single buffer
  Linux kernel IP fuzzing with Intel PT/BTS
  Allow to choose between user and kernel perf
  fuzz: fix hfuzz.fileName setting google#2
  fuzz: fix hfuzz.fileName setting
  FIX: Failing when dynamic file was accessed by multiple threads. Now each thread has its own dynamic file according to fuzzNo
  examples/openssl: use SSL_CTX_set_ecdh_auto
  fuzz: a couple of smaller fixes
  fuzz: *bsd doesn't support circleq
  examples/openssl: add SSL_shutdown
  fuzz: use different mutexes for feedback and the dynamic file queue
  linux/arch: modify signal mask for sigtimedwait once only
  Use CIRCLEQ to avoid iterating over TAILQ
  ...
anestisb added a commit to anestisb/honggfuzz that referenced this issue Apr 8, 2017
* master: (99 commits)
  examples/linux_ip: Remove data from tun before using it
  examples/linux_ip: send multiple packets from one input
  examples/ip: udp_lite support
  examples/ip: ip offload
  examples/ip: +sctp
  BTS: Save high PC addresses when kernelOnly is enabled
  ipt: ioctl reset after use
  Linux kernel IP fuzzing with Intel PT/BTS: single buffer
  Linux kernel IP fuzzing with Intel PT/BTS
  Allow to choose between user and kernel perf
  fuzz: fix hfuzz.fileName setting google#2
  fuzz: fix hfuzz.fileName setting
  FIX: Failing when dynamic file was accessed by multiple threads. Now each thread has its own dynamic file according to fuzzNo
  examples/openssl: use SSL_CTX_set_ecdh_auto
  fuzz: a couple of smaller fixes
  fuzz: *bsd doesn't support circleq
  examples/openssl: add SSL_shutdown
  fuzz: use different mutexes for feedback and the dynamic file queue
  linux/arch: modify signal mask for sigtimedwait once only
  Use CIRCLEQ to avoid iterating over TAILQ
  ...

Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>

# Conflicts:
#	files.c
anestisb added a commit to anestisb/honggfuzz that referenced this issue Apr 8, 2017
* master: (99 commits)
  examples/linux_ip: Remove data from tun before using it
  examples/linux_ip: send multiple packets from one input
  examples/ip: udp_lite support
  examples/ip: ip offload
  examples/ip: +sctp
  BTS: Save high PC addresses when kernelOnly is enabled
  ipt: ioctl reset after use
  Linux kernel IP fuzzing with Intel PT/BTS: single buffer
  Linux kernel IP fuzzing with Intel PT/BTS
  Allow to choose between user and kernel perf
  fuzz: fix hfuzz.fileName setting google#2
  fuzz: fix hfuzz.fileName setting
  FIX: Failing when dynamic file was accessed by multiple threads. Now each thread has its own dynamic file according to fuzzNo
  examples/openssl: use SSL_CTX_set_ecdh_auto
  fuzz: a couple of smaller fixes
  fuzz: *bsd doesn't support circleq
  examples/openssl: add SSL_shutdown
  fuzz: use different mutexes for feedback and the dynamic file queue
  linux/arch: modify signal mask for sigtimedwait once only
  Use CIRCLEQ to avoid iterating over TAILQ
  ...
anestisb added a commit to anestisb/honggfuzz that referenced this issue Apr 8, 2017
* master: (99 commits)
  examples/linux_ip: Remove data from tun before using it
  examples/linux_ip: send multiple packets from one input
  examples/ip: udp_lite support
  examples/ip: ip offload
  examples/ip: +sctp
  BTS: Save high PC addresses when kernelOnly is enabled
  ipt: ioctl reset after use
  Linux kernel IP fuzzing with Intel PT/BTS: single buffer
  Linux kernel IP fuzzing with Intel PT/BTS
  Allow to choose between user and kernel perf
  fuzz: fix hfuzz.fileName setting google#2
  fuzz: fix hfuzz.fileName setting
  FIX: Failing when dynamic file was accessed by multiple threads. Now each thread has its own dynamic file according to fuzzNo
  examples/openssl: use SSL_CTX_set_ecdh_auto
  fuzz: a couple of smaller fixes
  fuzz: *bsd doesn't support circleq
  examples/openssl: add SSL_shutdown
  fuzz: use different mutexes for feedback and the dynamic file queue
  linux/arch: modify signal mask for sigtimedwait once only
  Use CIRCLEQ to avoid iterating over TAILQ
  ...
anestisb added a commit to anestisb/honggfuzz that referenced this issue Apr 8, 2017
* master: (99 commits)
  examples/linux_ip: Remove data from tun before using it
  examples/linux_ip: send multiple packets from one input
  examples/ip: udp_lite support
  examples/ip: ip offload
  examples/ip: +sctp
  BTS: Save high PC addresses when kernelOnly is enabled
  ipt: ioctl reset after use
  Linux kernel IP fuzzing with Intel PT/BTS: single buffer
  Linux kernel IP fuzzing with Intel PT/BTS
  Allow to choose between user and kernel perf
  fuzz: fix hfuzz.fileName setting google#2
  fuzz: fix hfuzz.fileName setting
  FIX: Failing when dynamic file was accessed by multiple threads. Now each thread has its own dynamic file according to fuzzNo
  examples/openssl: use SSL_CTX_set_ecdh_auto
  fuzz: a couple of smaller fixes
  fuzz: *bsd doesn't support circleq
  examples/openssl: add SSL_shutdown
  fuzz: use different mutexes for feedback and the dynamic file queue
  linux/arch: modify signal mask for sigtimedwait once only
  Use CIRCLEQ to avoid iterating over TAILQ
  ...

Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>

# Conflicts:
#	files.c
robertswiecki added a commit that referenced this issue Feb 20, 2019
robertswiecki added a commit that referenced this issue Apr 7, 2019
robertswiecki added a commit that referenced this issue May 9, 2019
robertswiecki added a commit that referenced this issue Sep 3, 2019
robertswiecki added a commit that referenced this issue Sep 3, 2019
robertswiecki added a commit that referenced this issue Oct 7, 2019
robertswiecki added a commit that referenced this issue Oct 7, 2019
robertswiecki added a commit that referenced this issue Oct 8, 2019
robertswiecki added a commit that referenced this issue Dec 28, 2019
robertswiecki added a commit that referenced this issue Jan 7, 2020
robertswiecki added a commit that referenced this issue Mar 18, 2020
robertswiecki added a commit that referenced this issue Mar 25, 2020
robertswiecki added a commit that referenced this issue Jan 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant