New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Known_peers.json grows larger that allowed file/socket limit in Linux #1611

Open
fr1t2 opened this Issue Jan 5, 2019 · 0 comments

Comments

Projects
None yet
1 participant
@fr1t2
Copy link
Contributor

fr1t2 commented Jan 5, 2019

Describe the bug
The ~/.qrl/data/known_peers.json file grows quite large and upon starting the qrl node using start_qrl with a large known_peers.json file returns this error:

2019-01-05 18:22:05,668|1.1.11 python|unsynced|MainThread | CRITICAL : [TWISTED] Unhandled Error
Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python3.5/site-packages/twisted/internet/base.py", line 428, in fireEvent
  File "/home/ubuntu/.local/lib/python3.5/site-packages/twisted/internet/defer.py", line 321, in addCallback
  File "/home/ubuntu/.local/lib/python3.5/site-packages/twisted/internet/defer.py", line 310, in addCallbacks
  File "/home/ubuntu/.local/lib/python3.5/site-packages/twisted/internet/defer.py", line 653, in _runCallbacks
    
--- <exception caught here> ---
  File "/home/ubuntu/.local/lib/python3.5/site-packages/twisted/internet/base.py", line 441, in _continueFiring
  File "/home/ubuntu/.local/lib/python3.5/site-packages/twisted/internet/base.py", line 1238, in _reallyStartRunning
  File "/home/ubuntu/.local/lib/python3.5/site-packages/twisted/internet/posixbase.py", line 298, in _handleSignals
  File "/home/ubuntu/.local/lib/python3.5/site-packages/twisted/internet/posixbase.py", line 200, in __init__
  File "/home/ubuntu/.local/lib/python3.5/site-packages/twisted/internet/posixbase.py", line 133, in __init__
    
builtins.OSError: [Errno 24] Too many open files

This error points to the open files limit default settings in most unix OS's. This is not a critical issue, and the node will continue syncing even with the error. Ubuntu 16.04 has as a default open files limit of 1024

ubuntu@server:~$ ulimit -a
open files                      (-n) 1024

You can see that the num_known_peers has grown past this limit.

ubuntu@server:~$ qrl state
info {
  num_known_peers: 1122
  }

To Reproduce
Steps to reproduce the behavior:

  1. Sync a local node so the ~/.qrl/data/known_peers.json file is filed in
  2. Issue the qrl state command and find the num_known_peers
  3. Issue ulimit -n to check the open files limit
  4. Stop the node
  5. Restart the node with start_qrl
  6. If limit is smaller than the num_known_peers the error will be raised when you start the node

Expected behavior
Respect the OS limit and split the operation into smaller sized chunks to avoid hitting the warning.

Details:

  • OS: Ubuntu 16.04.4 LTS
  • QRL version: 1.1.11 python

Additional context
This is not critical and only presents a warning when starting the node. Setting the limit to something larger than the known peers removes the warning.

Changing the default in a local session with ulimit -n 1200 removed the warning upon starting the node.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment