Too many open files

[base698]

Adding lots of stuff to the queue results in this error:

**Test case:

from queuelib import FifoDiskQueue, PriorityQueue as DiskPriorityQueue
for i in xrange(0, 100000):
   pq.push(str(i), i)

Traceback (most recent call last):
File "fit.py", line 595, in
File "fit.py", line 557, in init
File "/usr/local/lib/python2.7/site-packages/queuelib/pqueue.py", line 33, in push
File "fit.py", line 541, in
File "/usr/local/lib/python2.7/site-packages/queuelib/queue.py", line 47, in init
File "/usr/local/lib/python2.7/site-packages/queuelib/queue.py", line 67, in _openchunk
IOError: [Errno 24] Too many open files: 'queue-dir-113/q00000'

[dangra]

hi @base698, you can increase the limit of open files (ulimit) or use the sqlite backend https://github.com/scrapy/queuelib/blob/master/queuelib/queue.py#L181