Too many open files when trying to edit multiple files

[vinitkumar]

Hi,

Thanks for creating this tool. I wanted to update all the templates of my project like this:

find . -name "*.html" -type f -exec djhtml -i -t2 {} +, But it gives me this error:

But I get this error:

usage: djhtml [-h] [-i] [-q] [-t N] [-o filename] [filenames ...]
djhtml: error: argument filenames: can't open './demo_cms/template/content_news.html': [Errno 24] Too many open files: './demo_cms/template/content_news.html'

Please let me know if I am doing something wrong. I don't want to edit each file by hand.

[JaapJoris]

Wow, you must have a lot of HTML files! I think the problem is the + option you're giving to find. According to man find:

-exec command {} +
          This  variant  of  the -exec action runs the specified command on the selected files, but the command line is built by
          appending each selected file name at the end; the total number of invocations of the command will be  much  less  than
          the  number  of  matched  files.   The command line is built in much the same way that xargs builds its command lines.

I think simply removing the + from your command should fix this:

find . -name "*.html" -type f -exec djhtml -i -t2 {}

Another way is to use the command xargs that's mentioned in the above documentation:

find . -name "*.html" -type f | xargs djhtml -i -t2

I think xargs will automatically limit the amount of filenames passed to djhtml. But I'm not sure. However, you can limit it to for instance 10 filenames at a time:

find . -name "*.html" -type f | xargs -n10 djhtml -i -t2

Could you try any of these solutions and check if they work?

[JaapJoris]

However, the real error is that Python's argparse.FileType("r") will try to immediately open all files passed on the command line, which is totally unnecessary because they are only handled one at a time. I'll replace this with more optimized file handling in the next commit!

[vinitkumar]

@JaapJoris Thanks a lot for the quick reply. I also tried xargs and it didn't work for me. I see you fixed this with bb72181, is this also released to pypi, already?

[JaapJoris]

No, not yet. Until then, you can install the latest "bleeding-edge" version with:

pip install git+https://github.com/rtts/djhtml

Please let me know if that works. Otherwise, I will create a release especially for you 🤗

[vinitkumar]

Awesome. Thanks :)

[vinitkumar]

Please let me know if that works. Otherwise, I will create a release especially for you 🤗

Please take your time with the release. This will work for me for now. Just post in this thread whenever you do the release, so I will get the email about it.

Thanks a tonne again for all the efforts ☕

[JaapJoris]

Hi there, I would like to inform you that your suggested improvement has been included in the latest release of DjHTML, both available here and on PyPi. Thanks again for reporting this issue 🙏