Problem when loading several videos

[EderSantana]

Hi,
First, thanks for imageio, really good software here!!!

I'm loading videos with imageio and saving results to an hdf5 file.
I'm wrapping everything with an with as statement.
My previous problem was with the limit of open files as in #79.
That was solved by using this repo instead of pip. But now I'm getting:

Exception in thread Thread-21:
Traceback (most recent call last):
  File "/home/eders/anaconda/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/home/eders/python/imageio/imageio/plugins/ffmpeg.py", line 822, in run
    self._lines = limit_lines(self._lines)
TypeError: 'NoneType' object is not callable

It runs ok up to a hundred of videos. Does anybody have any idea of what it might be?

Thanks!

[almarklein]

Did you perhaps in some way force a re-import of imageio? limit_lines is defined in the same module where it's used, and having such functions be None is something I've seen when reloading a module.

Although if the error occurs after already having read so many videos, that's different. Another reason can be that the module is being cleaned up because Python is shutting down. It might be that the processing is done, and the thread processes the last lines during shutdown (note that the error does not take place in the main thread). Is the result of what you want to do correct? If it is the case then you can considere this a spurious error message (and should be easy to fix).

[EderSantana]

Did you perhaps in some way force a re-import of imageio?

I'm not doing that. I actually have just one single threaded script. And it quits after a few messages like those. It didn't solve the problem.

Well, I can do manual calls to ffmpeg and use subprocess to wait it to execute. I just thought that doing that with imageio would be a better looking solution.

Let me know if you have other ideas about what it might be.

[almarklein]

Sorry, I think I was a bit unclear. Forget my comment about the re-import. My best bet right now is that your script actually works as expected, exits, Python shuts down, the StreamCatcher thread of the ffmpeg plugin wants to parse some last received lines, but fails because the module is already being cleaned up.

The error that is shown is not produced from the main thread. So my question is: does your script actually produce the expected results?

[almarklein]

ping @EderSantana

[EderSantana]

I went back to a shell script to solve my problem. If this is not happening for anybody else, we can just close this. Thank you!

[mizu-ki]

This is happening to me. Could you fix it?

[almarklein]

@M1ZUKI What exactly is happening in your case? Is there a traceback?

Also, could you verify whether the video is actually correctly written to disk? If so, it's likely just a problem during shutdown, not related to the writing of the video itself.

[mizu-ki]

Exception in thread Thread-4:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/home/mizuki/tensorflow/local/lib/python2.7/site-packages/imageio/plugins/ffmpeg.py", line 823, in run
self._lines = limit_lines(self._lines)
TypeError: 'NoneType' object is not callable

[mizu-ki]

@almarklein
Thank you very much for your reply.

I guess it's a problem during shutdown because my codes looks actually working until shutdown.
How can I handle this problem?

[almarklein]

Great! Thanks for verifying that. This is good news, I suppose, as we only need to make sure the message does not get printed. I will have a closer look soon. Don't hesitate to ping me if I don't get back to this within a few days.

[mizu-ki]

Actually, I would like to let you know that files opened by "with statement" does not close/save well when the message shows.