Fix out of bounds error

[shawwn]

Closes #19, #246, #269, #281.

iter_chunks now uses np.linspace to generate indices. In addition
to fixing the out of bounds error, this also ensures each chunk
generates a uniform number of indices.

[shawwn]

Hmm, it looks like this PR doesn't solve all the cases. I'll keep investigating.

[coveralls]

Coverage increased (+0.1%) to 54.185% when pulling 2f569d0 on shawwn:fixes/indices-out-of-range into f63a985 on Zulko:master.

[mbeacom]

Is it intended to add + 1 to nchunks on L70 and add + 1 again in pospos ?

[shawwn]

Yes. Since we're generating endpoint=True, we need to pass in one more than the number of chunks. I.e. if nchunks is 1, we tell linspace to return two points (a start and an end).

[shawwn]

As far as I can tell, that last commit fixes all remaining errors on my machine. I've been making heavy use of moviepy to generate spectrograms of a large number of videos, and I kept running into the IndexError problem. After these commits, I haven't seen any errors after processing ~30 videos.

[coveralls]

Coverage increased (+0.1%) to 54.183% when pulling aec97bf on shawwn:fixes/indices-out-of-range into f63a985 on Zulko:master.

[shawwn]

Ok, I've successfully stress-tested this PR by processing several hour-long videos without encountering any errors at all. I believe this PR is ready to merge.

The test suite seems to pass, but on my machine python3 setup.py test results in six errors like:

E           OSError: [Errno 32] Broken pipe
E           
E           MoviePy error: FFMPEG encountered the following error while writing file /tmp/crop1.webm:
E           
E            crop1TEMP_MPY_wvf_snd.ogg: No such file or directory

moviepy/video/io/ffmpeg_writer.py:178: OSError

I'm not quite sure why that's happening, but I don't think it's related to this PR.

If you have any questions or concerns, let me know.

[tburrows13]

@shawwn I don't think those errors are relevant. As long as it passes on Travis, I think it should be ok.

[shawwn]

Ok, cool. In that case, this PR is ready to merge.

[shawwn]

After heavy usage, I still haven't run into any out of bounds errors (or any other problems).

[stonebig]

merging soon ?

[kevinmcalear]

Any idea when this can be merged? This would fix an issue that I'm currently dealing with.

[shawwn]

@mbeacom I still haven't run into any out of bounds errors, and @stonebig @kevinmcalear have requested this be merged. Since this PR is now four months old, can it be accepted?

[shawwn]

(Or @Zulko.)

[shawwn]

@Zulko @mbeacom, could this PR be merged? Several users have asked for this, and after heavy usage I still haven't run into any out of bounds errors or any other problems.

I suspect this PR has languished because it changes the bounds checking. In general, it's good to be suspicious of doing this carelessly. But the math is correct now; the reason users have been running into errors is that the math was off in certain corner cases.

Another reason is probably that this seems like a magical, unexplained change in moviepy/audio/io/readers.py:

             try:
                  result = np.zeros((len(tt),self.nchannels))
                  indices = frames - self.buffer_startframe
 +                if len(self.buffer) < self.buffersize // 2:
 +                    indices = indices - (self.buffersize // 2 - len(self.buffer) + 1)
                  result[in_time] = self.buffer[indices]
                  return result

Again, it's good to be hesitant of changes like that, but FWIW I arrived at this solution by running moviepy on a video that breaks, then sat in pdb.trace() and looked at the actual data. The above snippet fixes the out of bounds errors.

If other users are running into out of bounds errors with moviepy and just need a fix, I suppose you can apply the above changes manually. But it'd be nice to get this merged.

[Zulko]

Sorry for the delay. I believe most maintainers are too busy right now to give this much thought so we'll have to trust you on this (I hope that was the right decision :) ).