Memory leak by parsing special mp3 file

[Newan]

On parsing special single file, the node enviremont used more then 1,4 gb of ram and exit with:

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

@Borewit I sended you one file by mail for test, if u or another want more example please contact me.

[Borewit]

Will give an update on this annoying problem.

This typically happens on corrupted MP3 files without any ID3v2 header / Xing header.

It will then scan the MPEG frame., and jump from one frame to the next frame.
But if these frames contain garbage, it needs to re-sync, to look for the next valid frame.

Re-syncing is done by searching byte by byte.

The asynchronous nature of JavaScript, in combination with promise wrapped recursion loop, will result on a zero-wised (corrupt MP3 file ;-) file of 3 MB, in a recursion depth of ~3 million.
This in combination with the actual file read, is apparently extremely expensive for Node.

I don't see a a nice solution, I am pretty sure it not caused by some kind of mistake.

Two solutions I can think of are:

1. Give up after x amount of unsynced bytes
   • will break other stuff, cause will not complete find all information (metadata, format info)
2. Cache bigger chunks of data:
   • ~10 faster, probably keeps memory under control
   • Ugly code, looks so overcoplicated for something stupid to read until a byte sequence has been found

Related/cause: strtok3 issue #5

[Newan]

i analysed this issues in the last few days and found an handfull mp3 in my collection with this error. But there are files they are not corrupt for other programms like vlc. windows explorer can read the id-tags also. if you want i can send you some oof this files too.

Now it is for me not important which solution you pick, important for me is that one of the solution stop this memory overrun.

thanks for your support!

[Borewit]

Related:

• Building a promise chain recursively in javascript - memory considerations
• How do I stop memory leaks with recursive javascript promises?

[Newan]

o.k it seems that is not so easy to fix. my workaround is to make an optional parameter like execution time. if in mp3 is in this time read everythink is o.k if an mp3 need more time, it will be skippt or genrate an error like, excution time exceeded.

so we have no problem to read many mp3 files in a loop

[Borewit]

Good to hear you found a workaround. You could also use unsynced, counting the number of unsychronized bytes in MpegParser.ts

There is indeed not an easy fix. I don't want to go in that direction for a permanent solution.
In the future, I would like to make data corruption part of the result.

I solved the issue, solution only has some side effects, once that is fixed I will make it available.

[Borewit]

Addressed in v0.8.2

@Newan & @mycoboco : would be great if you could give it a try.

[Newan]

For me the first try is very great !

Fast and sure! If i found some errors i will reopen this issues or make a new one!

Thanks, great work!