Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: invalid Wave: truncated #41

Open
X-Raym opened this Issue Jun 3, 2018 · 8 comments

Comments

Projects
None yet
3 participants
@X-Raym
Copy link
Contributor

X-Raym commented Jun 3, 2018

Hi !

I have some file which display this in the console: invalid Wave: truncated

The metadata displays just fine in other application I use, like REAPER.

If other apps can display this right, maybe BWFMetaEdit is a bit too strict ?
Or, how could this be fixed right on the file ?

Thanks for your support !

@dericed

This comment has been minimized.

Copy link
Collaborator

dericed commented Jun 3, 2018

Letting you know that your wav file is malformed isn't simply being strict but being informative. Although other applications don't have these warnings in scope, this type of validation issue is certainly in scope of BWF MetaEdit. Your wav file likely has a size declaration in its header that is larger than the actual file, so you file is likely a partial or truncated copy of another file and there is audio at the end that is missing.

@X-Raym

This comment has been minimized.

Copy link
Contributor Author

X-Raym commented Jun 3, 2018

@dericed Oh yes it is informative, but not having, the possibility to see/process information, is restrictive (considering that other apps can show them). It is not just a informative warning, I can't for eg dump the avaible metadata.

Apart from re-render the file, is there anyway this could be fixed ?

@JeromeMartinez

This comment has been minimized.

Copy link
Member

JeromeMartinez commented Jun 3, 2018

the possibility to see/process information, is restrictive

With other tools you thought that all is OK, this is not the case. It is already a good thing to have a messenger saying something went wrong, don't you think? Then you can use other tools for digging in the file.

I can't for eg dump the avaible metadata.

Which "metadata" do you need? the exact position and value of the issue? True, it is not in BWF MetaEdit because it was not expected in the design, when we started the project. It could be implemented if there is enough need about it. If you look for a byte per byte dump of the file right now, you can check MediaConch, open the file for checking then click on "MediaTrace" button for the file. You still need some knowledge about WAV structure as the exact issue is not showed (it is only a dump).

Apart from re-render the file, is there anyway this could be fixed ?

It is not possible to invent missing data. No need to re-render the file, finding a good (before the truncation) copy of the file should be enough, I personally expect that the rendering tool correctly stored the file. If you lost this file, the only solution is to re-render the file, as the data is lost.

@X-Raym

This comment has been minimized.

Copy link
Contributor Author

X-Raym commented Jun 3, 2018

@JeromeMartinez

I thought there might be a tool which could define new end for the file (at the last readable data ) and update metadata accordingly.

Anyway, thanks for your infos and expertise.

For sure having good file right from the start would be the better way but sometimes there us just no way to get back.

Note that I use BWFMediaEdit CLI version mostly so other non CLI version might not work in my case.
I have coded a lua script which is able to analyse wav file header and display metadata so I might go with this solution for the dump. But for metadata injection, it seems that only rerender can work.

@X-Raym

This comment has been minimized.

Copy link
Contributor Author

X-Raym commented Jun 3, 2018

@JeromeMartinez Seems that MediaTrace is able to dump xml file with the metadata even with broken file.

This is from a broken file:

<?xml version="1.0" encoding="UTF-8"?>
<MediaInfo
    xmlns="https://mediaarea.net/mediainfo"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="https://mediaarea.net/mediainfo https://mediaarea.net/mediainfo/mediainfo_2_0.xsd"
    version="2.0">
<creatingLibrary version="18.05" url="https://mediaarea.net/MediaInfo">MediaInfoLib</creatingLibrary>
<media ref="G:\REAPER Scripts\RPP-Tests\BWF\AutoRaceProStocks_6043_73 - Copie.wav">
<track type="Général">
<AudioCount>1</AudioCount>
<FileExtension>wav</FileExtension>
<Format>Wave</Format>
<FileSize>5978664</FileSize>
<Duration>20.426</Duration>
<OverallBitRate_Mode>CBR</OverallBitRate_Mode>
<OverallBitRate>2341590</OverallBitRate>
<StreamSize>95784</StreamSize>
<Title>Series 6000 Extension I</Title>
<Title_More>All Sound Effects &amp; Music are Copyright Sound Ideas - All Rights Reserved</Title_More>
<Album_Performer>Series 6000 Extension I</Album_Performer>
<Track>Series 6000 Extension I</Track>
<Track_More>All Sound Effects &amp; Music are Copyright Sound Ideas - All Rights Reserved</Track_More>
<Track_Position>0</Track_Position>
<Original_Album>www.sound-ideas.com</Original_Album>
<Performer>Sound Ideas</Performer>
<Director>Sound Ideas</Director>
<Producer>Sound Ideas</Producer>
<Genre>AUTO, RACE, PRO STOCKS / AUTO, RACE, PRO STOCKS</Genre>
<Description>AUTO, RACE, PRO STOCKS - SINGLE STOCK CAR:  PASS BY</Description>
<Original_Released_Date>2016</Original_Released_Date>
<Recorded_Date>2016 / 2016-05-20</Recorded_Date>
<Encoded_Date>2007-05-03 14:09:13</Encoded_Date>
<File_Created_Date>UTC 2018-06-03 17:25:25.256</File_Created_Date>
<File_Created_Date_Local>2018-06-03 19:25:25.256</File_Created_Date_Local>
<File_Modified_Date>UTC 2018-06-03 17:23:20.174</File_Modified_Date>
<File_Modified_Date_Local>2018-06-03 19:23:20.174</File_Modified_Date_Local>
<Encoded_Application>Soundminer</Encoded_Application>
<OriginalSourceForm_Name>Series 6000 Extension I</OriginalSourceForm_Name>
<Owner>Series 6000 Extension I</Owner>
<Copyright>2016 Sound Ideas (Series 6000 Extension I) </Copyright>
<Cover>Yes</Cover>
<Cover_Type>Cover (front)</Cover_Type>
<Cover_Mime>image/jpeg</Cover_Mime>
<Comment>AUTO, RACE, PRO STOCKS - SINGLE STOCK CAR:  PASS BY / AUTO, RACE, PRO STOCKS - SINGLE STOCK CAR:  PASS BY</Comment>
<extra>
<Producer_Reference>Series 6000 Extension I</Producer_Reference>
</extra>
</track>
<track type="Audio">
<Format>PCM</Format>
<Format_Settings_Endianness>Little</Format_Settings_Endianness>
<Format_Settings_Sign>Signed</Format_Settings_Sign>
<CodecID>1</CodecID>
<Duration>20.426</Duration>
<BitRate_Mode>CBR</BitRate_Mode>
<BitRate>2304000</BitRate>
<Channels>2</Channels>
<SamplingRate>48000</SamplingRate>
<SamplingCount>980448</SamplingCount>
<BitDepth>24</BitDepth>
<Delay>0.000</Delay>
<Delay_Source>Container (bext)</Delay_Source>
<StreamSize>5882880</StreamSize>
<StreamSize_Proportion>0.98398</StreamSize_Proportion>
</track>
</media>
</MediaInfo>

As you can see there is some metadata infos there. This is pretty nice, but doesn't solve the data injection problem (which was what I wanted to do in a first place). For MediaConch, the file appears to be valid. So it is just BWFMetaEdit who can't process it (extracting or injecting metadata).

@X-Raym

This comment has been minimized.

Copy link
Contributor Author

X-Raym commented Jun 4, 2018

Just an extra post for clarification. My case was the following:

  1. I have a file, which appears as truncated in both GUI and CLI version of BWFMetaEdit
  2. I wanted to edit the file Metadata (fields like description, Originator etc) with the BWFMetaEdit CLI version.

But it breaks.

I tried with other BWF injection solution like wavefile and this works. It sure didn't reinvent the missing data (if there is missing data, maybe it is just wrong metadata infos), but it made BWFMetaData readable and writable again in BWFMetaEdit.

So, a way to fix metadata sounds possible, having that with BWFMetaEdit CLI version would be nice !

@JeromeMartinez

This comment has been minimized.

Copy link
Member

JeromeMartinez commented Jun 4, 2018

I thought there might be a tool which could define new end for the file (at the last readable data ) and update metadata accordingly.

Usually (important: not the case here, it is just an explanation about the idea behind the current way it is working), a truncated file is due to bad transfer, data is missing.
It is doable to "define new end for the file", BUT it would just be hiding your issue, this is like a lie to whoever is interested in knowing if the file is wealthy. We don't want to create tools helping in hiding issues, as we are interested in fixing the issue (by getting a wealthy file), not hiding it. It is better IMO to store the file as is in that case, rather than hiding the issue by changing the header (the file is missing content, and you accept it, OK so no need to hide that)

Seems that MediaTrace is able to dump xml file with the metadata even with broken file.

You provided a MediaInfo XML, not a MediaTrace one.

Checking your file, issue is not the one @dericed thought about, the message was misleading (reason MediaConch ignored it, as it shows an issue only if the file is shorter than expected. Note that MediaConch does not have a WAV file conformance checker, the message "valid" should be "not applicable", something we need to fix in MediaConch).

MediaTrace output:

000000 WAVE (12 bytes)
000000  Header (12 bytes)
000000   Name:                                 RIFF
000004   Size:                                 5978656 (0x005B3A20)
000008   Real Name:                            WAVE
5B2CDE  _PMX (3402 bytes)
5B2CDE   Header (8 bytes)
5B2CDE    Name:                                _PMX
5B2CE2    Size:                                3658 (0x00000E4A)
5B2CE6   Unknown:                              (3394 bytes)
(...)

You can see that there is a problem: WAVE chunk says that end is at 0x5B3A28 (8 bytes of start of header + 0x005B3A20), _PMX chunck says that end is at 0x5B3A30 (offset 0x5B2CE6 + size of 0xE4A), 8 bytes later --> This is incoherent, the file is buggy.

In that case, currently BWF MetaEdit prefers to not edit the file, as there is a risk that something went wrong somewhere (this is not a file transfer issue, as the WAVE chunk header is coherent with file size), and it is more development work to edit other fields (e.g. bext chunk) while keeping such bug during rewrite (in order not to hide the issue). @dericed, a quick and dirty fix could be to ignore such bug in a file and "correct" it silently (1 line patch), but I am personally not in favor of such solution because I don't like to hide the issue in the file.

Checking the file, the XMP content is with spaces at the end, so low risk that some data was lost (just missing 8 spaces?), so we could add a check of such buggy file and fix it, this is just not yet implemented.

In other words, your file is a scenario of non conforming file we chose to avoid in our development, as it was not requested by the sponsor. Support of such buggy file (with different scenarios in order to be sure that BWF MetaEdit does what the user expects, whatever is the user expectation) could be added on request.

summary: I see 2 issues:

  • support of buggy sub-chunk sizes in a chunk. I would assign this ticket to this issue. I change the title to "Support edition of some non conforming files" and add a "need sponsor" label. @dericed don't hesitate to change my change if you dislike it much.
  • support of an optional fixer for buggy sub-chunk sizes in a chunk.

I tried with other BWF injection solution like wavefile and this works.

If it fits your needs, that's fine. BWF MetaEdit is just (currently) not adapted to your need of edition of such buggy file, we may have just have different priorities (my priority is to not miss a bug in a file, and to not hide it silently during rewrite, as wavefile does).

@X-Raym

This comment has been minimized.

Copy link
Contributor Author

X-Raym commented Jun 4, 2018

@JeromeMartinez Thanks for this detailed report and your time on this !

We now know what could be done to fix such issue 👍

Cheers !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.