"Input is not proper UTF-8" message

[Lawrence58]

Hello,
I analyzed a video file against the Memoriav Video Files Recommendations policy. According to Media Info the file I tested is YUV, 10 bit, 4:2:2. According to my understanding this should have passed the "Format is uncompressed 4:2:2 10 bit (and)" sub-policy but in the results I get a big red X with the following explanation.

Validation generated an internal error: Entity: line 121: parser error : Input is not proper UTF-8, indicate encoding ! Bytes: 0xA9 0x54 0x49 0x4D ame">FIEL</d></b><d o="7786" n="Data">(18 bytes)</d></b><b o="7804" n="Text" i="

What does this mean? Complete Media Info output follows.
Thank you.

Full Media Info report
Complete name : V:\Video\Pres Masters and Proxies\1271_PRM.mov
Format : MPEG-4
Format profile : QuickTime
Codec ID : qt 2005.03 (qt )
File size : 1.23 GiB
Duration : 46 s 880 ms
Overall bit rate : 225 Mb/s
Encoded date : UTC 2017-04-22 20:19:55
Tagged date : UTC 2017-04-22 20:19:57
Writing library : Apple QuickTime
©TIM : 00;00;00;00
©TSC : 30000
©TSZ : 1001

Video
ID : 1
Format : YUV
Codec ID : v210
Codec ID/Hint : AJA Video Systems Xena
Duration : 46 s 880 ms
Bit rate mode : Constant
Bit rate : 224 Mb/s
Width : 720 pixels
Height : 486 pixels
Display aspect ratio : 4:3
Frame rate mode : Constant
Frame rate : 29.970 (30000/1001) FPS
Standard : NTSC
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 10 bits
Scan type : Interlaced
Scan type, store method : Interleaved fields
Scan order : Bottom Field First
Compression mode : Lossless
Bits/(Pixel*Frame) : 21.333
Stream size : 1.22 GiB (99%)
Language : English
Encoded date : UTC 2017-04-22 20:19:55
Tagged date : UTC 2017-04-22 20:19:55
Color primaries : BT.601 NTSC
Transfer characteristics : BT.709
Matrix coefficients : BT.601

Audio
ID : 2
Format : PCM
Format settings, Endianness : Little
Format settings, Sign : Signed
Codec ID : sowt
Duration : 46 s 880 ms
Bit rate mode : Constant
Bit rate : 1 536 kb/s
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 kHz
Bit depth : 16 bits
Stream size : 8.58 MiB (1%)
Language : English
Encoded date : UTC 2017-04-22 20:19:55
Tagged date : UTC 2017-04-22 20:19:55

Other
ID : 3
Type : Time code
Format : QuickTime TC
Duration : 46 s 880 ms
Time code of first frame : 00:00:00;00
Time code, striped : Yes
Language : English
Encoded date : UTC 2017-04-22 20:19:55
Tagged date : UTC 2017-04-22 20:19:55

[JeromeMartinez]

The message shows that the error is not due to your file, but is due to a problem in MediaConch during parsing of the file.
We got a similar issue with some files but in old versions of MediaConch.
Which version of MediaConch do you use? (menu "Help", then "About"; current version is 17.11).
Please click also (when you have version 17.11) on "force analyze" in the "status" column in order to clear the cache in case the cash contains an old parsing.

[Lawrence58]

Sorry. Yes. I am running 17.11 GUI (on a PC). Just installed this morning. The only Status column I see is in the Results but I do not see a force analyze option associated with it. I have tried closing and re-launching the application and reanalyzing. I got the same results.

[Lawrence58]

Okay. I see the force analyze option. I used it and this time I get a failure X for both implementation (not valid) and policy. In case it matters, the file being analyzed is on a network share.

[Lawrence58]

I uninstalled Media Conch using unist.exe then reinstalled it. When I launched the application the GUI had already loaded the file that I analyzed in the previous installation with the same failed results. Where is the cache that made that possible?

[JeromeMartinez]

Cache is in C:\Users\%USERNAME%\AppData\Local\MediaConch.
I'll try to reproduce the issue with files I have, else I'll ask for your file.

[Lawrence58]

The cache isn't the issue nor do I think the file I am testing is. I have done the following 5 times with the same results so either it is the application, the implementation, or me.
Media Conch v17.11
Windows 7 Pro v6.1

• Uninstall the application using uninst.exe

• Delete the cache folder

• Install and launch the application

• Select the Memoriav Video Files Recommendations policy and a file to test (the same file represented above) then press Check Files.

• "File added successfully" appears in the Check Files section. A row appears in the Results section with the filename. The Status column says "In queue". Analysis does not begin. "Force analyze" is not available. After 10 minutes nothing changes.

• Press the Check Files button again. Analysis does not begin. A message appears in the Check Files section "Please Select One or More Files".

• Close and relaunch the application.

• The Results section displays the previously selected file. The Status column says "Analyzed." Implementation and Policy both have red "X" in them. Pressing "Force analyze" returns the same results.

[JeromeMartinez]

The cache isn't the issue nor do I think the file I am testing is.

This is what I said yesterday.
Please on hold, as said I'll try to reproduce the issue with files I have, else I'll ask for your file.

[Lawrence58]

Thank you, Jerome. I'm just trying to provide you with as much info as possible.

[JeromeMartinez]

Would be actually faster for me (no need to find other files) if you can provide the file. Is it possible for you to provide the file? (privately if the file can not be publicly shared)

[Lawrence58]

I can share the file privately. Do you have a way I can send it to you? It is 1.2 GB.

[JeromeMartinez]

please drop an email at info@mediaarea.net

[dericed]

mediainfo is placing a copyright symbol directly into an attribute which seems to break parsing in some (maybe not all) environments. The copyright symbol could be escaped such as info1="©TIM"

[Lawrence58]

Jerome - This is to confirm that I sent an email to info@mediaarea.net.

[JeromeMartinez]

@Lawrence58 I answered with credentials when I get the email, did you receive the answer?
Sorry for the delay, some personal issues preventing me to check the issue, but still on my priority list.

[JeromeMartinez]

I apologize for the delay. I confirm I can reproduce the issue and that it is a MediaConch bug.
Working on it.

[JeromeMartinez]

Not seen before because we map most of such metadata to MediaInfo fields, not the case with some of them (the ones for time code in metadata, supporting them is on my ToDo-list) and we were badly storing them with wrong character encoding.
Fixed, Windows Snapshot, Mac Snapshot.

You need to reprocess files ("force analyze" button).

[bturkus]

Thanks Jerome! You're the best!

[dericed]

I suggest to reopen as this error persists:

ffmpeg -f lavfi -i testsrc -vframes 1 -y test.mov
mediaconch -mt --Force test.mov | xml fo

provides

-:414.45: Input is not proper UTF-8, indicate encoding !
Bytes: 0xA9 0x73 0x77 0x72: Bytes: 0xA9 0x73 0x77 0x72

            <data offset="3492" name="Name">?swr</data>
                                            ^

[JeromeMartinez]

@g-maxime I tested the GUI and CLI, GUI looks fine but CLI looks to have an issue.