_parse_container does not receive proper data to process

[cucdat]

Hi,

I am trying to use this tool to parse some blf files but there is one of them throwing me the Parsing Error.
I am tried to fix it by updating the source as well as relog it with different CANalyzer Version but it cannot work so I post it here to ask for your support.

---

Issue Description: The data given to _parse_container does not start with the proper signature (LOBJ), as a consequence, the _parse_data raised the BLFParseError.

Environment:

• Python 3.8 64bit
• python-can: dev branch

---

If you have any idea to figure out the root cause of this, kindly let me know. I will appreciate it so much.
I can share with you the blf for debugging via email in case you need it.

[christiansandberg]

It would be interesting to know which obj_type came before the exception. There is a problem with blf files in that different object types are padded differently. One person had issues with FlexRay data in the log.

[christiansandberg]

It’s also normal that containers do not start with LOBJ. Objects can be split up in two containers since containers usually have a fixed size.

[cucdat]

It is not the data of container actually, it is the data that will transfer to _parse_data. I will check what is the object type comes before this exception then get back to you later.

Usually, this tool works perfectly with other blf files, even up to hours, but only there are some files coming from specific PC/Vector Application have this issue.

[cucdat]

Hi,

Thanks for your hint!
I tried to trace back to the root cause that made the unpacking exception.
First, the object is just a normal CAN 2 Message object, so I do not think it will make the exception here

The real root cause seems to be the buffer size required for Struct.unpack_from is bigger than the buffer size that we defined.
Actual Size: 131106
Required: 131120

By the way, the uncompressed when I unpacked the Container is 131072 = exact 128kb.

Shall we change this buffer size a little bit bigger to handle this case and how should we change it? (Where should we put the fixed code to not affect other functions)
It seems like we should check the buffer size and resize it to a proper one before unpacking
https://docs.python.org/3.0/library/struct.html#struct.unpack_from

[christiansandberg]

I think I know what the problem is. I did some optimizations recently in develop where I skipped checking if the object is split up in the next container and relies on struct raising an error if that’s the case. With CanMessage2 however we only read the first part of it so it could succeed even if the rest of the object is missing.

So I think the latest released version should work for you. I’ll fix it in the coming days.

[cucdat]

That would be perfect.
Just left me a line when you fixed it, I will be happy to be the first one to test it.

By the way, can you explain why don't we need the tail of the CAN2? Doesn't it contain data?

Thank you so much.

[christiansandberg]

The rest of the data is not anything we use in our message class.

[duracell013]

Hello,

I've only been using this module (version 3.3.2) for a day, but I'm also facing some BLFParseError with some files due to header issues. I've modified the blf.py to print the header here.

In the console output, it seems like the file position is incorrect at the start of the iteration. In the exemple below, I think the position should be one byte earlier as we can clearly see the end of "LOBJ".

In another example, the position is "late" and I only see the end of "LOBJ".

For info, I'm using genuine BLF files generated by CANanlyzer.
Please, let me know if you need an example file.

Thank you for your help!
Stéphan

[cucdat]

Hi Duracell,
Use the dev branch might fix this issue.
I used to encounter this with 3.3.2 but not anymore with the dev branch.

[christiansandberg]

This makes me slightly more confused. Any example log files would be great! You can email me at christiansandberg at me dot com if you don’t want to share them publicly.

[christiansandberg]

@duracell013, I just realized a bug fix for CAN-FD messages has not been released yet so you can try develop instead.

[duracell013]

@christiansandberg I've emailed you a sample file. I have tested the develop branch without success (though I'm not 100% sure the module installation from the ZIP file was correct).

Thank you for your help!

[christiansandberg]

I've made a fix that seems to work.

pip install https://github.com/hardbyte/python-can/archive/bugfix/763-fix-blf-reader.zip

[duracell013]

@christiansandberg Thank you so much for looking into it so quickly! That seems to work perfectly!

[cucdat]

Sorry for my late feedback.

I have tested this on the blf files that I had problems with and it worked without any Parsing issues.
Also tested on other working files to check if it is compatible or not and found no issues.

The solution seems to work perfectly.

Thank you a lot for your support @christiansandberg!