libtsk13 4.4.2: crash in the dfvfs test suite (from Debian Project)

[eribertomota]

Hi,

This is a bug report from Debian[1]. The libtsk 4.4.2 is causing a Floating point exception when compiling dfvfs test suite[2]. It is a 'serious' bug in Debian and TSK will be remove from 'testing' release until the problem is solved. Several packages that depends of TSK will be removed too.

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=873374
[2] https://github.com/log2timeline/dfvfs

The original bug says:

https://tests.reproducible-builds.org/debian/history/dfvfs.html
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/dfvfs.html

...
   debian/rules override_dh_auto_test
make[1]: Entering directory '/build/1st/dfvfs-20170723'
python run_tests.py
testIterateVolumes (volume.vshadow_volume_system.VShadowVolumeSystemTest)
Test the iterate volumes functionality. ... ok
testIterateVolumes (volume.tsk_volume_system.TSKVolumeSystemTest)
Test the iterate volumes functionality. ... debian/rules:23: recipe for target 'override_dh_auto_test' failed
make[1]: *** [override_dh_auto_test] Floating point exception


Program terminated with signal SIGFPE, Arithmetic exception.
#0  0x00007fa2ac0f606a in dos_load_prim_table (test=1 '\001', 
    vs=0x55c56dcae270) at dos.c:821
821     dos.c: No such file or directory.
(gdb) bt
#0  0x00007fa2ac0f606a in dos_load_prim_table (test=1 '\001', 
    vs=0x55c56dcae270) at dos.c:821
#1  tsk_vs_dos_open (img_info=0x7fa2a322c070, offset=0, 
    test=test@entry=1 '\001') at dos.c:1064
#2  0x00007fa2ac0f3161 in tsk_vs_open (img_info=0x7fa2a322c070, offset=0, 
    type=<optimized out>) at mm_open.c:56
#3  0x00007fa2ac3bf41e in Volume_Info_Con (self=0x55c56dcbe2f0, 
    img=<optimized out>, type=<optimized out>, offset=<optimized out>)
    at tsk3.c:659
#4  0x00007fa2ac3b3ade in pyVolume_Info_init (self=self@entry=0x7fa2a31b4c70, 
    args=args@entry=(<TSKFileSystemImage(_file_object=<OSFile(_size=67108864, _is_cached=True, _resolver_context=<Context(_file_object_cache=<ObjectsCache(_maximum_number_of_cached_values=128, _values={u'type: OS, location: /tmp/dfvfs-20170723/test_data/vsstest.qcow2\n': <ObjectsCacheValue(_reference_count=1, vfs_object=<OSFile(_size=751104, _is_cached=True, _resolver_context=<...>, _is_open=True, _file_object=<file at remote 0x7fa2a3174d20>) at remote 0x7fa2a31b4550>) at remote 0x7fa2a31b4590>, u'type: OS, location: /tmp/dfvfs-20170723/test_data/bdetogo.raw\n': <ObjectsCacheValue(_reference_count=1, vfs_object=<...>) at remote 0x7fa2a31b4bd0>}) at remote 0x7fa2a3fbad90>, _file_system_cache=<ObjectsCache(_maximum_number_of_cached_values=16, _values={}) at remote 0x7fa2a3fbad50>) at remote 0x7fa2a3fba390>, _is_open=True, _file_object=<file at remote 0x7fa2a3174db0>) at remote 0x7fa2a31b4890>) at remote 0x7fa2a31994b0>,), kwds=kwds@entry=0x0)
    at pytsk3.c:20444
#5  0x000055c56b688bf4 in type_call.lto_priv () at ../Objects/typeobject.c:765
#6  0x000055c56b683163 in PyObject_Call () at ../Objects/abstract.c:2547
...


Works after downgrading libtsk13 to 4.4.0-5

Thanks a lot in advance.

Regards,

Eriberto

[bcarrier]

Looks like vs->block_size is set to 0. It should have been set to something else. Do you know what image format the test image is? Is the test image available?

[eribertomota]

2017-09-14 14:03 GMT-03:00 Brian Carrier <notifications@github.com>:

Looks like vs->block_size is set to 0. It should have been set to something else. Do you know what image format the test image is? Is the test image available?

Thanks for your reply, The problem is occurring when building dfvfs using libtsk13 as dependency, not when using the system.

[bcarrier]

But it looks like it is a test suite from dfvfs that is run during build that is exposing the problem. I'm trying to better understand what the test is that caused this error to occur so that I can more easily recreate it.

[eribertomota]

2017-09-14 14:53 GMT-03:00 Brian Carrier <notifications@github.com>:

But it looks like it is a test suite from dfvfs that is run during build that is exposing the problem.

Yeap!

I'm trying to better understand what the test is that caused this error to occur so that I can more easily recreate it.

Can you compile dfvfs with libtsk?

[bcarrier]

@joachimmetz, do you know which test image is having problems in dfvfs and where I can find it?

[joachimmetz]

@eribertomota first of all this report should have better details:

when compiling dfvfs test suite

dfvfs is Python and is not "compiled" (https://en.wikipedia.org/wiki/Compiled_language)

what version of pytsk is used?

@bcarrier Looks like this test is failing:
testIterateVolumes (volume.tsk_volume_system.TSKVolumeSystemTest)

which maps to https://github.com/log2timeline/dfvfs/blob/master/tests/volume/tsk_volume_system.py#L42

Which uses this test image https://github.com/log2timeline/dfvfs/blob/master/test_data/tsk_volume_system.raw

[eribertomota]

Hi Joachim, 2017-09-14 17:36 GMT-03:00 Joachim Metz <notifications@github.com>:

@eribertomota <https://github.com/eribertomota> first of all this report should have better details: when compiling dfvfs test suite dfvfs is Python and is not "compiled" (https://en.wikipedia.org/ wiki/Compiled_language)

Sorry for this mistake. I was say building, not compiling. It is packaged in Debian and the command 'debuild' will start all building process.

what version of pytsk is used?

20170802 Thanks. @bcarrier <https://github.com/bcarrier> Looks like this test is failing:

…

testIterateVolumes (volume.tsk_volume_system.TSKVolumeSystemTest) which maps to https://github.com/log2timeline/dfvfs/blob/ master/tests/volume/tsk_volume_system.py#L42 Which uses this test image https://github.com/log2timeline/dfvfs/blob/ master/test_data/tsk_volume_system.raw

[joachimmetz]

@eribertomota was pytsk recompiled after updating libtsk?

[joachimmetz]

Sorry for this mistake. I was say building, not compiling. It is packaged
in Debian and the command 'debuild' will start all building process.

No worries.

Kali seems to have a similar issue: log2timeline/plaso#1404

[bcarrier]

I can't recreate this with the image with the TSK tools and all code paths that I see in TSK's usage set sector_size to non-zero. Must be something in pyTSK that is allowing it to be non-zero. But, I added checks to not open file or volume systems if it is set to 0.

[eribertomota]

Thanks @joachimmetz and @bcarrier

I can see a light. On Debian, I rebuilt pytsk and, now, dfvfs builds too. The pytsk is maintained on Debian by @hillu (Hilko Bengen).

I will do more tests in some hours.

Thanks.

[joachimmetz]

@eribertomota there is (likely) an ABI dependency between pytsk and libtsk.

[eribertomota]

Hi all,

I can confirm that rebuilding pytsk in Debian (same version, a rebuild only) solves the problem.

I think this bug can be closed. Thanks a lot for your help!

Regards,

Eriberto

[hillu]

A quick comment from the maintainer of the pytsk package in Debian:

I had to modify the build script for two reasons: (1) libtalloc is provided as a Debian package, so embedding it is a bad idea. (2) Fetching sleuthkit sources via git at build time is against policy. So I initially had pytsk3.so linked against both shared libraries. As of pytsk/20170802-2 (uploaded today), pytsk3.so links libtsk statically (and carries "Built-Using" information that should trigger rebuilds on new Sleuthkit versions) because of the issue described here.

[joachimmetz]

@hillu thx