[0.13.71]: test suite is failing #97
Comments
|
Also compare #102 |
|
I would like to close it based on not supporting automake builds anymore. In the cmake variant the test-binaries are downloaded in the configure-step. Since it is hard to create problematic zip-files on-the-fly, those "bad zip" files are needed to get a proper idea if things do work. |
Are you aware fact that most of the build systems used by differebt distributions are intentionally cut off from public network? |
|
@kloczek - there is no tarball generation anymore in the sense that automake would package a list of files. The intention of the current setup is to allow tests to be skipped if the binaries are not available. Stopping the download with -DZZIP_TESTCVE=OFF should be possible. |
That is OK however I think that those test units are not about testing donloading sometning so IMO still it would be good to add those test files to source tree and sist tarr ball. |
|
BTW just tested 0.13.72 and looks like there in new version when cmake is used there is no test suite + cd zziplib-0.13.72
+ /usr/bin/make -O -j48 V=1 VERBOSE=1 -C x86_64-redhat-linux-gnu test ARGS=--output-on-failure
make: Nothing to be done for 'test'.[tkloczko@barrel x86_64-redhat-linux-gnu]$ cmake -L
CMake Warning:
No source or binary directory provided. Both will be assumed to be the
same as the current working directory, but note that this warning will
become a fatal error in future CMake releases.
CMake Error: The source directory "/home/tkloczko/rpmbuild/BUILD/zziplib-0.13.72/x86_64-redhat-linux-gnu" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
-- Cache values
BUILD_SHARED_LIBS:BOOL=ON
BUILD_STATIC_LIBS:BOOL=OFF
BUILD_TESTS:BOOL=ON
CMAKE_BUILD_TYPE:STRING=RelWithDebInfo
CMAKE_INSTALL_PREFIX:PATH=/usr
MKZIP:FILEPATH=/usr/bin/zip
MSVC_STATIC_RUNTIME:BOOL=ON
UNZIP:FILEPATH=/usr/bin/unzip
XMLTO:FILEPATH=/usr/bin/xmlto
ZZIPBINS:BOOL=ON
ZZIPDOCS:BOOL=ON
ZZIPFSEEKO:BOOL=ON
ZZIPMMAPPED:BOOL=ON
ZZIPSDL:BOOL=ON
ZZIPTEST:BOOL=ON
ZZIPWRAP:BOOL=ON
ZZIP_COMPAT:BOOL=ON
ZZIP_HTMLSITE:BOOL=OFF
ZZIP_HTMPAGES:BOOL=OFF
ZZIP_INSTALL_ACLOCAL:BOOL=ON
ZZIP_INSTALL_BINS:BOOL=ON
ZZIP_LIBTOOL:BOOL=ON
ZZIP_MANPAGES:BOOL=ON
ZZIP_PKGCONFIG:BOOL=ON
ZZIP_TESTCVE:BOOL=ON |
|
You can disable tests from the default ZZIPTEST:BOOL=ON and there is target "check" as I used it to have. |
|
The binaries will not go into the source tree. I am doing devops consulting as a day job where the existance of a binary repository can be assumed. The current project allows to pre-download the CVE zips and to copy them to the internal network if the build processes require that depth of checks. For packagers of opensource distributions it will work to wrap the pre-downloaded cve zips into a tarball as a kind of patch object. I don't see a need to support more than that. In general it would be great if the cve zips can be generated through code - at the moment I don't know how they came to be originally, and I am just including them for regression testing, so that the problems do not come up again. |
Hmm .. but why you are breaking semantics of cmake/ctest and provide test suite under DIY target?🤔 BTW: do you have any plans to make new release soon? |
|
Any update about use standard ctest instead custom target? 🤔 |
======================================================================
ERROR: test_59800_infozipdir_CVE_2017_5980 (main.ZZipTest)
run info-zip dir test0.zip
Traceback (most recent call last):
File "zziptests.py", line 2010, in test_59800_infozipdir_CVE_2017_5980
run = shell("cd {tmpdir} && {exe} -o {filename}".format(**locals()),
File "zziptests.py", line 102, in shell
raise subprocess.CalledProcessError(run.returncode, sh_command, output)
subprocess.CalledProcessError: Command 'cd tmp.test_59800 && unzip -o 00154-zziplib-nullptr-zzip_mem_entry_new' returned non-zero exit status 12.
======================================================================
ERROR: test_65430 (main.ZZipTest)
info unzip -l $(CVE).zip
Traceback (most recent call last):
File "zziptests.py", line 3042, in test_65430
run = shell("cd {tmpdir} && {exe} -o {filename}".format(**locals()),
File "zziptests.py", line 102, in shell
raise subprocess.CalledProcessError(run.returncode, sh_command, output)
subprocess.CalledProcessError: Command 'cd tmp.test_65430 && unzip -o c006-unknown-add-main' returned non-zero exit status 12.
======================================================================
ERROR: test_65440 (main.ZZipTest)
info unzip -l $(CVE).zip
Traceback (most recent call last):
File "zziptests.py", line 3082, in test_65440
run = shell("cd {tmpdir} && {exe} -o {filename}".format(**locals()),
File "zziptests.py", line 102, in shell
raise subprocess.CalledProcessError(run.returncode, sh_command, output)
subprocess.CalledProcessError: Command 'cd tmp.test_65440 && unzip -o c008-main-unknown-de' returned non-zero exit status 12.
======================================================================
ERROR: test_65470 (main.ZZipTest)
info unzip -l $(CVE).zip
Traceback (most recent call last):
File "zziptests.py", line 3293, in test_65470
run = shell("cd {tmpdir} && {exe} -o {filename}".format(**locals()),
File "zziptests.py", line 102, in shell
raise subprocess.CalledProcessError(run.returncode, sh_command, output)
subprocess.CalledProcessError: Command 'cd tmp.test_65470 && unzip -o 003-unknow-def-zip' returned non-zero exit status 12.
======================================================================
ERROR: test_65480 (main.ZZipTest)
info unzip -l $(CVE).zip
Traceback (most recent call last):
File "zziptests.py", line 3413, in test_65480
run = shell("cd {tmpdir} && {exe} -o {filename}".format(**locals()),
File "zziptests.py", line 102, in shell
raise subprocess.CalledProcessError(run.returncode, sh_command, output)
subprocess.CalledProcessError: Command 'cd tmp.test_65480 && unzip -o 002-mem-leaks-zip' returned non-zero exit status 12.
======================================================================
FAIL: test_59754_zzipdir_zap_CVE_2017_5975 (main.ZZipTest)
run unzzip -l $(CVE_2017_5975).zip
Traceback (most recent call last):
File "zziptests.py", line 1860, in test_59754_zzipdir_zap_CVE_2017_5975
self.assertErrorMessage(run.errors, 0)
File "zziptests.py", line 190, in assertErrorMessage
self.assertIn(': ' + os.strerror(errno), errors)
AssertionError: ': Success' not found in 'tmp.test_59754/00151-zziplib-heapoverflow-__zzip_get64: No such file or directory\n'
======================================================================
FAIL: test_59781_zzipdir_big_CVE_2017_5978 (main.ZZipTest)
run info-zip -l $(CVE_2017_5978).zip
Traceback (most recent call last):
File "zziptests.py", line 1463, in test_59781_zzipdir_big_CVE_2017_5978
self.assertIn(" stored (null)", run.output)
AssertionError: ' stored (null)' not found in ''
======================================================================
FAIL: test_59804_zzipdir_zap_CVE_2017_5980 (main.ZZipTest)
run unzzip -l $(CVE_2017_5980).zip
Traceback (most recent call last):
File "zziptests.py", line 2089, in test_59804_zzipdir_zap_CVE_2017_5980
self.assertErrorMessage(run.errors, 0)
File "zziptests.py", line 190, in assertErrorMessage
self.assertIn(': ' + os.strerror(errno), errors)
AssertionError: ': Success' not found in 'tmp.test_59804/00154-zziplib-nullptr-zzip_mem_entry_new: No such file or directory\n'
======================================================================
FAIL: test_59814_zzipdir_zap_CVE_2017_5981 (main.ZZipTest)
run unzzip-zap -l $(CVE_2017_5981).zip
Traceback (most recent call last):
File "zziptests.py", line 2201, in test_59814_zzipdir_zap_CVE_2017_5981
self.assertErrorMessage(run.errors, 0)
File "zziptests.py", line 190, in assertErrorMessage
self.assertIn(': ' + os.strerror(errno), errors)
AssertionError: ': Success' not found in 'tmp.test_59814/00161-zziplib-assertionfailure-seeko_C: No such file or directory\n'
======================================================================
FAIL: test_63014 (main.ZZipTest)
unzzip-zap -l $(CVE).zip
Traceback (most recent call last):
File "zziptests.py", line 2311, in test_63014
self.assertErrorMessage(run.errors, 0)
File "zziptests.py", line 190, in assertErrorMessage
self.assertIn(': ' + os.strerror(errno), errors)
AssertionError: ': Success' not found in 'tmp.test_63014/zziplib_0-13-67_zzdir_invalid-memory-access_main.zip: No such file or directory\n'
======================================================================
FAIL: test_63114 (main.ZZipTest)
unzzip-zap -l $(CVE).zip
Traceback (most recent call last):
File "zziptests.py", line 2433, in test_63114
self.assertErrorMessage(run.errors, 0)
File "zziptests.py", line 190, in assertErrorMessage
self.assertIn(': ' + os.strerror(errno), errors)
AssertionError: ': Success' not found in 'tmp.test_63114/zziplib_0-13-67_unzzip_infinite-loop_unzzip_cat_file.zip: No such file or directory\n'
======================================================================
FAIL: test_63814 (main.ZZipTest)
unzzip-zap -l $(CVE).zip
Traceback (most recent call last):
File "zziptests.py", line 2547, in test_63814
self.assertErrorMessage(run.errors, 0)
File "zziptests.py", line 190, in assertErrorMessage
self.assertIn(': ' + os.strerror(errno), errors)
AssertionError: ': Success' not found in 'tmp.test_63814/zziplib_0-13-67_unzip-mem_buffer-access-with-incorrect-length-value_zzip_disk_fread.zip: No such file or directory\n'
======================================================================
FAIL: test_64844 (main.ZZipTest)
unzzip-zap -l $(CVE).zip
Traceback (most recent call last):
File "zziptests.py", line 2657, in test_64844
self.assertErrorMessage(run.errors, 0)
File "zziptests.py", line 190, in assertErrorMessage
self.assertIn(': ' + os.strerror(errno), errors)
AssertionError: ': Success' not found in 'tmp.test_64844/zziplib_0-13-67_zzdir_memory-alignment-errors___zzip_fetch_disk_trailer.zip: No such file or directory\n'
======================================================================
FAIL: test_65404 (main.ZZipTest)
unzzip-zap -l $(CVE).zip
Traceback (most recent call last):
File "zziptests.py", line 2780, in test_65404
self.assertErrorMessage(run.errors, 0)
File "zziptests.py", line 190, in assertErrorMessage
self.assertIn(': ' + os.strerror(errno), errors)
AssertionError: ': Success' not found in 'tmp.test_65404/zziplib_0-13-67_unzip-mem_memory-alignment-errors_zzip_disk_findfirst.zip: No such file or directory\n'
======================================================================
FAIL: test_65414 (main.ZZipTest)
unzzip-zap -l $(CVE).zip
Traceback (most recent call last):
File "zziptests.py", line 2890, in test_65414
self.assertErrorMessage(run.errors, 0)
File "zziptests.py", line 190, in assertErrorMessage
self.assertIn(': ' + os.strerror(errno), errors)
AssertionError: ': Success' not found in 'tmp.test_65414/zziplib_0-13-67_unzzip_memory-aligment-errors___zzip_fetch_disk_trailer.zip: No such file or directory\n'
======================================================================
FAIL: test_65424 (main.ZZipTest)
unzzip-zap -l $(CVE).zip
Traceback (most recent call last):
File "zziptests.py", line 3007, in test_65424
self.assertErrorMessage(run.errors, 0)
File "zziptests.py", line 190, in assertErrorMessage
self.assertIn(': ' + os.strerror(errno), errors)
AssertionError: ': Success' not found in 'tmp.test_65424/zziplib_0-13-67_unzip-mem_memory-alignment-errors_zzip_disk_findfirst_64.zip: No such file or directory\n'
======================================================================
FAIL: test_65474 (main.ZZipTest)
unzzip-zap -l $(CVE).zip
Traceback (most recent call last):
File "zziptests.py", line 3376, in test_65474
self.assertErrorMessage(run.errors, 0)
File "zziptests.py", line 190, in assertErrorMessage
self.assertIn(': ' + os.strerror(errno), errors)
AssertionError: ': Success' not found in 'tmp.test_65474/003-unknow-def-zip: No such file or directory\n'
Ran 209 tests in 16.927s
FAILED (failures=12, errors=6, skipped=8)
make[2]: *** [Makefile:701: tests] Error 1
make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/zziplib-0.13.71/test'
make[1]: *** [Makefile:551: check-am] Error 2
make: *** [Makefile:498: check-recursive] Error 1
The text was updated successfully, but these errors were encountered: