Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JsonChecker tests broken #1102

Closed
baylesj opened this issue Nov 14, 2019 · 10 comments
Closed

JsonChecker tests broken #1102

baylesj opened this issue Nov 14, 2019 · 10 comments
Labels

Comments

@baylesj
Copy link
Contributor

baylesj commented Nov 14, 2019

After the recent trailing commas change, json checker tests aren't broken.

A fix means making these tests run as part of checkin again, and resolving the actual test failures as well.

baylesj added a commit that referenced this issue Nov 14, 2019
A recent PR broken the JsonChecker tests by adding support for trailing
commas. This didn't end up breaking the build, because those tests
aren't run, except locally and only using CMake.

This patch fixes the tests by adding exclusions for trailing comma
tests, as well as updates Meson to run these tests as part of `ninja
test`.

See issue #1102.
@LocutusOfBorg
Copy link

Hello, I have some test failures on Debian experimental, version 1.9.2
first error is:

Testing EscapeSequenceTest/writeEscapeSequence: FAILED
* Detail of EscapeSequenceTest/writeEscapeSequence test failure:
/<<PKGBUILDDIR>>/src/test_lib_json/main.cpp(3370): expected == result
  Expected: '["\"","\\","\b","\f","\n","\r","\t","\u0278","\ud852\udf62"]
  '
  Actual  : '["\"","\\","\b","\f","\n","\r","\t","ɸ","𤭢"]
  '
101/102 tests passed (1 failure(s))

And happens on arm64, armhf, armel. ppc64el, s390x (and others)
https://buildd.debian.org/status/fetch.php?pkg=libjsoncpp&arch=arm64&ver=1.9.2-2&stamp=1580476809&raw=0
https://buildd.debian.org/status/fetch.php?pkg=libjsoncpp&arch=armhf&ver=1.9.2-2&stamp=1580476979&raw=0
https://buildd.debian.org/status/fetch.php?pkg=libjsoncpp&arch=armel&ver=1.9.2-2&stamp=1580477812&raw=0
https://buildd.debian.org/status/fetch.php?pkg=libjsoncpp&arch=ppc64el&ver=1.9.2-2&stamp=1580476937&raw=0
https://buildd.debian.org/status/fetch.php?pkg=libjsoncpp&arch=s390x&ver=1.9.2-2&stamp=1580476882&raw=0

@LocutusOfBorg
Copy link

The second kind of error is on mips64el:
https://buildd.debian.org/status/fetch.php?pkg=libjsoncpp&arch=mips64el&ver=1.9.2-2&stamp=1580514060&raw=0


2/3 Test #1: jsoncpp_readerwriter ................***Failed    2.54 sec
Unhandled exception:
Exceeded stackLimit in readValue().
Failed to parse input file: 
* Line 1, Column 5
  Missing ',' or ']' in array declaration

TESTING: /<<PKGBUILDDIR>>/test/data/legacy_test_array_03.json FAILED


Failure details:
* Test /<<PKGBUILDDIR>>/test/data/legacy_test_array_03.json
  Difference in input at line 1:
  Expected: '.=[]'
  Actual:   ''


Test results: 57 passed, 1 failed.

3/3 Test #2: jsoncpp_readerwriter_json_checker ...***Failed    3.25 sec
Unhandled exception:
Exceeded stackLimit in readValue().
Failed to parse input file: 
* Line 1, Column 5
  Missing ',' or ']' in array declaration

Failed to parse input file: 
* Line 1, Column 22
  Missing '}' or object member name

Failed to parse input file: 
* Line 1, Column 5
  Syntax error: value, object or array expected.

Failed to parse input file: 
* Line 1, Column 23
  Syntax error: value, object or array expected.

Failed to parse input file: 
* Line 1, Column 16
  Syntax error: value, object or array expected.

Failed to parse input file: 
* Line 1, Column 12
  Missing ',' or ']' in array declaration

Failed to parse input file: 
* Line 1, Column 41
  Missing '}' or object member name

Failed to parse input file: 
* Line 1, Column 2
  '0e+' is not a number.

Failed to parse input file: 
* Line 1, Column 2
  '0e+' is not a number.

Failed to parse input file: 
* Line 1, Column 2
  Missing '}' or object member name

Failed to parse input file: 
* Line 1, Column 2
  '0e' is not a number.

Failed to parse input file: 
* Line 1, Column 2
  Bad escape sequence in string
See Line 2, Column 1 for detail.

Failed to parse input file: 
* Line 1, Column 2
  Bad escape sequence in string
See Line 1, Column 8 for detail.

Failed to parse input file: 
* Line 1, Column 2
  Syntax error: value, object or array expected.
/<<PKGBUILDDIR>>/obj-mips64el-linux-gnuabi64/src/jsontestrunner/jsontestrunner_exe  --json-writer StyledWriter "/<<PKGBUILDDIR>>/test/data/legacy_test_comment_02.json"Failed to parse input file: 
* Line 1, Column 15
  Syntax error: value, object or array expected.

Failed to parse input file: 
* Line 1, Column 26
  Missing ',' or ']' in array declaration

Failed to parse input file: 
* Line 1, Column 26
  Missing ':' after object member name

Failed to parse input file: 
* Line 1, Column 17
  Syntax error: value, object or array expected.

Failed to parse input file: 
* Line 1, Column 18
  Missing ',' or ']' in array declaration

Failed to parse input file: 
* Line 1, Column 18
  Missing ':' after object member name

Failed to parse input file: 
* Line 1, Column 2
  Bad escape sequence in string
See Line 1, Column 31 for detail.

Failed to parse input file: 
* Line 1, Column 2
  Syntax error: value, object or array expected.

Failed to parse input file: 
* Line 1, Column 2
  Bad escape sequence in string
See Line 1, Column 31 for detail.

Failed to parse input file: 
* Line 1, Column 28
  Missing ',' or '}' in object declaration

Failed to parse input file: 
* Line 1, Column 24
  Syntax error: value, object or array expected.

Failed to parse input file: 
* Line 1, Column 26
  Missing ',' or '}' in object declaration

Failed to parse input file: 
* Line 1, Column 1
  A valid JSON document must be either an array or an object value.


Failure details:
* Test /<<PKGBUILDDIR>>/test/data/legacy_test_basic_05.json
  Difference in rewrite at line 1:
  Expected: '.=null'
  Actual:   ''


Test results: 57 passed, 1 failed.


33% tests passed, 2 tests failed out of 3

Total Test time (real) =   3.27 sec

The following tests FAILED:
	  1 - jsoncpp_readerwriter (Failed)
	  2 - jsoncpp_readerwriter_json_checker (Failed)

Probably similar to the failure in mipsel
https://buildd.debian.org/status/fetch.php?pkg=libjsoncpp&arch=mipsel&ver=1.9.2-2&stamp=1580514054&raw=0

I'm uploading in Debian the patch for this issue you proposed, I hope it fixes also the above kind of failures!

@LocutusOfBorg
Copy link

nope, looks like the patch didn't work for my failures:
http://debomatic-arm64.debian.net/distribution#experimental/libjsoncpp/1.9.2-4/buildlog
at least on arm64 it didn't make differences

* Detail of EscapeSequenceTest/writeEscapeSequence test failure:
/<<PKGBUILDDIR>>/src/test_lib_json/main.cpp(3370): expected == result
  Expected: '["\"","\\","\b","\f","\n","\r","\t","\u0278","\ud852\udf62"]
  '
  Actual  : '["\"","\\","\b","\f","\n","\r","\t","ɸ","𤭢"]
  '
101/102 tests passed (1 failure(s))

Do you have any clue?

@ggardet
Copy link

ggardet commented Feb 11, 2020

And happens on arm64, armhf, armel. ppc64el, s390x (and others)

It seems to be all non-x86 archs.

@ggardet
Copy link

ggardet commented Feb 12, 2020

It should be fixed by #1120

@LocutusOfBorg
Copy link

After applying #1103 and #1120 I got no failures left, except from when I run tests with parallel!=1

if you do make test -j 8 for example you can get lots of failures...

@baylesj
Copy link
Contributor Author

baylesj commented Feb 21, 2020

I'm working on actually getting #1103 closed out, hopefully will be soon

baylesj added a commit that referenced this issue Feb 21, 2020
A recent PR broken the JsonChecker tests by adding support for trailing
commas. This didn't end up breaking the build, because those tests
aren't run, except locally and only using CMake.

This patch fixes the tests by adding exclusions for trailing comma
tests, as well as updates Meson to run these tests as part of `ninja
test`.

See issue #1102.
@cdunn2001
Copy link
Contributor

@LocutusOfBorg, it's definitely important for tests to pass in parallel. We'll try to get to that for the upcoming release.

@cdunn2001 cdunn2001 added the bug label Apr 24, 2020
cdunn2001 pushed a commit that referenced this issue Apr 24, 2020
A recent PR broken the JsonChecker tests by adding support for trailing
commas. This didn't end up breaking the build, because those tests
aren't run, except locally and only using CMake.

This patch fixes the tests by adding exclusions for trailing comma
tests, as well as updates Meson to run these tests as part of `ninja
test`.

See issue #1102.
cdunn2001 pushed a commit that referenced this issue Apr 24, 2020
A recent PR broken the JsonChecker tests by adding support for trailing
commas. This didn't end up breaking the build, because those tests
aren't run, except locally and only using CMake.

This patch fixes the tests by adding exclusions for trailing comma
tests, as well as updates Meson to run these tests as part of `ninja
test`.

See issue #1102.
@cdunn2001
Copy link
Contributor

Fixed for meson. No idea about cmake, but when we reorganize the test-suite we might address that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants