SMW 2.5.0-alpha failed phpunit tests in MW 1.28.0

[lloonngg]

Setup and configuration

• SMW version: 2.5.0-alpha (fedc048)
• MW version: 1.28.0
• PHP version: Using HHVM 3.15.3 (5.6.99-hhvm), php5.6
• DB (MySQL etc.): SMWSQLStore3, mysql [10.1.21-MariaDB-1~xenial]

Issue

I'm testing the latest dev-master SMW 2.5.0-alpha using the phpunit testing and got the following stack trace with 6 failures. I also got similar failures with SMW 2.4.6 so I tried 2.5.0-alpha thinking that some of the failures would be fixed.

#1, #2, and #3 are due to a familiar hhvm issue (resolved if I run with php alone w/o hhvm). Are 4-6 serious issues? It seems those errors have been encountered before but are still being addressed at 1825?

1) SMW\Tests\IntlTimeFormatterTest::testFormat with data set #3 ('2/1300/11/02/12/03/25.888499949', 'en', 'Y-m-d H:i:s.u', '1300-11-02 12:03:25.888500')
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'1300-11-02 12:03:25.888500'
+'1300-11-02 12:03:25.888499'

/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Unit/IntlTimeFormatterTest.php:49
/var/www/html/mediawiki/maintenance/doMaintenance.php:111

2) SMW\Tests\IntlTimeFormatterTest::testFormat with data set #4 ('2/1300/11/02/12/03/25.888499949', 'en', 'H:i:s.u', '12:03:25.888500')
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'12:03:25.888500'
+'12:03:25.888499'

/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Unit/IntlTimeFormatterTest.php:49
/var/www/html/mediawiki/maintenance/doMaintenance.php:111

3) SMW\Tests\Integration\JSONScript\JsonTestCaseScriptRunnerTest::executeTestCases with data set "p-0414.json" ('/var/www/html/mediawiki/exten...4.json')
Failed on `#13 `.888499949` is being rounded to `.888500`` for <table class="sortable wikitable smwtable"><th>&#160;</th><th class="Has-date"><a href="/Property:Has_date" title="Property:Has date">Has date</a></th><th class="Has-date"><a href="/Property:Has_date" title="Property:Has date">Has date</a></th><tr data-row-number="1" class="row-odd"><td class="smwtype_wpg"><a href="/Example/P0414/7" title="Example/P0414/7">Example/P0414/7</a></td><td data-sort-value="2456116.9682395" class="Has-date smwtype_dat">8 July 2012 11:14:15</td><td data-sort-value="2456116.9682395" class="Has-date smwtype_dat">11:14:15.888499</td></tr></table>

<!--
NewPP limit report
Cached time: 20170223031823
Cache expiry: 86400
Dynamic content: false
[SMW] In‐text annotation parser time: 0.003 seconds
CPU time usage: 0.016 seconds
Real time usage: 0.020 seconds
Preprocessor visited node count: 11/1000000
Preprocessor generated node count: 0/1000000
Post‐expand include size: 469/2097152 bytes
Template argument size: 0/2097152 bytes
Highest expansion depth: 2/40
Expensive parser function count: 0/100
-->

<!--
Transclusion expansion time report (%,ms,calls,template)
100.00%    0.000      1 - -total
-->
 with (StringContains) [ <td data-sort-value="2456116.9682395" class="Has-date smwtype_dat">11:14:15.888500</td> ]
Failed asserting that 1 matches expected 2.
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Utils/Validators/StringValidator.php:82
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Utils/Validators/StringValidator.php:30
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Integration/JSONScript/ParserTestCaseProcessor.php:138
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Integration/JSONScript/ParserTestCaseProcessor.php:71
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Integration/JSONScript/JsonTestCaseScriptRunnerTest.php:239
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Integration/JSONScript/JsonTestCaseScriptRunnerTest.php:150
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/JsonTestCaseScriptRunner.php:160
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/MwDBaseUnitTestCase.php:125
/var/www/html/mediawiki/maintenance/doMaintenance.php:111

4) SMW\Tests\Integration\MediaWiki\LinksUpdateTest::testDoUpdateUsingUserdefinedAnnotations
SMW\Tests\Utils\Validators\SemanticDataValidator::assertContainsPropertyKeys asserts property key for 'Aa' with (_SKEY, _MDAT, HasFirstLinksUpdatetest, HasSecondLinksUpdatetest)
Failed asserting that an array contains 'Aa'.

/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Utils/Validators/SemanticDataValidator.php:321
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Utils/Validators/SemanticDataValidator.php:230
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Integration/MediaWiki/LinksUpdateTest.php:142
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/MwDBaseUnitTestCase.php:125
/var/www/html/mediawiki/maintenance/doMaintenance.php:111

5) SMW\Tests\Integration\MediaWiki\Import\RedirectPageTest::testPageImportToCreateRedirect
SMW\Tests\Utils\Validators\SemanticDataValidator::assertContainsPropertyValues for '_INST' as '__sin' with (Regression test, Redirect test, Simple redirect test)
Failed asserting that an array contains 'Lorem ipsum'.

/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Utils/Validators/SemanticDataValidator.php:379
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Utils/Validators/SemanticDataValidator.php:290
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Utils/Validators/SemanticDataValidator.php:78
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Integration/MediaWiki/Import/RedirectPageTest.php:154
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Integration/MediaWiki/Import/RedirectPageTest.php:123
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/MwDBaseUnitTestCase.php:125
/var/www/html/mediawiki/maintenance/doMaintenance.php:111

6) SMW\Tests\Integration\MediaWiki\Import\TimeDataTypeTest::testImportOfDifferentDateWithAssortmentOfOutputConversion
SMW\Tests\Utils\Validators\SemanticDataValidator::assertContainsPropertyValues for '_INST' as '__sin' with (Regression test)
Failed asserting that an array contains 'Lorem ipsum'.

/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Utils/Validators/SemanticDataValidator.php:379
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Utils/Validators/SemanticDataValidator.php:290
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Utils/Validators/SemanticDataValidator.php:78
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/Integration/MediaWiki/Import/TimeDataTypeTest.php:207
/var/www/html/mediawiki/extensions/SemanticMediaWiki/tests/phpunit/MwDBaseUnitTestCase.php:125
/var/www/html/mediawiki/maintenance/doMaintenance.php:111

[mwjames]

1. SMW\Tests\IntlTimeFormatterTest::testFormat with data set Do not spam IRC when a pull request is not broken #3 ('2/1300/11/02/12/03/25.888499949', 'en', 'Y-m-d H:i:s.u', '1300-11-02 12:03:25.888500')
   Failed asserting that two strings are equal.
2. SMW\Tests\IntlTimeFormatterTest::testFormat with data set Improve entry point #4 ('2/1300/11/02/12/03/25.888499949', 'en', 'H:i:s.u', '12:03:25.888500')
3. SMW\Tests\Integration\JSONScript\JsonTestCaseScriptRunnerTest::executeTestCases with data set "p-0414.json" ('/var/www/html/mediawiki/exten...4.json')

The first three relate to HHVM and have been reported (see #1456).

4. SMW\Tests\Integration\MediaWiki\LinksUpdateTest::testDoUpdateUsingUserdefinedAnnotations
   SMW\Tests\Utils\Validators\SemanticDataValidator::assertContainsPropertyKeys asserts property key for 'Aa' with (_SKEY, _MDAT, HasFirstLinksUpdatetest, HasSecondLinksUpdatetest)
   Failed asserting that an array contains 'Aa'.
5. SMW\Tests\Integration\MediaWiki\Import\RedirectPageTest::testPageImportToCreateRedirect
   SMW\Tests\Utils\Validators\SemanticDataValidator::assertContainsPropertyValues for '_INST' as '__sin' with (Regression test, Redirect test, Simple redirect test)
   Failed asserting that an array contains 'Lorem ipsum'.
6. SMW\Tests\Integration\MediaWiki\Import\TimeDataTypeTest::testImportOfDifferentDateWithAssortmentOfOutputConversion
   SMW\Tests\Utils\Validators\SemanticDataValidator::assertContainsPropertyValues for '_INST' as '__sin' with (Regression test)
   Failed asserting that an array contains 'Lorem ipsum'.

Are 4-6 serious issues? It seems those errors have been encountered before but are still being addressed at 1825?

The remaining three are a result of changes to MW1.28 in how categories are updated which has some practical relevance as seen in [0]. Of course, we could disable the test but somehow that would defeat the purpose of knowing that something changed from MW 1.27 to 1.28 and as one can see in [1], 1.29 contains the same issues.

You can check the sandbox and see the effect [2] by creating a page with a category (and doesn't need to contain SMW annotations) and after creating the category, you will see that the page doesn't appear in the category page as one would expect (PS: update to the category page is an issue in MW-core and not related to SMW).

[0] SemanticMediaWiki/semantic-mediawiki.org#22
[1] https://travis-ci.org/SemanticMediaWiki/SemanticMediaWiki/jobs/204621710
[2] https://sandbox.semantic-mediawiki.org/wiki/Cat%C3%A9gorie:MissingUpdateToCategory

[mwjames]

1.28 is tested [0] and passed but the HHVM specific issues have not been addressed as there are failures on that platform.

[0] https://travis-ci.org/SemanticMediaWiki/SemanticMediaWiki/jobs/231842075