Skip to content

Test(for releasing 12.0)

Jump to bottom Edit New page
Goun Lee edited this page Jun 1, 2017 · 54 revisions

Browsers test

Test Environment

  • Windows7
  • JsUnit 2.2 TestRunner (using js\test\testSuite.html)

Chrome

Running on Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36 NetHelper70

Result: Runs: 35951 Errors: 0 Failures: 0

FireFox

Running on Mozilla/5.0 (Windows NT 6.3; WOW64; rv:39.0) Getcko/20100101 Firefox/39.0 NetHelper70

Result: Runs: 35951 Errors: 0 Failures: 0

Opera

Running on Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36 OPR/44.0.2510.1449

Result: Runs: 35951 Errors: 0 Failures: 1

Tests with problems (1 total) - JsUnit

1. root/test/testRootDyn.html:testGetLocaleBrowser failed

Expected <ko> (String) but was <en-US> (String)

NOTE: Fail case 1 is known issue caused by locale.

Safari

Running on Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534.57.2 (KHTML, like Gecko) Version/5.1.7 Safari/534.57.2

Result: Runs: 35951 Errors: 0 Failures: 2

Tests with problems (2 total) - JsUnit

1. root/test/testGlobal.html:testGetLocaleBrowser failed

Expected <ko-KR> (String) but was <en-US> (String)


2. root/test/testGlobalCompiled.html:testGetLocaleBrowser failed

Expected <ko-KR> (String) but was <en-US> (String)

NOTE: Fail case 1, 2 is known issue caused by locale.

IE11

Running on Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; .NET4.0C; .NET4.0E; NetHelper70; CNS_UA; AD_LOGON=4C47452E4E4554; rv:11.0) like Gecko

Result: Runs: 35951 Errors: 0 Failures: 10

Tests with problems (10 total) - JsUnit

1. root/test/testGlobal.html:testGetLocaleBrowser failed

Expected <ko-KR> (String) but was <en-US> (String)


2. root/test/testGlobalCompiled.html:testGetLocaleBrowser failed

Expected <ko-KR> (String) but was <en-US> (String)


3. collate/test/testCollation.html:testCollatorWithSortUpperFirst failed

" found nested Array@0
"
Expected <e> (String) but was <E> (String)


4. collate/test/testCollation.html:testCollatorCaseMixedWithIndexMarkers_zh_Hans failed

" found nested Array@2
"
Expected <啊> (String) but was <�Banana> (String)


5. collate/test/testCollation.html:testCollatorCaseMixedWithIndexMarkers_zh_Hant_stroke failed

" found nested Array@0
"
Expected <丿> (String) but was <�.10> (String)


6. collate/test/testCollation.html:testCollatorCaseMixedWithIndexMarkers_zh_Hant_zhuyin failed

" found nested Array@1
"
Expected <褒> (String) but was <�ㄆ> (String)


7. collate/test/testCollationCompiled.html:testCollatorWithSortUpperFirst failed

" found nested Array@0
"
Expected <e> (String) but was <E> (String)


8. collate/test/testCollationCompiled.html:testCollatorCaseMixedWithIndexMarkers_zh_Hans failed

" found nested Array@2
"
Expected <啊> (String) but was <�Banana> (String)


9. collate/test/testCollationCompiled.html:testCollatorCaseMixedWithIndexMarkers_zh_Hant_stroke failed

" found nested Array@0
"
Expected <丿> (String) but was <�.10> (String)


10. collate/test/testCollationCompiled.html:testCollatorCaseMixedWithIndexMarkers_zh_Hant_zhuyin failed

" found nested Array@1
"
Expected <褒> (String) but was <�ㄆ> (String)

NOTE: Fail case 1, 2 is known issue caused by locale. Is there a way to change language(navigator.browserLanguage, navigator.browserLanguage or navigator.language) in IE11? I tried to set language to en-US in its option, but it didn't work. I could not find the way to change locale, so I modified LocaleInfo class like below:

ilib.getLocale = function () {
    if (typeof(ilib.locale) !== 'string') {
    	var plat = ilib._getPlatform();
    	switch (plat) {
    	    case 'browser':
                ilib.locale = 'en-US';
                break;

The remaining fail cases are about collation. But they are using useNative: false. (Fail case 3, 4, 5, 6 are duplicated with Fail case 7, 8, 9, 10) In Fail case 3, IE11 has same result with result of Chrome/FF, so if statement of test case is not needed anymore. But in Fail case 4, 5, 6, IE11 has different sort result compared with result of Chrome/FF. Maybe IE11 has something different in the implementation of Array.sort compared with Chrome/FF.

// IE11
�APPLE,�Apple,�Banana,�Lemon,�ORange,�Orange,�Peach,�RASPBERRY,�Raspberry,�Yam,啊,波,吃,次,德,额,佛,各,和,记,科,里,摸,那,坡,起,日,食,四,体,吴,西,一,站,子
// Chrome/FF
�APPLE,�Apple,啊,�Banana,波,吃,次,德,额,佛,各,和,记,科,�Lemon,里,摸,那,�ORange,�Orange,�Peach,坡,起,�RASPBERRY,�Raspberry,日,食,四,体,吴,西,�Yam,一,站,子

IE8

NOTE: Could not install IE8 on Windows7 in the usual way. Is it necessary for release?

Edge

Running on Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393

  1. collate/legacy/testCollation.html:testCollatorGetComparatorWorksWithCase failed

"A < a" Call to assert(boolean) with false

  1. collate/legacy/testCollationCompiled.html:testCollatorGetComparatorWorksWithCase failed

"A < a" Call to assert(boolean) with false

  1. units/legacy/testUnitFmt.html:testUnitFormatTemperature6 failed

Expected <1 093,3333333333335 °C> (String) but was <1 093,3333333333334 °C> (String)

  1. maps/legacy/testToUpper.html:testToUpperGreekSigma_default failed

Expected <ΙΑΣΑΣ ΙΑΣΑΣ> (String) but was <ΙΑΣΑς ΙΑΣΑς> (String)

  1. maps/legacy/testToUpper.html:testToUpper_el_GR failed

Expected <ΙΑΣΑΣ ΙΑΣΑΣ> (String) but was <ΙΑΣΑς ΙΑΣΑς> (String)

  1. maps/legacy/testToUpperCompiled.html:testToUpperGreekSigma_default failed

Expected <ΙΑΣΑΣ ΙΑΣΑΣ> (String) but was <ΙΑΣΑς ΙΑΣΑς> (String)

  1. maps/legacy/testToUpperCompiled.html:testToUpper_el_GR failed

Expected <ΙΑΣΑΣ ΙΑΣΑΣ> (String) but was <ΙΑΣΑς ΙΑΣΑς> (String)

webOS 4.0 Browser

Test Environment

  • webOS 3.5 (goldilocks-master #1324)
  • JsUnit 2.2 TestRunner (using js\test\testSuite.html)

Running on Mozilla/5.0(Linux;NetCast;U) AppleWebKit/537.31(KHTML,like Gecko) Chrome/53.0.2785.34 Safari/537.32 SmartTV.6.0)

All passed without any erros and failures.Status Done(11695.366 seconds) Result: Runs: 35601 Errors: 0 Failures: 0

NodeJS Test

Test Environment

  • webOS 4.0
  • nodejs v0.12.0

Result

# node testSuite
Running uncompiled dynamic suites:   ["util",".","calendar","date1","date2","date3","daterange1","daterange2","daterange3","durfmt","number","maps","ctype","s	trings-ext","phone1","phone2","phone3","units","name","address1","address2","address3","collate"]	

Summary - 18695 tests run, 18695 pass, 0 fail, 422.394 seconds.

Test Environment

  • Windows 7
  • nodejs v4.0.0

Result

# node testSuite

D:\2017\ilib-WindowTest\js\test>node testSuite.js (node) util.print is deprecated. Use console.log instead. Running uncompiled dynamic suites ["util",".","calendar","date1","date2","date3","daterange1","daterange2","daterange3","durfmt","numb er","maps","ctype","strings-ext","phone1","phone2","phone3","units","name","address1","address2","address3","collate"] Summary - 18695 tests run, 18695 pass, 0 fail, 142.223 seconds.

Test Environment

  • Windows 7
  • nodejs v6.10.2

Result
Running uncompiled dynamic suites: ["util",".","calendar","date1","date2","date3","daterange1","daterange2","daterange3","durfmt","numb er","maps","ctype","strings-ext","phone1","phone2","phone3","units","name","address1","address2","address3","collate"] Summary - 18695 tests run, 18695 pass, 0 fail, 64.596 seconds.

QT test

Ubuntu

Test Environment

  • Ubuntu 12.04.5 LTS (GNU/Linux 3.11.0-26-generic x86_64)
  • Qt 5.6

goun@goun:~/Source/ilib-WindowTest/qt$ ant test.qt Buildfile: /home/goun/Source/ilib-WindowTest/qt/build.xml

test.qt: [exec] file:///home/goun/Source/ilib-WindowTest/qt/UnitTest/TestSuiteModule.js:98: ReferenceError: module is not defined [exec] qml: Running uncompiled dynamic suites: [".","address1","address2","address3","calendar","collate","ctype","date1","date2","date3","daterange1","daterange2","daterange3","durfmt","maps","name","number","phone1","phone2","phone3","strings-ext","units","util"] [exec] [exec] qml: Summary - 17185 tests run, 17185 pass, 0 fail, 746.91 seconds.

Test Environment

  • Ubuntu 16.04.5 LTS
  • Qt 5.6

Result

$ ant test.qt    
Buildfile: /codelab/iLib/migration/qt/build.xml
test.qt:
[exec] file:///codelab/iLib/migration/qt/UnitTest/TestSuiteModule.js:98: ReferenceError: module is not defined
[exec] qml: Running uncompiled dynamic suites:   
    [".","address1","address2","address3","calendar","collate","ctype","date1","date2","date3","daterange1","daterange2","daterange3","durfmt","maps","name","number","phone1","phone2","phone3","strings-ext","units","util"]
[exec]
[exec] qml: Summary - 17185 tests run, 17185 pass, 0 fail, 754.583 seconds.

webOS

Test Environment

  • webOS 4.0
  • Qt 5.6

except collation, calendar

2017-01-01T00:12:02.986358Z [726.375169845] user.warning qml-runner [] eos QWARNING {} : Summary - 14685 tests run, 14685 pass, 0 fail, 452.229 seconds.

calendar

2017-01-01T01:07:06.693848Z [4030.051935879] user.warning qml-runner [] eos QWARNING {} : Summary - 1706 tests run, 1706 pass, 0 fail, 126.889 seconds.

collate

  1. except zh

2017-01-01T01:05:34.028874Z [3937.435282584] user.warning qml-runner [] eos QWARNING {} : Summary - 737 tests run, 737 pass, 0 fail, 95.666 seconds.

  1. except zh-Hant

2017-01-01T01:20:50.453397Z [4853.859805271] user.warning qml-runner [] eos QWARNING {} : Summary - 17 tests run, 17 pass, 0 fail, 799.296 seconds.

  1. zh-Hant

2017-01-01T02:10:33.245261Z [8306.422512345] user.warning qml-runner [] eos QWARNING {} : Summary - 40 tests run, 40 pass, 0 fail, 8106.065 seconds.

iOS test

Due to constraints of the iOS platform, all browser must be built on top of the WebKit rendering engine. https://blog.chromium.org/2017/01/open-sourcing-chrome-on-ios.html

Chrome

Running on Mozilla/5.0 (iPad;CPU OS_10_2_1 like Mac OS X) AppleWebKit/602.1.50(KTHML,like Gecko) CriOS/57.2.2987.137 Mobile/14D27 Safari/602.1

Safari

Running on Mozilla/5.0 (iPad;CPU OS_10_2_1 like Mac OS X) AppleWebKit/602.4.6(KTHML,like Gecko) Version/10.0 Mobile/14D27 Safari/602.1

Opera

Running on Mozilla/5.0 (iPad;CPU OS_10_2_1 like Mac OS X) AppleWebKit/602.4.6(KTHML,like Gecko) Version/10.0 Mobile/14D27

Firefox

Running on Mozilla/5.0 (iPad;CPU OS_10_2_1 like Mac OS X) AppleWebKit/602.4.6(KTHML,like Gecko) FxiOS/7.3b2772 Mobile 14D27 Safari/602.4.6

All of browsers on iOS returns same fail cases as below:

  1. root/legacy/testGlobal.html:testGetTimeZoneDefault failed Expected (String) but was <America/Los_Angeles> (String)

  2. root/legacy/testGlobalCompiled.html:testGetTimeZoneDefault failed Expected (String) but was <America/Los_Angeles> (String)

  3. collate/legacy/testCollation.html:testCollatorDefaultCase failed "A < a" Call to assert(boolean) with false Stack

  4. collate/legacy/testCollation.html:testCollatorGetComparatorWorksWithCase failed "A < a" Call to assert(boolean) with false

  5. collate/legacy/testCollationCompiled.html:testCollatorDefaultCase failed "A < a" Call to assert(boolean) with false

  6. collate/legacy/testCollationCompiled.html:testCollatorGetComparatorWorksWithCase failed "A < a" Call to assert(boolean) with false

Android browser

Android Browser (Firefox)

Running on Mozilla/5.0 (Android 5.1; Mobile;rv:53.0) Gecko/53.0 Firefox /53.0

  1. collate/legacy/testCollation.html:testCollatorDefaultCase failed "a < A" Call to assert(boolean) with false

  2. collate/legacy/testCollation.html:testCollatorGetComparatorWorksWithCase failed "a < A" Call to assert(boolean) with false

  3. collate/legacy/testCollationCompiled.html:testCollatorDefaultCase failed "a < A" Call to assert(boolean) with false

  4. collate/legacy/testCollationCompiled.html:testCollatorGetComparatorWorksWithCase failed "a < A" Call to assert(boolean) with false

Android Browser (Opera)

Running on Mozilla/5.0 (Linux; Android 5.1; LG-F500L Build/LMY47D) AppleWebKit/537.36(KHTML, like Gecko) Chrome/55.0.2883.91 Mobile Safari/537.36 OPR/42.7.2246.114996

  • Lack of a memory, Running all of unnitest at once take sooooo long. Running unnitest manually per features. It passed them all with out any failures.
Add a custom footer
Add a custom sidebar

Clone this wiki locally