Enable iotivity-node tests on device #36

wanghongjuan · 2016-01-12T02:36:15Z

Using grunt to run the test of this repo on device

Set up the command grunt can run normally on test device:

root@intel-corei7-64:/usr/lib/node_modules/iotivity-node# grunt
-sh: grunt: command not found

Failed to run command grunt because of lack of modules grunt-cli under the directory iotivity-node/node_modules, so I copy the module grunt-cli from desktop to test device:

$ scp -r grunt-cli root@test-device-ip:/usr/lib/node_modules/iotivity-node/node_modules
$ ssh root@test-device-ip
$ cd /usr/lib/node_modules/iotivity-node
$ ./node_modules/grunt-cli/bin/grunt -h

The command grunt can run normally
2. Copy all relevant files with test to device

$ scp -r .jscsrc Gruntfile.js grunt-build package.json tests root@test-device-ip:/usr/lib/node_modules/iotivity-node/
$ ./node_modules/grunt-cli/bin/grunt test

Only show the part of output error message:

API Discovery
✓ @      2828: Server: device.configure() successful
✓ @      2837: Server: device.registerResource() successful
✓ @      5707: Client: open-ended resource discovery handle is initially undefined
✓ @      5716: Client: device.findResources() successful
✓ @      6019: Client: Resource found
✓ @      6022: Client: Resource present exactly once among resources
✓ @      6028: Client: open-ended resource discovery handle is set after one discovery
✗ @     30009: Test timed out
Actual:
null
Expected:
undefined
✗ @     30028: Expected 16 assertions, but 8 were run
Actual:
null
Expected:
undefined

Get Request
✓ @      2724: Server: OCInit successful
✓ @      2731: Server: OCCreateResource successful
✓ @      5457: Client: OCInit successful
✓ @      5463: Client: OCDoResource(discovery) successful
✓ @      5872: Client: Resource found
✓ @      5874: Client: OCDoResource(get) successful
✓ @      5961: Server: Incoming request has the correct flags
✓ @      5962: Server: Correct request received
✓ @      5965: Server: OCDoResponse successful
✓ @      5977: Client: Correct response received
✓ @      5980: Client: OCProcess succeeded 4 times
✓ @      5982: Client: OCStop successful
✓ @      5988: Client exited successfully (0)
✓ @      5990: Client did not segfault
✓ @      5993: Server: OCProcess succeeded 32 times
✓ @      5993: Server: OCDeleteResource successful
✓ @      6003: Server: OCStop successful
✓ @      6008: Server exited successfully (0)
✓ @      6008: Server did not segfault

Load Library
/usr/lib/node_modules/iotivity-node/node_modules/bindings/bindings.js:91
  throw err
        ^
Error: Could not locate the bindings file. Tried:
 → /usr/lib/node_modules/iotivity-node/build/dlopennow.node
 → /usr/lib/node_modules/iotivity-node/build/Debug/dlopennow.node
 → /usr/lib/node_modules/iotivity-node/build/Release/dlopennow.node
 → /usr/lib/node_modules/iotivity-node/out/Debug/dlopennow.node
 → /usr/lib/node_modules/iotivity-node/Debug/dlopennow.node
 → /usr/lib/node_modules/iotivity-node/out/Release/dlopennow.node
 → /usr/lib/node_modules/iotivity-node/Release/dlopennow.node
 → /usr/lib/node_modules/iotivity-node/build/default/dlopennow.node
 → /usr/lib/node_modules/iotivity-node/compiled/0.12.7/linux/x64/dlopennow.node
    at bindings (/usr/lib/node_modules/iotivity-node/node_modules/bindings/bindings.js:88:9)
    at Object.<anonymous> (/usr/lib/node_modules/iotivity-node/tests/tests/Load Library.js:2:35)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Function.Module.runMain (module.js:501:10)
    at startup (node.js:129:16)
    at node.js:814:3
✗ @       527: Test exited successfully (1)
Actual:
false
Expected:
true
✓ @       532: Test did not segfault
✗ @       546: Expected number of assertions to be defined, but expect() was not called.
Actual:
null
Expected:
undefined

Observation
✓ @      2729: Server: OCInit successful
✓ @      2736: Server: OCCreateResource successful
✓ @      5464: Client: OCInit successful
✓ @      5471: Client: OCDoResource(discovery) successful
✓ @      5881: Client: Resource found
✓ @      5884: Client: OCDoResource(observation) successful
✗ @      5959: Server: The first request has both the request and observe flags set
Actual:
{
  "OC_REQUEST_FLAG": true
}
Expected:
{
  "OC_OBSERVE_FLAG": true,
  "OC_REQUEST_FLAG": true
}
✓ @      5980: Server: Observation ID is not zero (18)
✓ @      5984: Server: Action is set to OC_OBSERVE_REGISTER
✓ @      5984: Server: OCDoResponse successful
✗ @      5985: Server: OCNotifyAllObservers(after 0 successful calls) successful
Actual:
"OC_STACK_NO_OBSERVERS"
Expected:
"OC_STACK_OK"
/usr/lib/node_modules/iotivity-node/tests/suite.js:191
              throw new Error( error );
                    ^
Error: Server requested teardown: Server: OCNotifyAllObservers(after 0 successful calls) failed
    at Object.spawnOptions.teardown (/usr/lib/node_modules/iotivity-node/tests/suite.js:191:14)
    at /usr/lib/node_modules/iotivity-node/tests/suite.js:82:13
    at arrayEach (/usr/lib/node_modules/iotivity-node/node_modules/lodash/index.js:1289:13)
    at Function.<anonymous> (/usr/lib/node_modules/iotivity-node/node_modules/lodash/index.js:3345:13)
    at Socket.serverStdoutData (/usr/lib/node_modules/iotivity-node/tests/suite.js:59:5)
    at Socket.emit (events.js:107:17)
    at readableAddChunk (_stream_readable.js:163:16)
    at Socket.Readable.push (_stream_readable.js:126:10)
    at Pipe.onread (net.js:538:20)
Warning: Task "testsuite" failed. Use --force to continue.

Aborted due to warnings.
root@intel-corei7-64:/usr/lib/node_modules/iotivity-node#

The text was updated successfully, but these errors were encountered:

zqzhang · 2016-01-12T08:57:57Z

@gabrielschulhof thank you for the reminding and updating of this issue.

Add more background and opens here.

I thought it would be valuable and beneficial to run tests from engineering team on real devices. For you, it can help to identify issues of implementation and/or tests within real environment; for me (QA), it helps to enable us to try and verify new implementation (features) quickly. Right?

Going forward, QA can make contributions to these tests, either fixing issues or adding new tests, based on our testing methodology. Thus you can help review and run the tests in your development environment. Acceptable?

Suppose you are going to use the same test framework as this when you are implementing Soletta C APIs bindings for Node.js; we can co-work the tests if it is acceptable. Make sense?

gabrielschulhof · 2016-01-12T09:15:12Z

You also need to copy the file dlopennow.node to the device because, as part of the testing, a second native addon is built which ensures that all of the bindings' dependencies are available.

zqzhang · 2016-01-12T09:22:33Z

2 more opens:

Is it possible to integrate the tests and dependencies into devices by default? https://github.com/otcshare/meta-iot-web/blob/master/recipes-web/iotivity-node/iotivity-node.bb#L82 says removing the test suite as of now.
Is there any background or why we haven't enabled the npm in devices?

Thanks.

zqzhang · 2016-01-12T09:26:51Z

Seems there is no dlopennow.node in this repo. Is that the main reason why you removed test suite from installation? Possible integrate it?

gabrielschulhof · 2016-01-12T09:27:03Z

@zqzhang the link you gave does not refer to the iotivity-node test suite, but rather to the test suite for a library called ffi on which iotivity-node used to depend but no longer depends.

gabrielschulhof · 2016-01-12T09:27:32Z

dlopennow.node is not distributed by default, however it is built as part of the testing process.

gabrielschulhof · 2016-01-12T09:28:24Z

https://github.com/otcshare/iotivity-node/blob/master/binding.gyp#L17-L18

zqzhang · 2016-01-13T04:49:05Z

@gabrielschulhof seems the dlopennow.node is a Node.js add-on, right? I wonder whether you can provide a built library somewhere in this repository, to help us enable the tests on devices without building the source code? Thanks.

zqzhang · 2016-01-13T04:51:03Z

By the way, seems only https://github.com/otcshare/iotivity-node/blob/master/tests/tests/Load%20Library.js#L2 references the dlopennow add-on.

wanghongjuan · 2016-01-13T08:09:53Z

Thanks, follow the comment update the test steps and rerun the tests:

The dlopennow.node can be built by run command $ sudo ./dish TESTING=true on desktop
Copy the file to device:
$ scp dlopennow.node root@test-device-ip:/usr/lib/node_modules/iotivity-node/build/Release
Rerun the tests
$ ./node_modules/grunt-cli/bin/grunt test
The test suites Load Library and Observation run successfully.
But the test also will aborted due to the warning, show the part of output error message:

API Discovery
✓ @      2827: Server: device.configure() successful
✓ @      2836: Server: device.registerResource() successful
✓ @      7540: Client: open-ended resource discovery handle is initially undefined
✓ @      7555: Client: device.findResources() successful
✓ @      8433: Client: Resource found
✓ @      8438: Client: Resource present exactly once among resources
✓ @      8450: Client: open-ended resource discovery handle is set after one discovery
✗ @     30039: Test timed out
Actual: 
null
Expected: 
undefined
✗ @     30063: Expected 16 assertions, but 8 were run
Actual: 
null
Expected: 
undefined

Resource Operations
✓ @      2738: OCInit successful
✓ @      2740: OCSetDeviceInfo successful
✓ @      2741: OCSetPlatformInfo successful
✓ @      2742: OCGetNumberOfResources(to establish initial count) successful
✓ @      2743: OCCreateResource(parent) successful
✓ @      2754: OCGetNumberOfResources successful
✓ @      2755: OCGetNumberOfResources reports an increase by one resource after OCCreateResource
✓ @      2756: OCGetResourceHandler initially returns the initial entity handler
✓ @      2760: OCBindResourceHandler successful
✓ @      2761: OCGetResourceHandler returns the alternate entity handler after setting it
✓ @      2763: OCGetResourceUri returns the correct URI
✓ @      2763: OCGetNumberOfResourceInterfaces successful
✓ @      2764: Initially there is only one interface on the resource
✓ @      2765: OCBindResourceInterfaceToResource successful
✓ @      2766: OCGetNumberOfResourceInterfaces successful
✓ @      2767: After adding an interface there are two interfaces on the resource
✓ @      2768: First interface is 'oic.if.baseline'
/usr/lib/node_modules/iotivity-node/tests/tests/Resource Operations.js:143
  iotivity.__compareResourceHandles( resourceHandleReceptacle.handle,
           ^
TypeError: undefined is not a function
    at Object.<anonymous> (/usr/lib/node_modules/iotivity-node/tests/tests/Resource Operations.js:143:11)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Function.Module.runMain (module.js:501:10)
    at startup (node.js:129:16)
    at node.js:814:3
✓ @      2769: Second interface is 'a.b.c'
✓ @      2771: OCBindResourceTypeToResource successful
✓ @      2771: OCGetNumberOfResourceTypes successful
✓ @      2772: After adding a type there are two types on the resource
✓ @      2773: First type is correct
✓ @      2774: Second type is correct
✓ @      2776: OCGetResourceProperties correctly returns properties set on resource
✗ @      2779: Test exited successfully (1)
Actual: 
false
Expected: 
true
✓ @      2793: Test did not segfault
✗ @      2810: Expected 35 assertions, but 26 were run
Actual: 
null
Expected: API Discovery
✓ @      2827: Server: device.configure() successful
✓ @      2836: Server: device.registerResource() successful
✓ @      7540: Client: open-ended resource discovery handle is initially undefined
✓ @      7555: Client: device.findResources() successful
✓ @      8433: Client: Resource found
✓ @      8438: Client: Resource present exactly once among resources
✓ @      8450: Client: open-ended resource discovery handle is set after one discovery
✗ @     30039: Test timed out
Actual: 
null
Expected: 
undefined
✗ @     30063: Expected 16 assertions, but 8 were run
Actual: 
null
Expected: 
undefined


Resource Operations
✓ @      2738: OCInit successful
✓ @      2740: OCSetDeviceInfo successful
✓ @      2741: OCSetPlatformInfo successful
✓ @      2742: OCGetNumberOfResources(to establish initial count) successful
✓ @      2743: OCCreateResource(parent) successful
✓ @      2754: OCGetNumberOfResources successful
✓ @      2755: OCGetNumberOfResources reports an increase by one resource after OCCreateResource
✓ @      2756: OCGetResourceHandler initially returns the initial entity handler
✓ @      2760: OCBindResourceHandler successful
✓ @      2761: OCGetResourceHandler returns the alternate entity handler after setting it
✓ @      2763: OCGetResourceUri returns the correct URI
✓ @      2763: OCGetNumberOfResourceInterfaces successful
✓ @      2764: Initially there is only one interface on the resource
✓ @      2765: OCBindResourceInterfaceToResource successful
✓ @      2766: OCGetNumberOfResourceInterfaces successful
✓ @      2767: After adding an interface there are two interfaces on the resource
✓ @      2768: First interface is 'oic.if.baseline'
/usr/lib/node_modules/iotivity-node/tests/tests/Resource Operations.js:143
  iotivity.__compareResourceHandles( resourceHandleReceptacle.handle,
           ^
TypeError: undefined is not a function
    at Object.<anonymous> (/usr/lib/node_modules/iotivity-node/tests/tests/Resource Operations.js:143:11)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Function.Module.runMain (module.js:501:10)
    at startup (node.js:129:16)
    at node.js:814:3
✓ @      2769: Second interface is 'a.b.c'
✓ @      2771: OCBindResourceTypeToResource successful
✓ @      2771: OCGetNumberOfResourceTypes successful
✓ @      2772: After adding a type there are two types on the resource
✓ @      2773: First type is correct
✓ @      2774: Second type is correct
✓ @      2776: OCGetResourceProperties correctly returns properties set on resource
✗ @      2779: Test exited successfully (1)
Actual: 
false
Expected: 
true
✓ @      2793: Test did not segfault
✗ @      2810: Expected 35 assertions, but 26 were run
Actual: 
null
Expected: 
undefined
Total assertions: ( 319 + 4 ) / 323
Warning: Task "testsuite" failed. Use --force to continue.

Aborted due to warnings.

gabrielschulhof · 2016-04-07T13:08:45Z

@zqzhang
Only the "Load Library" test needs dlopen. You can skip it if you run the tests like this:

( echo -n '"';
  ls tests/tests/ | grep -v 'Load Library' | awk '
    BEGIN {file="";}
    {if (file == "") file=$0; else { printf(file ","); file=$0;}}
    END { printf file;}
  ';
  echo -n '"' ) | xargs node tests/suite

gabrielschulhof · 2016-04-07T13:12:20Z

Shorter version:

ls tests/tests/ | grep -v 'Load Library' | awk '
    BEGIN {file="";}
    {if (file == "") file=$0; else { printf(file ","); file=$0;}}
    END { printf file;}
  ' | xargs -d '\n' | node tests/suite

gabrielschulhof · 2016-04-07T13:42:10Z

Shorter still:

ls tests/tests | grep -v 'Load Library' | tr '\n' ',' | sed 's/,$//' | xargs -d '\n' node tests/suite

wanghongjuan · 2016-04-12T03:25:22Z

Thanks for response, I will take it.

wanghongjuan changed the title ~~Enable iotivity-node tests on Minnow Board Max~~ Enable iotivity-node tests on Ostro OS Jan 12, 2016

gabrielschulhof changed the title ~~Enable iotivity-node tests on Ostro OS~~ Enable iotivity-node tests on device Jan 12, 2016

wanghongjuan closed this as completed Apr 12, 2016

wanghongjuan mentioned this issue Apr 26, 2016

Failed to run iotivity-node tests with latest commit 3908662 #47

Closed

sanbeam mentioned this issue Dec 17, 2016

high level client crash #98

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable iotivity-node tests on device #36

Enable iotivity-node tests on device #36

wanghongjuan commented Jan 12, 2016

zqzhang commented Jan 12, 2016

gabrielschulhof commented Jan 12, 2016

zqzhang commented Jan 12, 2016

zqzhang commented Jan 12, 2016

gabrielschulhof commented Jan 12, 2016

gabrielschulhof commented Jan 12, 2016

gabrielschulhof commented Jan 12, 2016

zqzhang commented Jan 13, 2016

zqzhang commented Jan 13, 2016

wanghongjuan commented Jan 13, 2016

gabrielschulhof commented Apr 7, 2016

gabrielschulhof commented Apr 7, 2016

gabrielschulhof commented Apr 7, 2016

wanghongjuan commented Apr 12, 2016

Enable iotivity-node tests on device #36

Enable iotivity-node tests on device #36

Comments

wanghongjuan commented Jan 12, 2016

Using grunt to run the test of this repo on device

zqzhang commented Jan 12, 2016

gabrielschulhof commented Jan 12, 2016

zqzhang commented Jan 12, 2016

zqzhang commented Jan 12, 2016

gabrielschulhof commented Jan 12, 2016

gabrielschulhof commented Jan 12, 2016

gabrielschulhof commented Jan 12, 2016

zqzhang commented Jan 13, 2016

zqzhang commented Jan 13, 2016

wanghongjuan commented Jan 13, 2016

gabrielschulhof commented Apr 7, 2016

gabrielschulhof commented Apr 7, 2016

gabrielschulhof commented Apr 7, 2016

wanghongjuan commented Apr 12, 2016