New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation about icetea testcases #8073

Merged
merged 1 commit into from Sep 22, 2018

Conversation

Projects
None yet
6 participants
@jarlamsa
Contributor

jarlamsa commented Sep 11, 2018

Description

Add test case descriptions to mbed-os for Icetea tests.
These were asked here: #8065

Pull request type

[ ] Fix
[ ] Refactor
[ ] Target update
[x] Functionality change
[ ] Breaking change
@jarlamsa

This comment has been minimized.

Contributor

jarlamsa commented Sep 11, 2018

This folder contains netsocket tests for Icetea
The tests located under this folder are dependent of the application [socket_app](https://github.com/ARMmbed/mbed-os/blob/master/TEST_APPS/device/socket_app)
The test cases under this folder are defined in [Network Socket test plan](https://github.com/ARMmbed/mbed-os/blob/master/TESTS/netsocket/README.md)

This comment has been minimized.

@MarceloSalazar

MarceloSalazar Sep 11, 2018

Contributor

@SeppoTakalo , can you help us to review this please?
The "Network Socket test plan" does not have references to Icetea, so I don't see the relationship with doc.

This comment has been minimized.

@SeppoTakalo

SeppoTakalo Sep 11, 2018

Contributor

Originally it did not refer to any testing tool, but since then I have added instructions of how to build existing test cases.
Icetea release was pending, so I could not add reference to this.
However, testcases are exactly same, so this sentence here is correct.
Only the test plan needs updating, but that is ongoing maintaining work for us to keep it up to date.

This comment has been minimized.

@MarceloSalazar

MarceloSalazar Sep 11, 2018

Contributor

As a user, I still don't understand how to run the Icetea test cases with the network sockets.
@jarlamsa @SeppoTakalo can we add a line explaining how to run the test cases?

We have this today (greentea):
mbed test --compile -t <toolchain> -m <target> -n mbed-os-tests-network-*,mbed-os-tests-netsocket*

I assume this is the corresponding command to use icetea but I'm not 100% sure.

mbed test --compile -t <toolchain> -m <target> -n mbed-os-tests-network-*,mbed-os-tests-netsocket* --icetea

This comment has been minimized.

@jarlamsa

jarlamsa Sep 11, 2018

Contributor

There is already a readme about running the icetea tests in the root of TEST_APPS -folder
https://github.com/ARMmbed/mbed-os/blob/master/TEST_APPS/readme.md

This comment has been minimized.

@MarceloSalazar

MarceloSalazar Sep 11, 2018

Contributor

Ok, that's good.
@SeppoTakalo , I assume every test case (mbed-os-tests-network-*,mbed-os-tests-netsocket*) can be used with Icetea. If that's the case, I'd be happy to close this item.

This comment has been minimized.

@jarlamsa

jarlamsa Sep 11, 2018

Contributor

Only test cases done with Icetea can be used with Icetea, those are located under TEST_APPS/testcases, test cases done with Greentea can be used with Greentea, those are located under TESTS. The test plan needs information about used test framework for each test.

This comment has been minimized.

@SeppoTakalo

SeppoTakalo Sep 11, 2018

Contributor

Yes.
Some testcases exist only in Greentea, some exist only in Icetea.
All testcases exist in the test plan, and the selection of tool is based on requirement of the testcase.

For demonstration purposes some testcases exist on both, so that the differences can be studied.

@MarceloSalazar

This comment has been minimized.

Contributor

MarceloSalazar commented Sep 11, 2018

@jarlamsa thanks for the PR. It's looking good.

@cmonr

This comment has been minimized.

Contributor

cmonr commented Sep 12, 2018

@jarlamsa thanks for the PR. It's looking good.

Taking this as an equivalent of a review checkmark.

/morph build

@cmonr cmonr added needs: CI and removed needs: review labels Sep 12, 2018

@mbed-ci

This comment has been minimized.

mbed-ci commented Sep 12, 2018

Build : SUCCESS

Build number : 3054
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/8073/

Triggering tests

/morph test
/morph export-build
/morph mbed2-build

@mbed-ci

This comment has been minimized.

@jarlamsa jarlamsa force-pushed the jarlamsa:icetea_test_docs branch from e9dea47 to 0e4ae40 Sep 12, 2018

@jarlamsa

This comment has been minimized.

Contributor

jarlamsa commented Sep 12, 2018

Added information about the disabled exampleapp and how to enable it if wanted to.

@mbed-ci

This comment has been minimized.

@cmonr cmonr added needs: review and removed needs: CI labels Sep 12, 2018

@cmonr

This comment has been minimized.

Contributor

cmonr commented Sep 12, 2018

@jarlamsa @SeppoTakalo @MarceloSalazar Is this now good to go? As a reminder, CI results are invalidated if the commit hash changes.

And we're aware of how silly it is to run CI on a docs-only PR, but that's what we do for now.

@jarlamsa

This comment has been minimized.

Contributor

jarlamsa commented Sep 13, 2018

I think it is, @MarceloSalazar @SeppoTakalo @OPpuolitaival do you have anything to add?

@MarceloSalazar

This comment has been minimized.

Contributor

MarceloSalazar commented Sep 13, 2018

@jarlamsa , all,
Thanks for the enhancements, this is great!
I'm happy for this ticket to be closed.

@cmonr

This comment has been minimized.

Contributor

cmonr commented Sep 13, 2018

I'm happy for this ticket to be closed.

Making sure, this superceedes #8065 correct?

@cmonr

cmonr approved these changes Sep 20, 2018

@cmonr

This comment has been minimized.

Contributor

cmonr commented Sep 22, 2018

/morph build

@mbed-ci

This comment has been minimized.

mbed-ci commented Sep 22, 2018

Build : SUCCESS

Build number : 3125
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/8073/

Triggering tests

/morph test
/morph export-build
/morph mbed2-build

@mbed-ci

This comment has been minimized.

@mbed-ci

This comment has been minimized.

@cmonr cmonr merged commit c5f2d16 into ARMmbed:master Sep 22, 2018

14 checks passed

ci-morph-build build completed
Details
ci-morph-exporter build completed
Details
ci-morph-mbed2-build build completed
Details
ci-morph-test test completed , RTOS ROM(+0.0%) RAM(-0.03%)
Details
continuous-integration/jenkins/pr-head This commit looks good
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
jenkins-ci/cloud_client_smoke_test Test job: successful
Details
travis-ci/astyle Passed, 598 files
Details
travis-ci/docs Local docs testing has passed
Details
travis-ci/events Passed, runtime is 10226 cycles (-315 cycles)
Details
travis-ci/gitattributestest Local gitattributestest testing has passed
Details
travis-ci/licence_check Local licence_check testing has passed
Details
travis-ci/littlefs Passed, code size is 8372B (+0.00%)
Details
travis-ci/tools-py2.7 Local tools-py2.7 testing has passed
Details

@cmonr cmonr removed the needs: CI label Sep 22, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment