Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TCP/TLS Socket tests will skip if TCP is not supported #10037

Merged
merged 1 commit into from Mar 29, 2019

Conversation

@michalpasztamobica
Copy link
Contributor

commented Mar 11, 2019

Description

I generally avoid using macros, but I couldn't come up with a better way in this case.
Two reasons why macros are used to wrap the open calls:

  1. I wanted to minimise the "boilerplate" which gets copied and pasted into every test.
  2. The SKIP_TEST_MESSAGE macro contains a "return;" which is intended to return from a current test code. Therefore we cannot use it inside any wrapper function, as then it would merely return from that function, not from the test as such.

The architectures which could make use of this (for example UBLOX_C030_N211) are not available in any RAAS server, but I agreed with @AriParkkila that he will make sure this executes correctly.

Pull request type

[ ] Fix
[ ] Refactor
[ ] Target update
[ ] Functionality change
[ ] Docs update
[x] Test update
[ ] Breaking change

Reviewers

@SeppoTakalo
@KariHaapalehto
@VeijoPesonen
@mtomczykmobica
@tymoteuszblochmobica
@AriParkkila

@michalpasztamobica michalpasztamobica force-pushed the michalpasztamobica:tcp_skip_if_unsupported branch 2 times, most recently to a206df9 Mar 11, 2019

@SeppoTakalo

This comment has been minimized.

Copy link
Contributor

commented Mar 11, 2019

Uh.. You completely lost me with those macros..

I was hoping a way more simpler solution, like:

bool is_tcp_supported() {
    static bool supported;
    static bool tested = false;
    if (!tested) {
        TCPSocket socket;
        supported = socket.open(NetworkInterface::get_default_instance()) == NSAPI_ERROR_OK;
    }
    return supported;
}

void TCPSOCKET_ECHOTEST()
{
    if (!is_tcp_supported()) {
        TEST_SKIP_MESSAGE("TCP not supported");
    }
    ...
}

Add extra 3 lines per TCP test case.. should not be too much.

@ciarmcom ciarmcom requested review from AriParkkila, KariHaapalehto, mtomczykmobica, SeppoTakalo, tymoteuszblochmobica, VeijoPesonen and ARMmbed/mbed-os-maintainers Mar 11, 2019

@michalpasztamobica

This comment has been minimized.

Copy link
Contributor Author

commented Mar 11, 2019

I did not consider this option to avoid influencing test results by creating, opening and closing an extra socket before every (or at least one) test routine.
I was trying to code it in situ .
But if you don't mind the one extra socket operation, we can go ahead with your solution.

In this case however, I would put it on top of #9959 and add it to the test case setup function, rather than copy it into the code of every test function...

@0xc0170

This comment has been minimized.

Copy link
Member

commented Mar 13, 2019

Review astyle travis failing job

@0xc0170 0xc0170 added needs: work and removed needs: review labels Mar 13, 2019

@michalpasztamobica

This comment has been minimized.

Copy link
Contributor Author

commented Mar 13, 2019

@0xc0170 I plan to rewrite the whole code, according to Seppo's suggestion, but I need to get #9959 merged, so I can use the greentea test case setup function...

@michalpasztamobica michalpasztamobica force-pushed the michalpasztamobica:tcp_skip_if_unsupported branch from a206df9 to 690cfc5 Mar 25, 2019

@michalpasztamobica

This comment has been minimized.

Copy link
Contributor Author

commented Mar 25, 2019

@SeppoTakalo , unfortunately I could not put the SKIP macro inside the testcase's setup function, because the macro returns void and the wrapper needs to return a value.

But I did the second best thing possible - I added the function as you suggested and wrapped it with a simple macro, which got added to every test case.

To test the changes I locally broke LWIP::socket_open() function to return NSAPI_ERROR_UNSUPPORTED when TCP protocol is in use. All TCP and TLS tests printed the SKIP message and showed up as passing in the summary.

Reviewers, please review :)

@michalpasztamobica michalpasztamobica force-pushed the michalpasztamobica:tcp_skip_if_unsupported branch from 690cfc5 to 69b22a4 Mar 25, 2019

@cmonr

This comment has been minimized.

Copy link
Contributor

commented Mar 26, 2019

@michalpasztamobica Could you check the astyle job results?

@cmonr

This comment has been minimized.

Copy link
Contributor

commented Mar 26, 2019

Also, it looks like the preceeding PR has been merged.

@michalpasztamobica michalpasztamobica force-pushed the michalpasztamobica:tcp_skip_if_unsupported branch from 69b22a4 to 48a3a3e Mar 26, 2019

@michalpasztamobica

This comment has been minimized.

Copy link
Contributor Author

commented Mar 26, 2019

astyle fixed and the preceding PR is indeed merged (although it turned out I cannot make use of it in the end).
@SeppoTakalo , I'll be grateful for your approval.

@cmonr

This comment has been minimized.

Copy link
Contributor

commented Mar 26, 2019

CI started

@cmonr cmonr added needs: CI and removed needs: work labels Mar 26, 2019

cmonr added a commit to cmonr/mbed-os that referenced this pull request Mar 26, 2019

Merge pull request ARMmbed#10037 from michalpasztamobica/tcp_skip_if_…
…unsupported

TCP/TLS Socket tests will skip if TCP is not supported
@mbed-ci

This comment has been minimized.

Copy link

commented Mar 26, 2019

Test run: SUCCESS

Summary: 6 of 6 test jobs passed
Build number : 1
Build artifacts

@cmonr cmonr added needs: review and removed needs: CI labels Mar 26, 2019

@cmonr

This comment has been minimized.

Copy link
Contributor

commented Mar 27, 2019

@SeppoTakalo @michalpasztamobica Should we wait on others to chime in?

@mtomczykmobica
Copy link
Contributor

left a comment

Looka good.

@michalpasztamobica

This comment has been minimized.

Copy link
Contributor Author

commented Mar 27, 2019

@cmonr , I think there was enough time, I'd suggest to merge in.

@michalpasztamobica michalpasztamobica force-pushed the michalpasztamobica:tcp_skip_if_unsupported branch from 48a3a3e to 09183c9 Mar 28, 2019

@cmonr

cmonr approved these changes Mar 28, 2019

Copy link
Contributor

left a comment

Thank you!

@cmonr cmonr added needs: CI and removed needs: review labels Mar 28, 2019

@cmonr

This comment has been minimized.

Copy link
Contributor

commented Mar 28, 2019

CI started

@mbed-ci

This comment has been minimized.

Copy link

commented Mar 29, 2019

Test run: SUCCESS

Summary: 6 of 6 test jobs passed
Build number : 2
Build artifacts

@cmonr cmonr added ready for merge and removed needs: CI labels Mar 29, 2019

@adbridge adbridge merged commit 0f6a835 into ARMmbed:master Mar 29, 2019

21 checks passed

continuous-integration/jenkins/pr-head This commit looks good
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
jenkins-ci/build-ARMC5 Success
Details
jenkins-ci/build-ARMC6 Success
Details
jenkins-ci/build-GCC_ARM Success
Details
jenkins-ci/build-IAR8 Success
Details
jenkins-ci/dynamic-memory-usage RTOS ROM(+0 bytes) RAM(+0 bytes)
Details
jenkins-ci/greentea-test Success
Details
travis-ci/astyle Local astyle testing has passed
Details
travis-ci/docs Local docs testing has passed
Details
travis-ci/doxy-spellcheck Local doxy-spellcheck testing has passed
Details
travis-ci/events Passed, runtime is 10103 cycles (+1112 cycles)
Details
travis-ci/gitattributestest Local gitattributestest testing has passed
Details
travis-ci/include_check Local include_check testing has passed
Details
travis-ci/licence_check Local licence_check testing has passed
Details
travis-ci/littlefs Passed, code size is 8408B (+0.00%)
Details
travis-ci/psa-autogen Local psa-autogen testing has passed
Details
travis-ci/tools-py2.7 Local tools-py2.7 testing has passed
Details
travis-ci/tools-py3.5 Local tools-py3.5 testing has passed
Details
travis-ci/tools-py3.6 Local tools-py3.6 testing has passed
Details
travis-ci/tools-py3.7 Local tools-py3.7 testing has passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.