Publish tests documentation for DNS, Emac, NetworkInterface and Wifi #9630

michalpasztamobica · 2019-02-06T11:49:42Z

Description

Documentation moved from internal Confluence pages.

Pull request type

[ ] Fix
[ ] Refactor
[ ] Target update
[ ] Functionality change
[x] Docs update
[ ] Test update
[ ] Breaking change

Reviewers

@SeppoTakalo
@VeijoPesonen
@KariHaapalehto
@mtomczykmobica
@tymoteuszblochmobica
@kjbracey-arm

ciarmcom · 2019-02-06T12:00:41Z

@michalpasztamobica, thank you for your changes.
@tymoteuszblochmobica @kjbracey-arm @VeijoPesonen @SeppoTakalo @KariHaapalehto @mtomczykmobica @ARMmbed/mbed-os-ipcore @ARMmbed/mbed-os-test @ARMmbed/mbed-os-maintainers please review.

VeijoPesonen · 2019-02-06T12:41:40Z

TESTS/netsocket/README.md

@@ -18,6 +18,7 @@ The target for this plan is to test:
 -   [UDPSocket](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/UDPSocket.h).
 -   [TCPSocket](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/TCPSocket.h).
 -   [TLSSocket](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/TLSSocket.h).
+-   [DNS](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/DNS.h)

 Reference documentation: https://os.mbed.com/docs/latest/reference/network-socket.html


OK, not part of your changes but I guess this should be changed to https://os.mbed.com/docs/mbed-os/latest/apis/network-socket.html . And maybe also convert it to proper link named as "Network Socket Documentation" - or something similar

Yes! Please fix the link.

VeijoPesonen · 2019-02-06T12:48:17Z

TESTS/network/interface/README.md

+Test environment
+----------------
+
+-   [Greentea](https://github.com/ARMmbed/greentea)


Superseded by https://github.com/ARMmbed/mbed-os-tools

Perhaps link to this https://os.mbed.com/docs/mbed-os/v5.11/tools/greentea-testing-applications.html

Replace "v5.11" with "latest"

VeijoPesonen · 2019-02-06T12:51:15Z

TESTS/network/wifi/README.md

+-   Channels to be used must be different for both APs. For secure on channel number is later referred as `<ch:secure>` and for unsecure on `<ch:unsecure>`
+-   MAC addresses of Wifi APs must be known. Later referred as `<mac:secure>` and `<mac:unsecure>`
+
+**NOTE:�**Echo server is referred here as it is a requirement for running Socket API tests. It is not directly used by test cases defined in this document.


Does not render properly

VeijoPesonen · 2019-02-06T12:57:17Z

TESTS/network/wifi/README.md

+
+**Preconditions:**
+
+1.  -


VeijoPesonen · 2019-02-06T12:58:09Z

TESTS/network/wifi/README.md

+
+This test case is to test whether the driver accepts valid credentials and reject ones that are not valid.
+
+**�Preconditions:**


Does not render correctly.

VeijoPesonen · 2019-02-06T13:00:28Z

@melwee01 would you also like to take a look if you have time?

VeijoPesonen · 2019-02-06T13:08:05Z

TESTS/network/interface/README.md

+----------------
+
+-   Mbed OS.
+-   Standard Mbed OS development tools as described in https://os.mbed.com/docs/latest/tools/index.html.


Could be turned into a proper link like on line 9

michalpasztamobica · 2019-02-06T13:24:20Z

Thanks for your feedback, @VeijoPesonen . Alle remarks fixed.
Is there some smart way to see the rendered output of the modified files? I am locally using a Chrome "Markdown Preview Plus" plugin, but I can see it renders differently from github. I am especially worried about the tables...

SeppoTakalo

Small changes requested.

SeppoTakalo · 2019-02-06T12:56:45Z

TESTS/netsocket/README.md

@@ -1678,3 +1693,308 @@ Subset for driver test
 ### For socket layer driver (AT-driven, external IP stack):

 All Socket, UDPSocket, TCPSocket and TLSSocket testcases.
+


Move these before the "Subset for driver test" section.

Otherwise the formatting seems OK.

SeppoTakalo · 2019-02-06T12:57:05Z

TESTS/netsocket/README.md

@@ -18,6 +18,7 @@ The target for this plan is to test:
 -   [UDPSocket](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/UDPSocket.h).
 -   [TCPSocket](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/TCPSocket.h).
 -   [TLSSocket](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/TLSSocket.h).
+-   [DNS](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/DNS.h)

 Reference documentation: https://os.mbed.com/docs/latest/reference/network-socket.html


Yes! Please fix the link.

SeppoTakalo · 2019-02-06T12:58:41Z

TESTS/network/interface/README.md

+Test environment
+----------------
+
+-   [Greentea](https://github.com/ARMmbed/greentea)


Perhaps link to this https://os.mbed.com/docs/mbed-os/v5.11/tools/greentea-testing-applications.html

Replace "v5.11" with "latest"

SeppoTakalo · 2019-02-06T13:01:40Z

TESTS/network/wifi/README.md

+| 5   | WIFI_GET_RSSI                           |                            | SHOULD   |
+| 6   | WIFI_CONNECT_PARAMS_NULL                |                            | MUST     |
+| 7   | WIFI_CONNECT_PARAMS_VALID_UNSECURE      |                            | MUST     |
+| 8   | WIFI_CONNECT_PARAMS_VALID_SECURE        | With secruity type:        |          |


secruity change to security

SeppoTakalo · 2019-02-06T13:18:50Z

TESTS/network/wifi/README.md

+
+**Preconditions:**
+
+1.  -


1. - renders to

1. o

So use 1. none instead.

SeppoTakalo · 2019-02-06T13:20:47Z

TESTS/network/wifi/README.md

+| ssid=NULL  | pass=NULL                      | security= NSAPI_SECURITY_NONE    | NSAPI_ERROR_PARAMETER                               |
+| ssid="OK"  | pass=NULL                      | security=NSAPI_SECURITY_WPA_WPA2 | NSAPI_ERROR_PARAMETER                               |
+| ssid="OK"  | pass=NULL                      | security=NSAPI_SECURITY_WEP      | NSAPI_ERROR_PARAMETER                               |
+|            |                                |                                  | because password is missing.                        |


These two line boxes are not rendering very well.
Just combine the content into one box, like | NSAPI_ERROR_PARAMETER because password is missing |

Markdown will then split the lines.

Or you can try <br /> HTML code to split the lines manually.

SeppoTakalo · 2019-02-06T13:23:47Z

TESTS/network/wifi/README.md

+First call should return zero, second call should return negative number between -10 and -100.
+
+**
+**


What are these extra markings? Remove...

Probably some artifacts from pandoc...

SeppoTakalo · 2019-02-06T13:24:30Z

TESTS/network/wifi/README.md

+`scan()` returns positive number that is higher or equivalent of two. (>=2).
+
+**
+**


Remove these extra markings.

michalpasztamobica · 2019-02-06T13:43:05Z

Thanks, @SeppoTakalo , remarks fixed.

SeppoTakalo · 2019-02-06T13:47:11Z

Not all remarks..

Change the order like I proposed earlier..

First DNS testcases, then "Subset for driver ...."

SeppoTakalo · 2019-02-06T13:50:39Z

TESTS/network/wifi/README.md

+
+**Expected result:**
+
+| Parameters |                                |                                  | Expected return code                                |


Can we change this table to

ssid password security Expected return code

The we can remove those ssid= and pass= which fill up the table and make it harder to read.

michalpasztamobica · 2019-02-06T13:56:06Z

I hope now I got it right...
This...

### For socket layer driver (AT-driven, external IP stack):

All Socket, UDPSocket, TCPSocket and TLSSocket testcases.

... part belongs to the sockets, right? (So it should go before the DNS testcases)? Or should we add DNS to the list and move it down to the end of document?

SeppoTakalo · 2019-02-06T13:58:51Z

No, that part belongs to bottom of the file as it was in the Confluence.

No we are just appending Socket and DNS test cases together so that it is a long list of testcases that live within this directory.

Then after the testcase listing. there would be subsets proposed.

VeijoPesonen · 2019-02-06T13:59:18Z

Thanks for your feedback, @VeijoPesonen . Alle remarks fixed.
Is there some smart way to see the rendered output of the modified files? I am locally using a Chrome "Markdown Preview Plus" plugin, but I can see it renders differently from github. I am especially worried about the tables...

One way is to just create a new branch to your fork and check the end result from your browser. Nothing better comes to my mind. Once the changes have already landed to a PR just use the "View File"-button.

VeijoPesonen · 2019-02-06T14:12:13Z

TESTS/netsocket/README.md


-Reference documentation: https://os.mbed.com/docs/latest/reference/network-socket.html
+See [Network Socket Documentation](https://os.mbed.com/docs/mbed-os/latest/apis/network-socket.html) for reference.


Would you please do the same for the line 29.

Documentation moved from internal Confluence pages.

Edit file, mostly for inclusion of articles.

VeijoPesonen · 2019-02-08T07:30:34Z

Thanks @michalpasztamobica and @AnotherButler.

SeppoTakalo · 2019-02-08T10:54:13Z

I would be happy to use a template that says **Preconditions:** as there can be zero or more conditions.

Does not much change the message if we tailor the template per test case for titles to be plural or singular.

michalpasztamobica · 2019-02-11T13:43:04Z

@0xc0170 , is there any other approval required here? @ARMmbed/mbed-docs ?

0xc0170 · 2019-02-11T13:45:15Z

Yes, @AnotherButler to approve. I can schedule CI now as it was already edited by docs team

0xc0170 · 2019-02-11T13:46:50Z

CI started

0xc0170 · 2019-02-11T13:53:30Z

Just to confirm, this update is targetting 5.11.x ?

michalpasztamobica · 2019-02-11T13:56:24Z

Not sure. @SeppoTakalo , @TuomoHautamaki , can you help us here? Should this documentation publishing go into 5.11.x?

0xc0170 · 2019-02-11T14:03:03Z

CI completed, waiting for docs to approve and ready for merge

SeppoTakalo · 2019-02-11T14:09:03Z

It does not matter whether it goes out in 5.11 or 5.12 as we always link into the master branch.

Basically I think that it can be merged in the 5.11 branch, because it describes existing test cases. But there is no code changes, so I don't see the point.

adbridge · 2019-02-12T15:12:27Z

@ARMmbed/mbed-docs Could you please still review this and mark any comments so that they can come into a follow on PR - this got prematurely merged...

AnotherButler

I requested several global edits because I didn't want to paste them every time.

AnotherButler · 2019-02-12T21:06:40Z

TESTS/netsocket/README.md

+
+**Expected result:**
+
+Return value is NSAPI_ERROR_DNS_FAILURE for first, third and fifth host name. Return value is NSAPI_ERROR_OK and IP address is obtained for second and fourth host name.


Please add code formatting. I'm not sure what this means.

AnotherButler · 2019-02-12T21:07:20Z

TESTS/netsocket/README.md

+
+**Test steps:**
+
+1.  Call `gethostbyname()` in a row 6 times with a different host names. Host names shall not be found from cache. First, third and fifth host name shall be invalid.


"...with different host names."

Also, I'm not sure what the rest of this line means.

AnotherButler · 2019-02-12T21:07:29Z

TESTS/netsocket/README.md

+
+**Preconditions:**
+
+1.  Network interface is initialised.


"initialized"

AnotherButler · 2019-02-12T21:07:55Z

TESTS/netsocket/README.md

+
+**Description:**
+
+Verify that DNS failure error is provided for invalid hosts. Call `NetworkInterface::gethostbyname()` in a row 6 times with a different host names. First, third and fifth host name shall be invalid.


"...with different host names."

I'm not sure what the rest of this line means.

AnotherButler · 2019-02-12T21:08:34Z

TESTS/netsocket/README.md

+
+**Expected result:**
+
+Return value is NSAPI_ERROR_OK and IP address is obtained from the function call 5 times. First request shall complete slower than the requests made after it (where the response is found from cache).


Please add code formatting.

"slower" -> "more slowly".

AnotherButler · 2019-02-12T21:16:03Z

TESTS/netsocket/README.md

+**Test steps:**
+
+1.  Call `gethostbyname_async()` in a row 6 times with a different host names. Host names shall not be found from cache.
+2.  Verify that last `gethostbyname_async()` operation is rejected since there is room only for 5 simultaneous operations.


"since" -> "because"

AnotherButler · 2019-02-12T21:16:31Z

TESTS/netsocket/README.md

+
+**Description:**
+
+Verify that the caching of DNS results works correctly with simultaneous asynchronous DNS queries. Call `NetworkInterface::gethostbyname_async()` in a row 6 times with a different host names. Wait for all requests to complete and verify result. Cache shall contain at least one host name used in asynchronous request. This can be achieved e.g. by running test "Asynchronous DNS simultaneous" before this test and using same host names in this run.


"in asynchronous request. You can achieve this, for example, by running ..."

AnotherButler · 2019-02-12T21:17:37Z

TESTS/netsocket/README.md

+
+**Description:**
+
+Verify that non-asynchronous i.e. blocking DNS queries and asynchronous i.e. non-blocking queries work at the same time. Call `NetworkInterface::gethostbyname_async()`. Right after that make 6 non-asynchronous `NetworkInterface::gethostbyname()` calls with different host names.


"Verify nonasnchronous (in other words, blocking) DNS queries and asynchronous (in other words, nonblocking) queries ... after that, make 6 nonasynchronous..."

AnotherButler · 2019-02-12T21:18:22Z

TESTS/netsocket/README.md

+
+**Description:**
+
+Verify that asynchronous DNS query cancel works correctly. Call `NetworkInterface::gethostbyname_async()` in a row 6 times with a different host names. Cache shall contain 3 host names used in requests. This can be achieved e.g. by running test "Asynchronous DNS non-asynchronous and asynchronous" before this test and using same host names in this run. For each request that was given an unique id, call cancel. Verify that callback is not called for cancelled requests.


Please globally delete the hyphen in "nonasynchronous".
Please globally capitalize "ID".
Please globally delete the extra "l" in "canceled".

AnotherButler · 2019-02-12T21:18:32Z

TESTS/netsocket/README.md

+1.  Call `gethostbyname_async()` in a row 6 times with a different host names. Cache shall contain in maximum 3 host names used in requests.
+2.  Call `gethostbyname_async_cancel()` for each request that was given an unique id.
+3.  Verify that for cancelled requests, callback is not called.
+4.  Verify that for other request, callback is called.


cmonr · 2019-02-12T23:29:37Z

@SeppoTakalo @michalpasztamobica @ARMmbed/mbed-os-ipcore Would someone mind following up with the above suggested doc edits in a new PR?

michalpasztamobica · 2019-02-13T08:36:14Z

@AnotherButler , many thanks for your thorough review. I hope I addressed all remarks in #9698.

ciarmcom requested review from KariHaapalehto, kjbracey, mtomczykmobica, SeppoTakalo, tymoteuszblochmobica, VeijoPesonen and a team February 6, 2019 12:00

ciarmcom added the needs: review label Feb 6, 2019

NirSonnenschein requested a review from a team February 6, 2019 12:44

VeijoPesonen reviewed Feb 6, 2019

View reviewed changes

VeijoPesonen suggested changes Feb 6, 2019

View reviewed changes

VeijoPesonen reviewed Feb 6, 2019

View reviewed changes

michalpasztamobica force-pushed the publish_documentation branch from b02c36e to c0e7f59 Compare February 6, 2019 13:22

SeppoTakalo suggested changes Feb 6, 2019

View reviewed changes

michalpasztamobica force-pushed the publish_documentation branch from c0e7f59 to ade4211 Compare February 6, 2019 13:34

michalpasztamobica force-pushed the publish_documentation branch from ade4211 to ff81db6 Compare February 6, 2019 13:46

SeppoTakalo reviewed Feb 6, 2019

View reviewed changes

michalpasztamobica force-pushed the publish_documentation branch from ff81db6 to cf4b678 Compare February 6, 2019 13:56

VeijoPesonen suggested changes Feb 6, 2019

View reviewed changes

Publish tests documentation for DNS, Emac, NetworkInterface and Wifi

ea3d70e

Documentation moved from internal Confluence pages.

Amanda Butler added 2 commits February 7, 2019 15:52

Edit README.md

e9c44a8

Edit file, mostly for inclusion of articles.

Edit README.md

ae20556

Edit file, mostly for inclusion of articles.

VeijoPesonen approved these changes Feb 8, 2019

View reviewed changes

0xc0170 added needs: CI and removed needs: review labels Feb 11, 2019

0xc0170 added the release-version: 5.11.5 label Feb 11, 2019

0xc0170 added needs: review and removed needs: CI labels Feb 11, 2019

0xc0170 added release-version: 5.12.0-rc1 and removed release-version: 5.11.5 labels Feb 11, 2019

cmonr approved these changes Feb 11, 2019

View reviewed changes

cmonr added ready for merge and removed needs: review labels Feb 11, 2019

cmonr merged commit 799476d into ARMmbed:master Feb 12, 2019

cmonr removed the ready for merge label Feb 12, 2019

SeppoTakalo deleted the publish_documentation branch February 12, 2019 15:15

AnotherButler reviewed Feb 12, 2019

View reviewed changes

michalpasztamobica mentioned this pull request Feb 13, 2019

Netsocket documentation cleanup #9698

Merged


		This test case is to test whether the driver accepts valid credentials and reject ones that are not valid.

		�Preconditions:

		@@ -1678,3 +1693,308 @@ Subset for driver test
		### For socket layer driver (AT-driven, external IP stack):

		All Socket, UDPSocket, TCPSocket and TLSSocket testcases.


		Expected result:

		\| Parameters \| \| \| Expected return code \|


		Reference documentation: https://os.mbed.com/docs/latest/reference/network-socket.html
		See [Network Socket Documentation](https://os.mbed.com/docs/mbed-os/latest/apis/network-socket.html) for reference.


		Expected result:

		Return value is NSAPI_ERROR_DNS_FAILURE for first, third and fifth host name. Return value is NSAPI_ERROR_OK and IP address is obtained for second and fourth host name.


		Test steps:

		1. Call `gethostbyname()` in a row 6 times with a different host names. Host names shall not be found from cache. First, third and fifth host name shall be invalid.


		Description:

		Verify that DNS failure error is provided for invalid hosts. Call `NetworkInterface::gethostbyname()` in a row 6 times with a different host names. First, third and fifth host name shall be invalid.


		Expected result:

		Return value is NSAPI_ERROR_OK and IP address is obtained from the function call 5 times. First request shall complete slower than the requests made after it (where the response is found from cache).


		Description:

		Verify that the caching of DNS results works correctly with simultaneous asynchronous DNS queries. Call `NetworkInterface::gethostbyname_async()` in a row 6 times with a different host names. Wait for all requests to complete and verify result. Cache shall contain at least one host name used in asynchronous request. This can be achieved e.g. by running test "Asynchronous DNS simultaneous" before this test and using same host names in this run.


		Description:

		Verify that non-asynchronous i.e. blocking DNS queries and asynchronous i.e. non-blocking queries work at the same time. Call `NetworkInterface::gethostbyname_async()`. Right after that make 6 non-asynchronous `NetworkInterface::gethostbyname()` calls with different host names.


		Preconditions:

		1. -


		Preconditions:

		1. -


		Description:

		Verify that asynchronous DNS query cancel works correctly. Call `NetworkInterface::gethostbyname_async()` in a row 6 times with a different host names. Cache shall contain 3 host names used in requests. This can be achieved e.g. by running test "Asynchronous DNS non-asynchronous and asynchronous" before this test and using same host names in this run. For each request that was given an unique id, call cancel. Verify that callback is not called for cancelled requests.

Publish tests documentation for DNS, Emac, NetworkInterface and Wifi #9630

Publish tests documentation for DNS, Emac, NetworkInterface and Wifi #9630

Conversation

michalpasztamobica commented Feb 6, 2019

Description

Pull request type

Reviewers

ciarmcom commented Feb 6, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

VeijoPesonen commented Feb 6, 2019

Choose a reason for hiding this comment

michalpasztamobica commented Feb 6, 2019

SeppoTakalo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michalpasztamobica commented Feb 6, 2019

SeppoTakalo commented Feb 6, 2019

Choose a reason for hiding this comment

michalpasztamobica commented Feb 6, 2019

SeppoTakalo commented Feb 6, 2019

VeijoPesonen commented Feb 6, 2019

VeijoPesonen Feb 6, 2019 • edited Loading

Choose a reason for hiding this comment

VeijoPesonen commented Feb 8, 2019

SeppoTakalo commented Feb 8, 2019 • edited Loading

michalpasztamobica commented Feb 11, 2019

0xc0170 commented Feb 11, 2019

0xc0170 commented Feb 11, 2019

0xc0170 commented Feb 11, 2019

michalpasztamobica commented Feb 11, 2019

0xc0170 commented Feb 11, 2019

SeppoTakalo commented Feb 11, 2019

adbridge commented Feb 12, 2019

AnotherButler left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmonr commented Feb 12, 2019

michalpasztamobica commented Feb 13, 2019 • edited Loading

VeijoPesonen Feb 6, 2019 •

edited

Loading

SeppoTakalo commented Feb 8, 2019 •

edited

Loading

michalpasztamobica commented Feb 13, 2019 •

edited

Loading