From ea3d70e0b10093a1e4e311ef9d9122072990d53e Mon Sep 17 00:00:00 2001 From: Michal Paszta Date: Wed, 6 Feb 2019 13:17:43 +0200 Subject: [PATCH 1/4] Publish tests documentation for DNS, Emac, NetworkInterface and Wifi Documentation moved from internal Confluence pages. --- TESTS/netsocket/README.md | 326 ++++++++++++++++++- TESTS/network/emac/README.md | 210 ++++++++++--- TESTS/network/interface/README.md | 166 ++++++++++ TESTS/network/wifi/README.md | 506 ++++++++++++++++++++++++++++++ 4 files changed, 1153 insertions(+), 55 deletions(-) create mode 100644 TESTS/network/interface/README.md create mode 100644 TESTS/network/wifi/README.md diff --git a/TESTS/netsocket/README.md b/TESTS/netsocket/README.md index 7c45460677b..654a25b2e4c 100644 --- a/TESTS/netsocket/README.md +++ b/TESTS/netsocket/README.md @@ -18,14 +18,15 @@ The target for this plan is to test: - [UDPSocket](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/UDPSocket.h). - [TCPSocket](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/TCPSocket.h). - [TLSSocket](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/TLSSocket.h). +- [DNS](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/DNS.h). -Reference documentation: https://os.mbed.com/docs/latest/reference/network-socket.html +See [Network Socket Documentation](https://os.mbed.com/docs/mbed-os/latest/apis/network-socket.html) for reference. Tools to use ---------------- - Mbed OS. -- Standard Mbed OS development tools as described in https://os.mbed.com/docs/latest/tools/index.html. +- Standard Mbed OS development tools as described in [Arm Mbed tools overview](https://os.mbed.com/docs/latest/tools/index.html). - Test server. These test cases themselves do not require any special tooling, other than @@ -227,6 +228,20 @@ pass the test if the driver implements the feature in question. | 60 | TLSSOCKET_SIMULTANEOUS_TEST | SHOULD | | 61 | TLSSOCKET_ECHOTEST_BURST | SHOULD | | 62 | TLSSOCKET_ECHOTEST_BURST_NONBLOCK | SHOULD | +| 63 | ASYNCHRONOUS_DNS | MUST | +| 64 | ASYNCHRONOUS_DNS_CACHE | MUST | +| 65 | ASYNCHRONOUS_DNS_CANCEL | MUST | +| 66 | ASYNCHRONOUS_DNS_EXTERNAL_EVENT_QUEUE | MUST | +| 67 | ASYNCHRONOUS_DNS_INVALID_HOST | MUST | +| 68 | ASYNCHRONOUS_DNS_NON_ASYNC_AND_ASYNC | MUST | +| 69 | ASYNCHRONOUS_DNS_SIMULTANEOUS_CACHE | MUST | +| 70 | ASYNCHRONOUS_DNS_SIMULTANEOUS_REPEAT | MUST | +| 71 | ASYNCHRONOUS_DNS_SIMULTANEOUS | MUST | +| 72 | ASYNCHRONOUS_DNS_TIMEOUTS | MUST | +| 73 | SYNCHRONOUS_DNS | MUST | +| 74 | SYNCHRONOUS_DNS_CACHE | MUST | +| 75 | SYNCHRONOUS_DNS_INVALID_HOST | MUST | +| 76 | SYNCHRONOUS_DNS_MULTIPLE | MUST | @@ -1654,6 +1669,311 @@ through a dedicated socket Echo server returns data to both threads and received data matches to send data. The additional thread isn't stopped prematurely. +Test cases for DNS class +--------------------------- + +### ASYNCHRONOUS_DNS + +**Description:** + +Verify basic functionality of asynchronous DNS. Call `NetworkInterface::gethostbyname_async()` with a valid host name and verify result. + +**Preconditions:** + +1. Network interface is initialised. +2. Network connection is up. + +**Test steps:** + +1. Call `gethostbyname_async()` with a valid host name and a callback. +2. Verify that callback is called with correct parameters. + +**Expected result:** + +Callback is called with NSAPI_ERROR_OK and IP address. + +### ASYNCHRONOUS_DNS_SIMULTANEOUS + +**Description:** + +Verify that simultaneous asynchronous DNS queries work correctly. Call `NetworkInterface::gethostbyname_async()` in a row 6 times with a different host names. Wait for all requests to complete and verify result. Cache shall not contain host names used in asynchronous request. + +**Preconditions:** + +1. Network interface is initialised. +2. Network connection is up. + +**Test steps:** + +1. Call `gethostbyname_async()` in a row 6 times with a different host names. Host names shall not be found from cache. +2. Verify that last `gethostbyname_async()` operation is rejected since there is room only for 5 simultaneous operations. +3. Verify that callback is called with correct parameters 5 times. + +**Expected result:** + +Sixth `gethostbyname_async()` is rejected. Callback is called with NSAPI_ERROR_OK and IP address 5 times. + +### ASYNCHRONOUS_DNS_SIMULTANEOUS_CACHE + +**Description:** + +Verify that the caching of DNS results works correctly with simultaneous asynchronous DNS queries. Call `NetworkInterface::gethostbyname_async()` in a row 6 times with a different host names. Wait for all requests to complete and verify result. Cache shall contain at least one host name used in asynchronous request. This can be achieved e.g. by running test "Asynchronous DNS simultaneous" before this test and using same host names in this run. + +**Preconditions:** + +1. Network interface is initialised. +2. Network connection is up. + +**Test steps:** + +1. Call `gethostbyname_async()` in a row 6 times with a different host names. At least one host name shall be found from cache. +2. Verify that callback is called with correct parameters 6 times. + +**Expected result:** + +Callback is called with NSAPI_ERROR_OK and IP address 6 times. + +### ASYNCHRONOUS_DNS_CACHE + +**Description:** + +Verify that the caching of DNS results works correctly. Call `NetworkInterface::gethostbyname_async()` 5 times with the same host name and verify result after each request. For first request, cache shall not contain the host name. Verify that first request completes slower than the requests made after it (where the response is found from cache). + +**Preconditions:** + +1. Network interface is initialised. +2. Network connection is up. + +**Test steps:** + +1. Call `gethostbyname_async()` with a host name. For first request, host name shall not be found from cache. +2. Verify that callback is called with correct parameters. +3. Repeat the sequence 4 times using the same host name. +4. For each request, calculate how long time it takes for DNS query to complete. + +**Expected result:** + +Callback is called with NSAPI_ERROR_OK and IP address 5 times. First request shall complete slower than the requests made after it (where the response is found from cache). + +### ASYNCHRONOUS_DNS_NON_ASYNC_AND_ASYNC + +**Description:** + +Verify that non-asynchronous i.e. blocking DNS queries and asynchronous i.e. non-blocking queries work at the same time. Call `NetworkInterface::gethostbyname_async()`. Right after that make 6 non-asynchronous `NetworkInterface::gethostbyname()` calls with different host names. + +**Preconditions:** + +1. Network interface is initialised. +2. Network connection is up. + +**Test steps:** + +1. Call `gethostbyname_async()` with a host name. Host name shall not be found from cache. +2. Call `gethostbyname()` 6 times with a different host names (none of the names shall be same as in step 1). +3. Verify that each `gethostbyname()` returns success. +4. Verify that the asynchronous callback is called with correct parameters. + +**Expected result:** + +All operations shall return NSAPI_ERROR_OK and IP address. + +### ASYNCHRONOUS_DNS_CANCEL + +**Description:** + +Verify that asynchronous DNS query cancel works correctly. Call `NetworkInterface::gethostbyname_async()` in a row 6 times with a different host names. Cache shall contain 3 host names used in requests. This can be achieved e.g. by running test "Asynchronous DNS non-asynchronous and asynchronous" before this test and using same host names in this run. For each request that was given an unique id, call cancel. Verify that callback is not called for cancelled requests. + +**Preconditions:** + +1. Network interface is initialised. +2. Network connection is up. + +**Test steps:** + +1. Call `gethostbyname_async()` in a row 6 times with a different host names. Cache shall contain in maximum 3 host names used in requests. +2. Call `gethostbyname_async_cancel()` for each request that was given an unique id. +3. Verify that for cancelled requests, callback is not called. +4. Verify that for other request, callback is called. + +**Expected result:** + +Callback shall be called only for requests that were not cancelled. + +### ASYNCHRONOUS_DNS_EXTERNAL_EVENT_QUEUE + +**Description:** + +Verify that providing an external event queue works correctly. Define a thread and an event queue running on it. Define a DNS call in callback function that uses the event queue (call_in_callback_cb_t). Enable external event queue. Call `NetworkInterface::gethostbyname_async()` in a row 6 times with a different host names. + +**Preconditions:** + +1. Network interface is initialised. +2. Network connection is up. + +**Test steps:** + +1. Define a thread and an event queue running on it. +2. Define a DNS call in callback function that uses the event queue (call_in_callback_cb_t). +3. Start thread and event queue. +4. Set DNS callback function using the `nsapi_dns_call_in_set()` call. +5. Call `gethostbyname_async()` in a row 6 times with a different host names. Host names shall not be found from cache. +6. Verify that last `gethostbyname_async()` operation is rejected since there is room only for 5 simultaneous operations. +7. Verify that callback is called with correct parameters 5 times. +8. Restore default DNS callback function using the `nsapi_dns_call_in_set()` call. + +**Expected result:** + +Sixth `gethostbyname_async()` is rejected. Callback is called with NSAPI_ERROR_OK and IP address 5 times. + +### ASYNCHRONOUS_DNS_INVALID_HOST + +**Description:** + +Verify that DNS failure error is provided for invalid hosts. Call `NetworkInterface::gethostbyname_async()` in a row 6 times with a different host names. First, third and fifth host name shall be invalid. + +**Preconditions:** + +1. Network interface is initialised. +2. Network connection is up. + +**Test steps:** + +1. Call `gethostbyname_async()` in a row 6 times with a different host names. Host names shall not be found from cache. First, third and fifth host name shall be invalid. +2. Verify that last `gethostbyname_async()` operation is rejected since there is room only for 5 simultaneous operations. +3. Verify that callback is called with correct parameters 5 times. + +**Expected result:** + +Sixth `gethostbyname_async()` is rejected. Callback is called with NSAPI_ERROR_DNS_FAILURE for first, third and fifth host name. Callback is called with NSAPI_ERROR_OK and IP address for second and fourth host name. + +### ASYNCHRONOUS_DNS_TIMEOUTS + +**Description:** + +Test DNS timeouts using an external event queue that is modified to timeout the events faster that standard event queue. In this test event queue shall not delay events, instead it handles those immediately. Call `NetworkInterface::gethostbyname_async()` in a row 6 times with a different host names. All or some of the request shall timeout and timeout return value is returned. + +**Preconditions:** + +1. Network interface is initialised. +2. Network connection is up. + +**Test steps:** + +1. Define a thread and an event queue running on it. +2. Define a DNS call in callback function that uses the event queue (call_in_callback_cb_t). Callback function shall not delay callbacks; instead it shall handle those immediately. +3. Start thread and event queue. +4. Set DNS callback function using the `nsapi_dns_call_in_set()` call. +5. Call `gethostbyname_async()` in a row 6 times with a different host names. Host names shall not be found from cache. +6. Verify that last `gethostbyname_async()` operation is rejected since there is room only for 5 simultaneous operations. +7. Verify that callback is called with correct parameters 5 times. + +**Expected result:** + +Sixth `gethostbyname_async()` is rejected. At least for one operation, callback is called with NSAPI_ERROR_TIMEOUT value. + +### ASYNCHRONOUS_DNS_SIMULTANEOUS_REPEAT + +**Description:** + +Verify that simultaneous asynchronous DNS queries work correctly when repeated in sequence. Call `NetworkInterface::gethostbyname_async()` in a row 5 times with a different host names. Wait for all requests to complete and verify result. Repeat the procedure 100 times. + +**Preconditions:** + +1. Network interface is initialised. +2. Network connection is up. + +**Test steps:** + +1. Call `gethostbyname_async()` in a row 5 times with a different host names. +2. Verify that callback is called with correct parameters 5 times for first operation. +3. Repeat 100 times steps 1 to 2. + +**Expected result:** + +Callback is called with NSAPI_ERROR_OK and IP address 5 times for every repeat. + +### SYNCHRONOUS_DNS + +**Description:** + +Verify basic functionality of synchronous DNS. Call `NetworkInterface::gethostbyname()` with a valid host name and verify result. + +**Preconditions:** + +1. Network interface is initialised. +2. Network connection is up. + +**Test steps:** + +1. Call `gethostbyname()` with a valid host name. +2. Verify that address was resolved and return value was valid. + +**Expected result:** + +Return value is NSAPI_ERROR_OK and IP address is obtained from the function call. + +### SYNCHRONOUS_DNS_MULTIPLE + +**Description:** + +Verify basic functionality of synchronous DNS. Call `NetworkInterface::gethostbyname()` with a list of 6 host names and verify result. + +**Preconditions:** + +1. Network interface is initialised. +2. Network connection is up. + +**Test steps:** + +1. Call `gethostbyname()` with a list of 6 host names +2. Verify that each of the addresses was resolved and return value was valid. + +**Expected result:** + +Return value is NSAPI_ERROR_OK and IP addresses are obtained from the function call. + +### SYNCHRONOUS_DNS_CACHE + +**Description:** + +Verify that the caching of DNS results works correctly. Call `NetworkInterface::gethostbyname()` 5 times with the same host name and verify result after each request. For first request, cache shall not contain the host name. Verify that first request completes slower than the requests made after it (where the response is found from cache). + +**Preconditions:** + +1. Network interface is initialised. +2. Network connection is up. + +**Test steps:** + +1. Call `gethostbyname()` with a host name. For the first request, host name shall not be found from cache. +2. Verify that address was resolved and return value was valid. +3. Repeat the sequence 4 times using the same host name. +4. For each request, calculate how long time it takes for DNS query to complete. + +**Expected result:** + +Return value is NSAPI_ERROR_OK and IP address is obtained from the function call 5 times. First request shall complete slower than the requests made after it (where the response is found from cache). + +### SYNCHRONOUS_DNS_INVALID_HOST + +**Description:** + +Verify that DNS failure error is provided for invalid hosts. Call `NetworkInterface::gethostbyname()` in a row 6 times with a different host names. First, third and fifth host name shall be invalid. + +**Preconditions:** + +1. Network interface is initialised. +2. Network connection is up. + +**Test steps:** + +1. Call `gethostbyname()` in a row 6 times with a different host names. Host names shall not be found from cache. First, third and fifth host name shall be invalid. +2. Verify that return value was valid and for valid hostnames the address was resolved 6 times. + +**Expected result:** + +Return value is NSAPI_ERROR_DNS_FAILURE for first, third and fifth host name. Return value is NSAPI_ERROR_OK and IP address is obtained for second and fourth host name. + Subset for driver test ---------------------- @@ -1677,4 +1997,4 @@ Subset for driver test ### For socket layer driver (AT-driven, external IP stack): -All Socket, UDPSocket, TCPSocket and TLSSocket testcases. +All Socket, UDPSocket, TCPSocket and TLSSocket testcases. \ No newline at end of file diff --git a/TESTS/network/emac/README.md b/TESTS/network/emac/README.md index 0a63709bb8b..254627c0b11 100644 --- a/TESTS/network/emac/README.md +++ b/TESTS/network/emac/README.md @@ -37,6 +37,26 @@ For Wi-Fi set the `json` configuration to: For Wi-Fi you also need to configure Wi-Fi SSID and security options to the configuration file. + +Test case priorities +-------------------- + +Please refer to the following table for priorities of test cases. Priorities +are labeled as MUST and SHOULD. MUST means this is a requirement and +therefore mandatory to pass the test. SHOULD means it is recommended to +pass the test if the driver implements the feature in question. + +| | Test case | Priority | +|-----|-----------------------------------------|----------| +| 1 | EMAC initialize | MUST | +| 2 | EMAC broadcast | MUST | +| 3 | EMAC unicast | MUST | +| 4 | EMAC unicast frame length | MUST | +| 5 | EMAC unicast burst | MUST | +| 6 | EMAC unicast long | MUST | +| 7 | EMAC multicast filter | MUST | +| 8 | EMAC memory | MUST | + ## Example commands ### CTP echo server @@ -107,90 +127,176 @@ To verify whether the echo server is receiving CTP Ethernet frames, enable `echo ### EMAC initialize -The test case initializes the EMAC driver and the test network stack. +**Description:** -The EMAC test environment uses the test network stack as the default stack. To enable the stack, set the `nsapi.default-stack` option in the `json` file of the test environment application to value `TEST`. +Test case initializes and connects the EMAC driver and the test network stack. -The test network stack is a bare minimum implementation and has the functionality needed to set up the network interface. The test network stack is defined in the `emac_TestNetworkStack.h` and `emac_TestNetworkStack.cpp` files. The stack uses the test memory manager for the EMAC. The test memory manager is defined in the `emac_TestMemoryManager.h` and `emac_TestMemoryManager.cpp` files. Message buffers sent to the EMAC in `link_out()` are allocated from the buffer pool of the test memory manager. The test memory manager pool allocation unit (buffer size) is 610 bytes. +**Preconditions:** -The initialization test constructs the network interface and connects to it. The test network stack and the EMAC are bound to the network interface using `get_default_instance()` calls to the stack and to the EMAC. +1. Device is ready to be connected (Ethernet cable is plugged or WIFI base station is available). -After the construction, the network interface is connected. A connect call triggers a set up call to the test network stack. The set up call triggers a call to `emac_if_init()` function in the EMAC initialization test case. +**Test steps:** -The `emac_if_init()` function of the test case configures and powers up the EMAC. +1. Constructs the network interface -The configuration steps are: +2. Connects the network interface -* Setting the test memory manager for the EMAC. -* Setting the EMAC link input and state callbacks to call the test environment input and state callback handlers. -* Reading and setting the Ethernet MAC address. +**Expected result:** + +Network interface is connected. ### EMAC broadcast -1. Sends three CTP broadcast messages (100 bytes each) -2. Waits for three seconds -3. Sends three CTP broadcast messages (60 bytes each). -4. Listens for the CTP echo server responses. -5. Stores the addresses of the echo servers if replies are received. +**Description:** + +Test case tests basic broadcast functionality and resolves CTP echo server MAC address. + +**Preconditions:** + +1. Network interface is connected. + +**Test steps:** -The test case passes if there are no responses from the echo server, but further test cases are skipped. +1. Sends three CTP broadcast messages (100 bytes each) +2. Waits for three seconds +3. Sends three CTP broadcast messages (60 bytes each). +4. Listens for the CTP echo server responses. +5. Stores the addresses of the echo servers if replies are received. + +**Expected result:** + +Echo server replies to broadcast messages. For each sent broadcast message reply is waited for three seconds. In case reply is not received sending is repeated six times before failing the test. ### EMAC unicast -1. Sends three CTP unicast messages (100 bytes each) to the CTP echo server. -2. Verifies that all are replied. +**Description:** + +Test case tests basic unicast functionality and that the CTP echo server replies to unicast messages. + +**Preconditions:** + +1. Network interface is connected. + +**Test steps:** + +1. Sends three CTP unicast messages (100 bytes each) to the CTP echo server. +2. Verifies that all are replied. + +**Expected result:** + +Echo server replies to unicast messages. For each sent unicast message reply is waited for three seconds. In case reply is not received sending is repeated five times before failing the test. ### EMAC unicast frame length - -1. Sends CTP unicast messages with Ethernet message length from 100 bytes to the maximum defined by the MTU of the EMAC with 50 bytes increments. -2. Verifies that all are replied. + +**Description:** + +Test case tests Ethernet frame lengths. + +**Preconditions:** + +1. Network interface is connected. + +**Test steps:** + +1. Sends CTP unicast messages with Ethernet message length from 100 bytes to the maximum defined by the MTU of the EMAC with 50 bytes increments. +2. Verifies that all are replied. + +**Expected result:** + +Echo server replies to messages that are sent to it. For each sent unicast message reply is waited for 500ms. In case reply is not received sending is repeated five times before failing the test. ### EMAC unicast burst - -1. Sends CTP unicast messages with Ethernet message length from 100 bytes to the maximum defined by the MTU of the EMAC with 50 bytes increments. -2. Repeats the sending 10 times. -3. Verifies that all are replied. + +**Description:** + +Test case tests Ethernet echoing at full speed. + +**Preconditions:** + +1. Network interface is connected. + +**Test steps:** + +1. Sends CTP unicast messages with Ethernet message length from 100 bytes to the maximum defined by the MTU of the EMAC with 50 bytes increments. +2. Repeats the sending 10 times. +3. Verifies that all are replied. + +**Expected result:** + +Echo server replies to messages that are sent to it. For each sent unicast message reply is waited for five seconds. In case reply is not received sending is repeated five times before failing the test. ### EMAC unicast long - -1. Sends CTP unicast messages with random Ethernet message length. -2. Repeats the sending 50000 times. -3. Verifies that all are replied. + +**Description:** + +Test case tests Ethernet echoing at full speed for an extended time. + +**Preconditions:** + +1. Network interface is connected. + +**Test steps:** + +1. Sends CTP unicast messages with random Ethernet message length. +2. Repeats the sending 50000 times. +3. Verifies that all are replied. + +**Expected result:** + +Echo server replies to messages that are sent to it. For each sent unicast message reply is waited for 350ms. In case reply is not received sending is repeated five times before failing the test. ### EMAC multicast filter - -Tests multicast filtering. Multicast filtering is an optional feature for the EMAC. The test does not fail if filtering is not implemented. -The multicast testing requests the CTP echo server to forward the CTP messages to a specified multicast address as the destination address. +**Description:** + +Test case tests multicast filtering. Multicast filtering is an optional feature for the EMAC. The test does not fail if filtering is not implemented. + +**Preconditions:** + +1. Network interface is connected. + +**Test steps:** -Test steps: +1. Using unicast, verify that the echo server responses are received. +2. Set the IPv6 multicast filter address and the echo server reply (forward) address to different values. Check if the echo response is filtered. +3. Set the IPv6 multicast filter address and the echo server reply address to same value. Check that the response is not filtered. +4. Set the IPv4 multicast filter address and the echo server reply address to different values. Check if the response is filtered. +5. Set the IPv4 multicast filter address and the echo server reply address to same value. Check that the response is not filtered. +6. Enable the receiving of all multicasts. Check that the response is not filtered. -1. Using unicast, verify that the echo server responses are received. -2. Set the IPv6 multicast filter address and the echo server reply (forward) address to different values. Check if the echo response is filtered. -3. Set the IPv6 multicast filter address and the echo server reply address to same value. Check that the response is not filtered. -4. Set the IPv4 multicast filter address and the echo server reply address to different values. Check if the response is filtered. -5. Set the IPv4 multicast filter address and the echo server reply address to same value. Check that the response is not filtered. -6. Enable the receiving of all multicasts. Check that the response is not filtered. +**Expected result:** + +Echo server replies to are received as expected by the test step. Supporting of the filtering will affect what messages are received. ### EMAC memory -Tests memory manager out-of-memory situations. The test case configures the test memory manager to reject memory buffer allocations made by the EMAC. Memory buffer allocations are divided into output and input memory allocations: +**Description:** + +Test case tests memory manager out-of-memory situations. The test case configures the test memory manager to reject memory buffer allocations made by the EMAC. Memory buffer allocations are divided into output and input memory allocations: -* The output memory allocations are the ones made by the EMAC in the `link_out()` function called by the network stack (test case). -* The input memory allocations are other memory allocations made by the EMAC. +- The output memory allocations are the ones made by the EMAC in the \`link\_out()\` function called by the network stack (test case). +- The input memory allocations are other memory allocations made by the EMAC. Depending on the EMAC implementation, it may or may not allocate memory manager buffers in the link output function. If the memory manager buffers are not allocated, disabling the link output memory allocations in the test does not affect the functionality. -In each test step, the test case sends CTP unicast messages with Ethernet message length from 100 bytes to the maximum defined by the MTU of the EMAC with 50 bytes increments. Memory buffers sent to the EMAC in the `link_out()` function are forced to be non-aligned in this test case. +In each test step, the test case sends CTP unicast messages with Ethernet message length from 100 bytes to the maximum defined by the MTU of the EMAC with 50 bytes increments. Memory buffers sent to the EMAC in the \`link\_out()\` function are forced to be non-aligned in this test case. + +**Preconditions:** + +1. Network interface is connected. + +**Test steps:** + +1. Memory buffer allocations are allowed. Verify that echo server responses are received. +2. Disable input memory buffer allocations. The echo server responses should not be received. +3. Allow memory buffer allocations. Verify that the echo server responses are received. +4. Disable output memory buffer allocations. The echo server responses may or may not be received depending on the EMAC link out implementation. +5. Allow memory buffer allocations. Verify that the echo server responses are received. +6. Disable input and output memory buffer allocations. The echo server responses should not be received. +7. Allow memory buffer allocations. Verify that the echo server responses are received. +8. Allocate memory buffers that are sent to the EMAC in link out from the heap (contiguous memory). Verify that the echo server responses are received. -Test steps: +**Expected result:** -1. Memory buffer allocations are allowed. Verify that echo server responses are received. -2. Disable input memory buffer allocations. The echo server responses should not be received. -3. Allow memory buffer allocations. Verify that the echo server responses are received. -4. Disable output memory buffer allocations. The echo server responses may or may not be received depending on the EMAC link out implementation. -5. Allow memory buffer allocations. Verify that the echo server responses are received. -6. Disable input and output memory buffer allocations. The echo server responses should not be received. -7. Allow memory buffer allocations. Verify that the echo server responses are received. -8. Allocate memory buffers that are sent to the EMAC in link out from the heap (contiguous memory). Verify that the echo server responses are received. +Echo server replies to messages that are sent to it based on whether the driver can allocate memory for frame or not. For each sent unicast message reply is waited for 500ms. In case reply is not received sending is repeated three times. If test expects that memory should be available and sending fails three times, test is failed. diff --git a/TESTS/network/interface/README.md b/TESTS/network/interface/README.md new file mode 100644 index 00000000000..70332e2f5a7 --- /dev/null +++ b/TESTS/network/interface/README.md @@ -0,0 +1,166 @@ +Network Interface test plan +======================== + +Target API +---------- + +The target for this plan is to test: + +- [NetworkInterface.h](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/NetworkInterface.h) + +Tools to be used +---------------- + +- Mbed OS. +- Standard Mbed OS development tools as described in [Arm Mbed tools overview](https://os.mbed.com/docs/latest/tools/index.html). + +Test environment +---------------- + +- [Greentea](https://os.mbed.com/docs/mbed-os/latest/tools/greentea-testing-applications.html) + +Test case priorities +-------------------- + +Please refer to the following table for priorities of test cases. Priorities +are labeled as MUST and SHOULD. MUST means this is a requirement and +therefore mandatory to pass the test. SHOULD means it is recommended to +pass the test if the driver implements the feature in question. + +| | Test case | Priority | +|-----|-----------------------------------------|----------| +| 1 | NETWORKINTERFACE_CONN_DISC_REPEAT | MUST | +| 2 | NETWORKINTERFACE_STATUS | MUST | +| 3 | NETWORKINTERFACE_STATUS_NONBLOCK | MUST | +| 4 | NETWORKINTERFACE_STATUS_GET | MUST | + +Building test binaries +---------------------- + +For testing the board and driver, test against the Mbed OS +master branch to get the most recent, up-to-date test cases and drivers. + +To create a build environment: + +```.sh +mbed new network_interface_test +cd network_interface_test +cd mbed-os +git checkout master +cd .. +``` + +Now build test binaries: + +```.sh +mbed test --compile -t -m -n mbed-os-tests-network-interface +``` + +Running tests +------------- + +When device is connected to network, or in case of wireless device near +the access point. + +```.sh +mbed test -n mbed-os-tests-network-interface +``` + +Test cases for NetworkInterface class +------------------------------------- + +### NETWORKINTERFACE_CONN_DISC_REPEAT + +**Description:** + +Test `NetworkInterface::connect()` and `NetworkInterface::disconnect()` + +**Preconditions:** + +1. Network interface is initialized. + +**Test steps:** + +1. Connect interface +2. Disconnect interface +3. Repeat connect and disconnect steps 1 to 2 four times + +**Expected result:** + +`Connect()` and `disconnect()` calls return NSAPI_ERROR_OK. + +### NETWORKINTERFACE_STATUS + +**Description:** + +Test `NetworkInterface::attach()` and status indications. + +**Preconditions:** + +1. Network interface is initialized +2. Status callback is attached to network interface +3. Network interface is on blocking mode + +**Test steps:** + +1. Connect interface +2. Check that interface indicates status NSAPI_STATUS_CONNECTING +3. Optional: check that interface indicates status NSAPI_STATUS_LOCAL_UP (whether this is indicated depends on network interface type) +4. Check that interface indicates status NSAPI_STATUS_GLOBAL_UP +5. Disconnect interface +6. Check that interface indicates status NSAPI_STATUS_DISCONNECTED +7. Repeat connect and disconnect steps 1 to 6 four times + +**Expected result:** + +Callback statuses are correct. `Connect()` and `disconnect()` calls return NSAPI_ERROR_OK. Note: network interface may indicate same status several times a row depending on network interface type. This will not fail the test. + +### NETWORKINTERFACE_STATUS_NONBLOCK + +**Description:** + +Test `NetworkInterface::attach()` and status indications on non-blocking mode. + +**Preconditions:** + +1. Network interface is initialized +2. Status callback is attached to network interface +3. Network interface is on non-blocking mode + +**Test steps:** + +1. Connect interface +2. Check that interface indicates status NSAPI_STATUS_CONNECTING +3. Optional: check that interface indicates status NSAPI_STATUS_LOCAL_UP (whether this is indicated depends on network interface type) +4. Check that interface indicates status NSAPI_STATUS_GLOBAL_UP +5. Disconnect interface +6. Check that interface indicates status NSAPI_STATUS_DISCONNECTED +7. Repeat connect and disconnect steps 1 to 6 four times + +**Expected result:** + +Callback statuses are correct. `Connect()` and `disconnect()` calls return NSAPI_ERROR_OK. Note: network interface may indicate same status several times a row depending on network interface type. This will not fail the test. + +### NETWORKINTERFACE_STATUS_GET + +**Description:** + +Test `NetworkInterface::get_connection_status()`. + +**Preconditions:** + +1. Network interface is initialized +2. Network interface is on blocking mode + +**Test steps:** + +1. Check that `get_connection_status()` returns status NSAPI_STATUS_DISCONNECTED +2. Connect interface +3. Poll the `get_connection_status()` until it returns status NSAPI_STATUS_GLOBAL_UP +4. Disconnect interface +5. Check that `get_connection_status()` returns status NSAPI_STATUS_DISCONNECTED +6. Repeat connect and disconnect steps 2 to 5 four times + +**Expected result:** + +`Connect()` and `disconnect()` calls return NSAPI_ERROR_OK. Right status is returned by `get_connection_status()`. diff --git a/TESTS/network/wifi/README.md b/TESTS/network/wifi/README.md new file mode 100644 index 00000000000..db1cfe96d18 --- /dev/null +++ b/TESTS/network/wifi/README.md @@ -0,0 +1,506 @@ +Wifi test plan +======================== + +This is a test plan for Mbed OS Wifi API. + +Target API +---------- + +Target for this plan is to test [WiFiInterface](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/WiFiInterface.h) class. + +Tools to be used +---------------- + +- Mbed OS +- Secure WiFi access point +- Unsecure WiFi access point + +This test plan is written in tool agnostic way and do no specify how test cases should be written, build or run. + +Test environment +---------------- + +General test environment consist of DUTs, base stations, network connection and test server. + +![Wi-Fi](../../netsocket/wifi_environment.png) + +- All DUTS should be in RAAS +- Two Wifi APs required + - non-secured one. SSID later referred as `` + - secured one: SSID later referred as ``, password referred as `` +- IPv4 and IPv6 must be routable between Wifi network IP address space and echo server. +- Firewall must not block test traffic to and from test server **echo.mbedcloudtesting.com** + - Echo Protocol, [RFC 862](https://tools.ietf.org/html/rfc862) is enabled in both TCP and UDP. Port 7. + - Discard Protocol, [RFC 863](https://tools.ietf.org/html/rfc863) is enabled in both TCP and UDP. Port 9 + - Character generator protocol, [RFC 864](https://tools.ietf.org/html/rfc864) is enabled in both TCP and UDP. Port 19 + - Daytime protocol, [RFC 867](https://tools.ietf.org/html/rfc867) in both TCP and UDP. Port 13. + - Time protocol, [RFC 868](https://tools.ietf.org/html/rfc868) in both TCP and UDP. Port 37. +- Channels to be used must be different for both APs. For secure on channel number is later referred as `` and for unsecure on `` +- MAC addresses of Wifi APs must be known. Later referred as `` and `` + +**NOTE:** Echo server is referred here as it is a requirement for running Socket API tests. It is not directly used by test cases defined in this document. + +Test case priorities +-------------------- + +Please refer to following table for priorities of test cases. Priorities are labelled as MUST and SHOULD. MUST means this is a requirement and therefore mandatory to pass the test. SHOULD means it is recommended to pass the test if the driver implements the feature in question. + +| | Test case | | Priority | +|-----|-----------------------------------------|----------------------------|----------| +| 1 | WIFI_CONSTRUCTOR | | MUST | +| 2 | WIFI_CONNECT_NOCREDENTIALS | | MUST | +| 3 | WIFI_SET_CREDENTIAL | | MUST | +| 4 | WIFI_SET_CHANNEL | | SHOULD | +| 5 | WIFI_GET_RSSI | | SHOULD | +| 6 | WIFI_CONNECT_PARAMS_NULL | | MUST | +| 7 | WIFI_CONNECT_PARAMS_VALID_UNSECURE | | MUST | +| 8 | WIFI_CONNECT_PARAMS_VALID_SECURE | With security type: | | +| | | NSAPI_SECURITY_WEP | SHOULD | +| | | NSAPI_SECURITY_WPA | SHOULD | +| | | NSAPI_SECURITY_WPA2 | SHOULD | +| | | NSAPI_SECURITY_WPA_WPA2 | MUST | +| 9 | WIFI_CONNECT_PARAMS_CHANNEL | | SHOULD | +| 10 | WIFI_CONNECT_PARAMS_CHANNEL_FAIL | | SHOULD | +| 11 | WIFI_CONNECT | | MUST | +| 12 | WIFI_CONNECT_SECURE | With security type: | | +| | | NSAPI_SECURITY_WEP | SHOULD | +| | | NSAPI_SECURITY_WPA | SHOULD | +| | | NSAPI_SECURITY_WPA2 | SHOULD | +| | | NSAPI_SECURITY_WPA_WPA2 | MUST | +| 13 | WIFI_CONNECT_SECURE_FAIL | | MUST | +| 14 | WIFI_CONNECT_DISCONNECT_REPEAT | | MUST | +| 15 | WIFI_SCAN_NULL | | SHOULD | +| 16 | WIFI_SCAN | | SHOULD | + +Building test binaries +---------------------- + +For testing the board and driver, test against the Mbed OS +master branch to get the most recent, up-to-date test cases and drivers. + +To create a build environment: + +```.sh +mbed new wifi_test +cd wifi_test +cd mbed-os +git checkout master +cd .. +``` + +Now build test binaries: + +```.sh +mbed test --compile -t -m -n mbed-os-tests-network-wifi +``` + +Running tests +------------- + +Bear in mind that RAAS user and password have to be set in the shell to access it. + +```.sh +mbedgt -g :raas_client:ruka.mbedcloudtesting.com:8000 -n mbed-os-tests-network-wifi -V -v +``` + +Test cases for WifiInterface class +------------------------------------- + +### WIFI_CONSTRUCTOR + +**Description:** + + Test that constructor of the driver works. + +**Preconditions:** + +1. None + +**Test steps:** + +1. Construct the driver by calling the constructor. + +**Expected result:** + +Constructor returns. + +### WIFI_CONNECT_NOCREDENTIALS + +**Description:** + + Test `WiFiInterface::connect()` without parameters. Don't set parameters with `set_credentials()` + +This must be a first test to run after constructing the driver. No credentials should be given for the driver before this test. + +**Preconditions:** + +1. Test enviroment is set up as specified in "Test Environment" chapter. + +**Test steps:** + +1. Initialise the driver +2. Call `WiFiInterface::connect()` +3. `disconnect()` + +**Expected result:** + +`connect()` call returns NSAPI_ERROR_PARAMETER or NSAPI_ERROR_NO_SSID + +### WIFI_SET_CREDENTIAL + +**Description:** + +This test case is to test whether the driver accepts valid credentials and reject ones that are not valid. + +**Preconditions:** + +1. Driver class is initialised. + +**Test steps:** + +1. Call `set_credentials()` with various parameters described in "Expected results" + +**Expected result:** + +| Parameters | Password | Security | Expected return code | +|------------|---------------------------|----------------------------------|-----------------------------------------------------| +| ssid=NULL | NULL | NSAPI_SECURITY_NONE | NSAPI_ERROR_PARAMETER | +| ssid="OK" | NULL | NSAPI_SECURITY_WPA_WPA2 | NSAPI_ERROR_PARAMETER | +| ssid="OK" | NULL | NSAPI_SECURITY_WEP | NSAPI_ERROR_PARAMETER because password is missing. | +| ssid="OK" | NULL | NSAPI_SECURITY_NONE | NSAPI_ERROR_OK | +| ssid="OK" | [any 64 character string] | NSAPI_SECURITY_WPA2 | NSAPI_ERROR_PARAMETER because password is too long | +| ssid="OK" | [any 63 character string] | NSAPI_SECURITY_WPA2 | NSAPI_ERROR_OK | +| ssid="OK" | "" | NSAPI_SECURITY_WPA_WPA2 | NSAPI_ERROR_PARAMETER | +| ssid="OK" | "" | NSAPI_SECURITY_WEP | NSAPI_ERROR_PARAMETER because password is missing. | +| ssid="OK" | "" | NSAPI_SECURITY_NONE | NSAPI_ERROR_OK | +| ssid="OK" | "12345678" | NSAPI_SECURITY_WPA_WPA2 | NSAPI_ERROR_OK | +| ssid="OK" | "12345678" | NSAPI_SECURITY_WPA2 | NSAPI_ERROR_OK | +| ssid="OK" | "12345678" | NSAPI_SECURITY_WPA | NSAPI_ERROR_OK | +| ssid="OK" | "12345678" | NSAPI_SECURITY_WEP | NSAPI_ERROR_OK or NSAPI_ERROR_UNSUPPORTED | +| ssid="" | "" | NSAPI_SECURITY_NONE | NSAPI_ERROR_PARAMETER | + +### WIFI_SET_CHANNEL + +**Description:** + +Test validity of `WiFiInterface::set_channel()`. + +When proper channel number is feed, it should return NSAPI_ERROR_OK. API documentation is unclear what to report on error case, but I'm assuming NSAPI_ERROR_PARAMETER. + +If the driver does not support setting channels for scan, it should report NSAPI_ERROR_UNSUPPORTED for all of the channels. + +**Preconditions:** + +1. Driver class is initialised + +**Test steps:** + +1. Call `set_channel()` with various parameters described in "Expected results" + +**Expected result:** + +For 2.4 Ghz chips + +| channel | Expected return code | +|---------|-----------------------------------------------------------------------------------| +| 0 | NSAPI_ERROR_OK (0 = any channel) | +| 1 | NSAPI_ERROR_OK | +| 13 | NSAPI_ERROR_OK if supporting European frequencies | +| | NSAPI_ERROR_PARAMETER if configured for North America | +| 15 | NSAPI_ERROR_PARAMETER | +| | because not a valid 2.4 Ghz channel | + +For 5 Ghz chips + +| channel | Expected return code | +|---------|-----------------------------------------------------------------------------------| +| 3 | NSAPI_ERROR_PARAMETER | +| | because not a valid 5 Ghz channel number | +| 36 | NSAPI_ERROR_OK | +| 169 | NSAPI_ERROR_PARAMETERs | +| | should not be allowed in any country | + +Drivers not supporting + +| channel | Expected return code | +|---------|-----------------------------------------------------------------------------------| +| | NSAPI_ERROR_UNSUPPORTED should be returned for ALL channel numbers > 0. | + +### WIFI_GET_RSSI + +**Description:** + + Test `WiFiInterface::get_rssi()` API. When connected, it should return valid RSSI value. When unconnected it should return 0. + +API is a bit unclear whether 0 can also mean that driver does not support reporting of RSSI. + +**Preconditions:** + +1. Test enviroment is set up as specified in "Test Environment" chapter. + +**Test steps:** + +1. Initialise the driver. +2. Call `get_rssi()`, store the result. +3. Call `connect()` with valid SSID parameters. Wait for connection +4. Call `get_rssi()` + +**Expected result:** + +First call should return zero, second call should return negative number between -10 and -100. + +### WIFI_CONNECT_PARAMS_NULL + +**Description:** + +Test `WiFiInterface::connect(ssid, pass, security, channel)` with NULL parameters + +**Preconditions:** + +1. Test enviroment is set up as specified in "Test Environment" chapter. + +**Test steps:** + +1. Initialise the driver +2. Call `WiFiInterface::connect(NULL, NULL)` +3. Call `WiFiInterface::connect("", "")` + +**Expected result:** + +Both `connect()` calls return NSAPI_ERROR_PARAMETER + +### WIFI_CONNECT_PARAMS_VALID_UNSECURE + +**Description:** + + Test `WiFiInterface::connect(ssid, pass, security)` with valid parameters for unsecure network + +**Preconditions:** + +1. Test enviroment is set up as specified in "Test Environment" chapter. + +**Test steps:** + +1. Initialise the driver +2. Call `WiFiInterface::connect( , NULL)` +3. `disconnect()` +4. Call `WiFiInterface::connect( , "")` +5. `disconnect()` + +**Expected result:** + +`connect()` calls return NSAPI_ERROR_OK + +### WIFI_CONNECT_PARAMS_VALID_SECURE + +**Description:** + + Test `WiFiInterface::connect(ssid, pass, security)` with valid parameters for secure network + +**Preconditions:** + +1. Test enviroment is set up as specified in "Test Environment" chapter. + +**Test steps:** + +1. Initialise the driver +2. Call `WiFiInterface::connect( , , NSAPI_SECURITY_WPA2)` +3. `disconnect()` + +**Expected result:** + +`connect()` call returns NSAPI_ERROR_OK + +  + +### WIFI_CONNECT_PARAMS_CHANNEL + +**Description:** + + Test `WiFiInterface::connect(ssid, pass, security, channel)` with valid parameters for secure network using channel specified. + +This test only applies to devices which support selecting the channel (passes WIFI-SET-CHANNEL). + +**Preconditions:** + +1. Test enviroment is set up as specified in "Test Environment" chapter. + +**Test steps:** + +1. Initialise the driver +2. Call `WiFiInterface::connect( , , NSAPI_SECURITY_WPA2, )` +3. `disconnect()` + +**Expected result:** + +`connect()` call returns NSAPI_ERROR_OK + +  + +### WIFI_CONNECT_PARAMS_CHANNEL_FAIL + +**Description:** + + Test `WiFiInterface::connect(ssid, pass, security, channel)` with valid parameters for secure network using channel specified. Channel specified must be wrong so that connect call should fail. + +This test only applies to devices which support selecting the channel (passes WIFI-SET-CHANNEL). + +**Preconditions:** + +1. Test enviroment is set up as specified in "Test Environment" chapter. + +**Test steps:** + +1. Initialise the driver +2. Call `WiFiInterface::connect( , , NSAPI_SECURITY_WPA2, )` +3. `disconnect()` + +**Expected result:** + +connect() call returns NSAPI_ERROR_CONNECTION_TIMEOUT or NSAPI_ERROR_NO_CONNECTION + +### WIFI_CONNECT + +**Description:** + + Test `WiFiInterface::connect()` without parameters. Use `set_credentials()` for setting parameters. Checks that driver does not try to retrieve SSID from location it's not controlling. + +**Preconditions:** + +1. Test enviroment is set up as specified in "Test Environment" chapter. + +**Test steps:** + +1. Initialise the driver +2. `Call WiFiInterface::set_credentials( , NULL)` +3. `Call WiFiInterface::connect()` +4. `disconnect()` +5. `Call WiFiInterface::set_credentials( , "")` +6. `Call WiFiInterface::connect()` +7. `disconnect()` +8. trash the memory storing SSID +9. `Call WiFiInterface::set_credentials( , "")` +10. `Call WiFiInterface::connect()` +11. `disconnect()` + +**Expected result:** + +`connect()` calls return NSAPI_ERROR_OK + +### WIFI_CONNECT_SECURE + +**Description:** + + Test `WiFiInterface::connect()` without parameters. Use secure settings for `set_credentials`. + +**Preconditions:** + +1. Test enviroment is set up as specified in "Test Environment" chapter. + +**Test steps:** + +1. Initialise the driver +2. Call `WiFiInterface::set_credentials( , , NSAPI_SECURITY_WPA2)` +3. Call `WiFiInterface::connect()` +4. `disconnect()` + +**Expected result:** + +`connect()` call returns NSAPI_ERROR_OK + +### WIFI_CONNECT_SECURE_FAIL + +**Description:** + + Test `WiFiInterface::connect()` failing with wrong password. + +**Preconditions:** + +1. Test enviroment is set up as specified in "Test Environment" chapter. + +**Test steps:** + +1. Initialise the driver +2. Call `WiFiInterface::set_credentials( , , NSAPI_SECURITY_WPA2)` +3. Call `WiFiInterface::connect()` +4. disconnect() + +**Expected result:** + +`connect()` call returns NSAPI_ERROR_AUTH_FAILUR, NSAPI_ERROR_CONNECTION_TIMEOUT or NSAPI_ERROR_NO_CONNECTION + +### WIFI_CONNECT_DISCONNECT_REPEAT + +**Description:** + + Test `WiFiInterface::connect()` - `disconnect()` repeatition works. + +**Preconditions:** + +1. Test enviroment is set up as specified in "Test Environment" chapter. + +**Test steps:** + +1. Initialise the driver +2. Call `WiFiInterface::set_credentials( , NULL)` +3. Repeat 10 times + 1. Call `WiFiInterface::connect()` + 2. disconnect() + +**Expected result:** + +Each `connect()` call returns NSAPI_ERROR_OK + +Each `disconnect()` returns NSAPI_ERROR_OK + +### WIFI_SCAN_NULL + +**Description:** + + Call `WiFiInterface::scan()` with null parameters to get number of networks available. + +**Preconditions:** + +1. Test enviroment is set up as specified in "Test Environment" chapter. + +**Test steps:** + +1. Initialise the driver +2. Call `WiFiInterface::scan(NULL, 0)`; + +**Expected result:** + +`scan()` returns positive number that is higher or equivalent of two. (>=2). + +### WIFI_SCAN + +**Description:** + + Call `WiFiInterface::scan()` with valid accesspoint list allocated + +**Preconditions:** + +1. Test enviroment is set up as specified in "Test Environment" chapter. +2. Test environment must contain less than 10 WiFi SSID within the listening range of DUT, otherwise adjust the value used in step 2. + +**Test steps:** + +1. Initialise the driver +2. Allocate array of 10 WiFiAccessPoint objects. +3. Call `WiFiInterface::scan(, 10)`; + +**Expected result:** + +`scan()` returns positive number that is higher or equivalent of two. (>=2). + +Scan elements from array indexes between zero and the number returned. + +- Call `get_ssid()` for each element. `` and `` must be found. +- Call `get_rssid()` for each element. Value must be between -10 and -100. + +For those known networks, test that following return values are found: + +| get_ssid() | get_bssid() | get_security() | get_channel() | +|-------------------------|------------------------|-----------------------|-----------------------| +| `` | `` | NSAPI_SECURITY_NONE | `` | +| `` | `` | NSAPI_SECURITY_WPA2 | `` | \ No newline at end of file From 629e8a0e0562371f420eb52ca8675b76b3b95df1 Mon Sep 17 00:00:00 2001 From: Amanda Butler Date: Thu, 7 Feb 2019 15:45:08 -0600 Subject: [PATCH 2/4] Edit README.md Edit file, mostly for inclusion of articles and complete sentences. --- TESTS/network/wifi/README.md | 315 +++++++++++++++++------------------ 1 file changed, 155 insertions(+), 160 deletions(-) diff --git a/TESTS/network/wifi/README.md b/TESTS/network/wifi/README.md index db1cfe96d18..31aeb0525b7 100644 --- a/TESTS/network/wifi/README.md +++ b/TESTS/network/wifi/README.md @@ -1,49 +1,49 @@ -Wifi test plan +Wi-Fi test plan ======================== -This is a test plan for Mbed OS Wifi API. +This is a test plan for Mbed OS Wi-Fi API. Target API ---------- -Target for this plan is to test [WiFiInterface](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/WiFiInterface.h) class. +The goal of this plan is to test the [WiFiInterface](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/WiFiInterface.h) class. -Tools to be used +Tools needed ---------------- -- Mbed OS -- Secure WiFi access point -- Unsecure WiFi access point +- Mbed OS. +- Secure Wi-Fi access point. +- Unsecure Wi-Fi access point. -This test plan is written in tool agnostic way and do no specify how test cases should be written, build or run. +This test plan is written in a tool-agnostic way and does not specify how to write, build or run test cases. Test environment ---------------- -General test environment consist of DUTs, base stations, network connection and test server. +The general test environment consists of DUTs, base stations, a network connection and a test server: ![Wi-Fi](../../netsocket/wifi_environment.png) -- All DUTS should be in RAAS -- Two Wifi APs required - - non-secured one. SSID later referred as `` - - secured one: SSID later referred as ``, password referred as `` -- IPv4 and IPv6 must be routable between Wifi network IP address space and echo server. -- Firewall must not block test traffic to and from test server **echo.mbedcloudtesting.com** - - Echo Protocol, [RFC 862](https://tools.ietf.org/html/rfc862) is enabled in both TCP and UDP. Port 7. - - Discard Protocol, [RFC 863](https://tools.ietf.org/html/rfc863) is enabled in both TCP and UDP. Port 9 - - Character generator protocol, [RFC 864](https://tools.ietf.org/html/rfc864) is enabled in both TCP and UDP. Port 19 - - Daytime protocol, [RFC 867](https://tools.ietf.org/html/rfc867) in both TCP and UDP. Port 13. - - Time protocol, [RFC 868](https://tools.ietf.org/html/rfc868) in both TCP and UDP. Port 37. -- Channels to be used must be different for both APs. For secure on channel number is later referred as `` and for unsecure on `` -- MAC addresses of Wifi APs must be known. Later referred as `` and `` - -**NOTE:** Echo server is referred here as it is a requirement for running Socket API tests. It is not directly used by test cases defined in this document. +- All DUTS should be in RAAS. +- Two Wi-Fi APs required: + - Nonsecured one: The SSID is later referred to as ``. + - Secured one: The SSID is later referred to as ``, password referred as ``. +- IPv4 and IPv6 must be routable between the Wi-Fi network IP address space and echo server. +- Firewall must not block test traffic to and from the test server **echo.mbedcloudtesting.com**. + - Echo Protocol, [RFC 862](https://tools.ietf.org/html/rfc862) is enabled in both TCP and UDP. Port 7. + - Discard Protocol, [RFC 863](https://tools.ietf.org/html/rfc863) is enabled in both TCP and UDP. Port 9. + - Character generator protocol, [RFC 864](https://tools.ietf.org/html/rfc864) is enabled in both TCP and UDP. Port 19. + - Daytime protocol, [RFC 867](https://tools.ietf.org/html/rfc867) in both TCP and UDP. Port 13. + - Time protocol, [RFC 868](https://tools.ietf.org/html/rfc868) in both TCP and UDP. Port 37. +- Channels to be used must be different for both APs. For secure on channel number is later referred as `` and for unsecure on ``. +- MAC addresses of Wi-Fi APs must be known. These are later referred to as `` and ``. + +**NOTE:** This document refers to an echo server because it is a requirement for running Socket API tests. The test cases defined in this document do not directly use it. Test case priorities -------------------- -Please refer to following table for priorities of test cases. Priorities are labelled as MUST and SHOULD. MUST means this is a requirement and therefore mandatory to pass the test. SHOULD means it is recommended to pass the test if the driver implements the feature in question. +Please refer to the following table for priorities of test cases. Priorities are labeled as MUST and SHOULD. MUST means this is a requirement and therefore mandatory to pass the test. SHOULD means it is recommended to pass the test if the driver implements the feature in question. | | Test case | | Priority | |-----|-----------------------------------------|----------------------------|----------| @@ -75,8 +75,7 @@ Please refer to following table for priorities of test cases. Priorities are lab Building test binaries ---------------------- -For testing the board and driver, test against the Mbed OS -master branch to get the most recent, up-to-date test cases and drivers. +For testing the board and driver, test against the Mbed OS master branch for the most recent, up-to-date test cases and drivers. To create a build environment: @@ -97,7 +96,7 @@ mbed test --compile -t -m -n mbed-os-tests-network-wifi Running tests ------------- -Bear in mind that RAAS user and password have to be set in the shell to access it. +The RAAS user and password have to be set in the shell to access it: ```.sh mbedgt -g :raas_client:ruka.mbedcloudtesting.com:8000 -n mbed-os-tests-network-wifi -V -v @@ -110,15 +109,15 @@ Test cases for WifiInterface class **Description:** - Test that constructor of the driver works. + Test that the constructor of the driver works. **Preconditions:** -1. None +None. -**Test steps:** +**Test step:** -1. Construct the driver by calling the constructor. +Construct the driver by calling the constructor. **Expected result:** @@ -128,37 +127,37 @@ Constructor returns. **Description:** - Test `WiFiInterface::connect()` without parameters. Don't set parameters with `set_credentials()` +Test `WiFiInterface::connect()` without parameters. Don't set parameters with `set_credentials()`. This must be a first test to run after constructing the driver. No credentials should be given for the driver before this test. -**Preconditions:** +**Precondition:** -1. Test enviroment is set up as specified in "Test Environment" chapter. +Test the enviroment is set up as specified in the "Test Environment" chapter. **Test steps:** -1. Initialise the driver -2. Call `WiFiInterface::connect()` -3. `disconnect()` +1. Initialize the driver. +2. Call `WiFiInterface::connect()`. +3. `disconnect()`. **Expected result:** -`connect()` call returns NSAPI_ERROR_PARAMETER or NSAPI_ERROR_NO_SSID +`connect()` call returns `NSAPI_ERROR_PARAMETER or NSAPI_ERROR_NO_SSID`. ### WIFI_SET_CREDENTIAL **Description:** -This test case is to test whether the driver accepts valid credentials and reject ones that are not valid. +This test case is to test whether the driver accepts valid credentials and rejects ones that are not valid. -**Preconditions:** +**Precondition:** -1. Driver class is initialised. +Driver class is initialized. -**Test steps:** +**Test step:** -1. Call `set_credentials()` with various parameters described in "Expected results" +Call `set_credentials()` with various parameters described in "Expected results". **Expected result:** @@ -185,23 +184,23 @@ This test case is to test whether the driver accepts valid credentials and rejec Test validity of `WiFiInterface::set_channel()`. -When proper channel number is feed, it should return NSAPI_ERROR_OK. API documentation is unclear what to report on error case, but I'm assuming NSAPI_ERROR_PARAMETER. +When proper channel number is feed, it should return `NSAPI_ERROR_OK`. API documentation is unclear what to report on error case, but we assume `NSAPI_ERROR_PARAMETER`. -If the driver does not support setting channels for scan, it should report NSAPI_ERROR_UNSUPPORTED for all of the channels. +If the driver does not support setting channels for scan, it should report `NSAPI_ERROR_UNSUPPORTED` for all of the channels. -**Preconditions:** +**Precondition:** -1. Driver class is initialised +Driver class is initialized. -**Test steps:** +**Test step:** -1. Call `set_channel()` with various parameters described in "Expected results" +Call `set_channel()` with various parameters described in "Expected results". **Expected result:** -For 2.4 Ghz chips +For 2.4 Ghz chips: -| channel | Expected return code | +| Channel | Expected return code | |---------|-----------------------------------------------------------------------------------| | 0 | NSAPI_ERROR_OK (0 = any channel) | | 1 | NSAPI_ERROR_OK | @@ -210,9 +209,9 @@ For 2.4 Ghz chips | 15 | NSAPI_ERROR_PARAMETER | | | because not a valid 2.4 Ghz channel | -For 5 Ghz chips +For 5 Ghz chips: -| channel | Expected return code | +| Channel | Expected return code | |---------|-----------------------------------------------------------------------------------| | 3 | NSAPI_ERROR_PARAMETER | | | because not a valid 5 Ghz channel number | @@ -230,277 +229,273 @@ Drivers not supporting **Description:** - Test `WiFiInterface::get_rssi()` API. When connected, it should return valid RSSI value. When unconnected it should return 0. +Test `WiFiInterface::get_rssi()` API. When connected, it should return a valid RSSI value. When unconnected, it should return 0. API is a bit unclear whether 0 can also mean that driver does not support reporting of RSSI. -**Preconditions:** +**Precondition:** -1. Test enviroment is set up as specified in "Test Environment" chapter. +Test enviroment is set up as specified in "Test Environment" chapter. **Test steps:** -1. Initialise the driver. +1. Initialize the driver. 2. Call `get_rssi()`, store the result. -3. Call `connect()` with valid SSID parameters. Wait for connection -4. Call `get_rssi()` +3. Call `connect()` with valid SSID parameters. Wait for connection. +4. Call `get_rssi()`. **Expected result:** -First call should return zero, second call should return negative number between -10 and -100. +The first call should return 0, and the second call should return a negative number between -10 and -100. ### WIFI_CONNECT_PARAMS_NULL **Description:** -Test `WiFiInterface::connect(ssid, pass, security, channel)` with NULL parameters +Test `WiFiInterface::connect(ssid, pass, security, channel)` with NULL parameters. -**Preconditions:** +**Precondition:** -1. Test enviroment is set up as specified in "Test Environment" chapter. +Test enviroment is set up as specified in "Test Environment" chapter. **Test steps:** -1. Initialise the driver -2. Call `WiFiInterface::connect(NULL, NULL)` -3. Call `WiFiInterface::connect("", "")` +1. Initialize the driver. +2. Call `WiFiInterface::connect(NULL, NULL)`. +3. Call `WiFiInterface::connect("", "")`. **Expected result:** -Both `connect()` calls return NSAPI_ERROR_PARAMETER +Both `connect()` calls return NSAPI_ERROR_PARAMETER. ### WIFI_CONNECT_PARAMS_VALID_UNSECURE **Description:** - Test `WiFiInterface::connect(ssid, pass, security)` with valid parameters for unsecure network +Test `WiFiInterface::connect(ssid, pass, security)` with valid parameters for the unsecure network. -**Preconditions:** +**Precondition:** -1. Test enviroment is set up as specified in "Test Environment" chapter. +Test enviroment is set up as specified in the "Test Environment" chapter. **Test steps:** -1. Initialise the driver -2. Call `WiFiInterface::connect( , NULL)` -3. `disconnect()` -4. Call `WiFiInterface::connect( , "")` -5. `disconnect()` +1. Initialize the driver. +2. Call `WiFiInterface::connect( , NULL)`. +3. `disconnect()`. +4. Call `WiFiInterface::connect( , "")`. +5. `disconnect()`. **Expected result:** -`connect()` calls return NSAPI_ERROR_OK +`connect()` calls return NSAPI_ERROR_OK. ### WIFI_CONNECT_PARAMS_VALID_SECURE **Description:** - Test `WiFiInterface::connect(ssid, pass, security)` with valid parameters for secure network +Test `WiFiInterface::connect(ssid, pass, security)` with valid parameters for secure network. **Preconditions:** -1. Test enviroment is set up as specified in "Test Environment" chapter. +Test enviroment is set up as specified in the "Test Environment" chapter. **Test steps:** -1. Initialise the driver -2. Call `WiFiInterface::connect( , , NSAPI_SECURITY_WPA2)` -3. `disconnect()` +1. Initialize the driver. +2. Call `WiFiInterface::connect( , , NSAPI_SECURITY_WPA2)`. +3. `disconnect()`. **Expected result:** -`connect()` call returns NSAPI_ERROR_OK - -  +`connect()` call returns NSAPI_ERROR_OK. ### WIFI_CONNECT_PARAMS_CHANNEL **Description:** - Test `WiFiInterface::connect(ssid, pass, security, channel)` with valid parameters for secure network using channel specified. +Test `WiFiInterface::connect(ssid, pass, security, channel)` with valid parameters for the secure network using the channel specified. -This test only applies to devices which support selecting the channel (passes WIFI-SET-CHANNEL). +This test only applies to devices that support selecting the channel (passes `WIFI-SET-CHANNEL`). -**Preconditions:** +**Precondition:** -1. Test enviroment is set up as specified in "Test Environment" chapter. +Test enviroment is set up as specified in the "Test Environment" chapter. **Test steps:** -1. Initialise the driver -2. Call `WiFiInterface::connect( , , NSAPI_SECURITY_WPA2, )` -3. `disconnect()` +1. Initialize the driver. +2. Call `WiFiInterface::connect( , , NSAPI_SECURITY_WPA2, )`. +3. `disconnect()`. **Expected result:** -`connect()` call returns NSAPI_ERROR_OK - -  +`connect()` call returns NSAPI_ERROR_OK. ### WIFI_CONNECT_PARAMS_CHANNEL_FAIL **Description:** - Test `WiFiInterface::connect(ssid, pass, security, channel)` with valid parameters for secure network using channel specified. Channel specified must be wrong so that connect call should fail. +Test `WiFiInterface::connect(ssid, pass, security, channel)` with valid parameters for the secure network using the channel specified. The channel specified must be wrong, so the connect call can fail. -This test only applies to devices which support selecting the channel (passes WIFI-SET-CHANNEL). +This test only applies to devices that support selecting the channel (passes `WIFI-SET-CHANNEL`). -**Preconditions:** +**Precondition:** -1. Test enviroment is set up as specified in "Test Environment" chapter. +Test enviroment is set up as specified in the "Test Environment" chapter. **Test steps:** -1. Initialise the driver -2. Call `WiFiInterface::connect( , , NSAPI_SECURITY_WPA2, )` -3. `disconnect()` +1. Initialize the driver. +2. Call `WiFiInterface::connect( , , NSAPI_SECURITY_WPA2, )`. +3. `disconnect()`. **Expected result:** -connect() call returns NSAPI_ERROR_CONNECTION_TIMEOUT or NSAPI_ERROR_NO_CONNECTION +The `connect()` call returns `NSAPI_ERROR_CONNECTION_TIMEOUT` or `NSAPI_ERROR_NO_CONNECTION`. ### WIFI_CONNECT **Description:** - Test `WiFiInterface::connect()` without parameters. Use `set_credentials()` for setting parameters. Checks that driver does not try to retrieve SSID from location it's not controlling. +Test `WiFiInterface::connect()` without parameters. Use `set_credentials()` for setting parameters. This checks that driver does not try to retrieve SSID from a location it's not controlling. **Preconditions:** -1. Test enviroment is set up as specified in "Test Environment" chapter. +1. Test enviroment is set up as specified in the "Test Environment" chapter. **Test steps:** -1. Initialise the driver -2. `Call WiFiInterface::set_credentials( , NULL)` -3. `Call WiFiInterface::connect()` -4. `disconnect()` -5. `Call WiFiInterface::set_credentials( , "")` -6. `Call WiFiInterface::connect()` -7. `disconnect()` -8. trash the memory storing SSID -9. `Call WiFiInterface::set_credentials( , "")` -10. `Call WiFiInterface::connect()` -11. `disconnect()` +1. Initialize the driver. +2. `Call WiFiInterface::set_credentials( , NULL)`. +3. `Call WiFiInterface::connect()`. +4. `disconnect()`. +5. `Call WiFiInterface::set_credentials( , "")`. +6. `Call WiFiInterface::connect()`. +7. `disconnect()`. +8. Trash the memory storing SSID. +9. `Call WiFiInterface::set_credentials( , "")`. +10. `Call WiFiInterface::connect()`. +11. `disconnect()`. **Expected result:** -`connect()` calls return NSAPI_ERROR_OK +`connect()` calls return `NSAPI_ERROR_OK`. ### WIFI_CONNECT_SECURE **Description:** - Test `WiFiInterface::connect()` without parameters. Use secure settings for `set_credentials`. +Test `WiFiInterface::connect()` without parameters. Use secure settings for `set_credentials`. -**Preconditions:** +**Precondition:** -1. Test enviroment is set up as specified in "Test Environment" chapter. +The test enviroment is set up as specified in the "Test Environment" chapter. **Test steps:** -1. Initialise the driver -2. Call `WiFiInterface::set_credentials( , , NSAPI_SECURITY_WPA2)` -3. Call `WiFiInterface::connect()` -4. `disconnect()` +1. Initialize the driver. +2. Call `WiFiInterface::set_credentials( , , NSAPI_SECURITY_WPA2)`. +3. Call `WiFiInterface::connect()`. +4. `disconnect()`. **Expected result:** -`connect()` call returns NSAPI_ERROR_OK +The `connect()` call returns `NSAPI_ERROR_OK`. ### WIFI_CONNECT_SECURE_FAIL **Description:** - Test `WiFiInterface::connect()` failing with wrong password. + Test `WiFiInterface::connect()` failing with the wrong password. -**Preconditions:** +**Precondition:** -1. Test enviroment is set up as specified in "Test Environment" chapter. +The test enviroment is set up as specified in the "Test Environment" chapter. **Test steps:** -1. Initialise the driver -2. Call `WiFiInterface::set_credentials( , , NSAPI_SECURITY_WPA2)` -3. Call `WiFiInterface::connect()` -4. disconnect() +1. Initialize the driver. +2. Call `WiFiInterface::set_credentials( , , NSAPI_SECURITY_WPA2)`. +3. Call `WiFiInterface::connect()`. +4. `disconnect()`. **Expected result:** -`connect()` call returns NSAPI_ERROR_AUTH_FAILUR, NSAPI_ERROR_CONNECTION_TIMEOUT or NSAPI_ERROR_NO_CONNECTION +`connect()` call returns `NSAPI_ERROR_AUTH_FAILUR`, `NSAPI_ERROR_CONNECTION_TIMEOUT` or `NSAPI_ERROR_NO_CONNECTION`. ### WIFI_CONNECT_DISCONNECT_REPEAT **Description:** - Test `WiFiInterface::connect()` - `disconnect()` repeatition works. +Test `WiFiInterface::connect()` - `disconnect()` repetition works. -**Preconditions:** +**Precondition:** -1. Test enviroment is set up as specified in "Test Environment" chapter. +The test enviroment is set up as specified in the "Test Environment" chapter. **Test steps:** -1. Initialise the driver -2. Call `WiFiInterface::set_credentials( , NULL)` -3. Repeat 10 times - 1. Call `WiFiInterface::connect()` - 2. disconnect() +1. Initialize the driver. +2. Call `WiFiInterface::set_credentials( , NULL)`. +3. Repeat 10 times: + 1. Call `WiFiInterface::connect()`. + 2. `disconnect()`. **Expected result:** -Each `connect()` call returns NSAPI_ERROR_OK +Each `connect()` call returns `NSAPI_ERROR_OK`. -Each `disconnect()` returns NSAPI_ERROR_OK +Each `disconnect()` returns `NSAPI_ERROR_OK`. ### WIFI_SCAN_NULL **Description:** - Call `WiFiInterface::scan()` with null parameters to get number of networks available. +Call `WiFiInterface::scan()` with null parameters to get the number of networks available. -**Preconditions:** +**Precondition:** -1. Test enviroment is set up as specified in "Test Environment" chapter. +The test enviroment is set up as specified in the "Test Environment" chapter. **Test steps:** -1. Initialise the driver -2. Call `WiFiInterface::scan(NULL, 0)`; +1. Initialize the driver. +2. Call `WiFiInterface::scan(NULL, 0)`;. **Expected result:** -`scan()` returns positive number that is higher or equivalent of two. (>=2). +`scan()` returns a positive number that is higher or equal to two. (>=2). ### WIFI_SCAN **Description:** - Call `WiFiInterface::scan()` with valid accesspoint list allocated + Call `WiFiInterface::scan()` with a valid accesspoint list allocated. **Preconditions:** -1. Test enviroment is set up as specified in "Test Environment" chapter. -2. Test environment must contain less than 10 WiFi SSID within the listening range of DUT, otherwise adjust the value used in step 2. +1. The test enviroment is set up as specified in the "Test Environment" chapter. +2. The test environment must contain less than 10 Wi-Fi SSID within the listening range of DUT; otherwise, adjust the value used in step 2. **Test steps:** -1. Initialise the driver -2. Allocate array of 10 WiFiAccessPoint objects. +1. Initialize the driver. +2. Allocate array of 10 WiFiAccessPoint objects. 3. Call `WiFiInterface::scan(, 10)`; **Expected result:** -`scan()` returns positive number that is higher or equivalent of two. (>=2). +`scan()` returns a positive number that is higher or equal to two. (>=2). -Scan elements from array indexes between zero and the number returned. +Scan elements from array indices between 0 and the number returned: -- Call `get_ssid()` for each element. `` and `` must be found. -- Call `get_rssid()` for each element. Value must be between -10 and -100. +- Call `get_ssid()` for each element. `` and `` must be found. +- Call `get_rssid()` for each element. The value must be between -10 and -100. -For those known networks, test that following return values are found: +For those known networks, test that the following return values are found: | get_ssid() | get_bssid() | get_security() | get_channel() | |-------------------------|------------------------|-----------------------|-----------------------| | `` | `` | NSAPI_SECURITY_NONE | `` | -| `` | `` | NSAPI_SECURITY_WPA2 | `` | \ No newline at end of file +| `` | `` | NSAPI_SECURITY_WPA2 | `` | From e9c44a86ac0435ca4b2dc71cd3287fd482f381d3 Mon Sep 17 00:00:00 2001 From: Amanda Butler Date: Thu, 7 Feb 2019 15:52:20 -0600 Subject: [PATCH 3/4] Edit README.md Edit file, mostly for inclusion of articles. --- TESTS/network/interface/README.md | 92 +++++++++++++++---------------- 1 file changed, 45 insertions(+), 47 deletions(-) diff --git a/TESTS/network/interface/README.md b/TESTS/network/interface/README.md index 70332e2f5a7..90e912dfda4 100644 --- a/TESTS/network/interface/README.md +++ b/TESTS/network/interface/README.md @@ -1,23 +1,21 @@ -Network Interface test plan +Network interface test plan ======================== Target API ---------- -The target for this plan is to test: +The goal of this plan is to test [NetworkInterface.h](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/NetworkInterface.h) -- [NetworkInterface.h](https://github.com/ARMmbed/mbed-os/blob/master/features/netsocket/NetworkInterface.h) - -Tools to be used +Tools needed ---------------- - Mbed OS. -- Standard Mbed OS development tools as described in [Arm Mbed tools overview](https://os.mbed.com/docs/latest/tools/index.html). +- Standard Mbed OS development tools as described in the [Arm Mbed tools overview](https://os.mbed.com/docs/latest/tools/index.html). Test environment ---------------- -- [Greentea](https://os.mbed.com/docs/mbed-os/latest/tools/greentea-testing-applications.html) +[Greentea](https://os.mbed.com/docs/mbed-os/latest/tools/greentea-testing-applications.html). Test case priorities -------------------- @@ -59,8 +57,8 @@ mbed test --compile -t -m -n mbed-os-tests-network-interfac Running tests ------------- -When device is connected to network, or in case of wireless device near -the access point. +When device is connected to network, or if a wireless device is near +the access point: ```.sh mbed test -n mbed-os-tests-network-interface @@ -75,19 +73,19 @@ Test cases for NetworkInterface class Test `NetworkInterface::connect()` and `NetworkInterface::disconnect()` -**Preconditions:** +**Precondition:** -1. Network interface is initialized. +Network interface is initialized. **Test steps:** -1. Connect interface -2. Disconnect interface -3. Repeat connect and disconnect steps 1 to 2 four times +1. Connect interface. +2. Disconnect interface. +3. Repeat connect and disconnect steps 1 to 2 four times. **Expected result:** -`Connect()` and `disconnect()` calls return NSAPI_ERROR_OK. +`Connect()` and `disconnect()` calls return `NSAPI_ERROR_OK`. ### NETWORKINTERFACE_STATUS @@ -97,49 +95,49 @@ Test `NetworkInterface::attach()` and status indications. **Preconditions:** -1. Network interface is initialized -2. Status callback is attached to network interface -3. Network interface is on blocking mode +1. Network interface is initialized. +2. Status callback is attached to the network interface. +3. Network interface is on blocking mode. **Test steps:** -1. Connect interface -2. Check that interface indicates status NSAPI_STATUS_CONNECTING -3. Optional: check that interface indicates status NSAPI_STATUS_LOCAL_UP (whether this is indicated depends on network interface type) -4. Check that interface indicates status NSAPI_STATUS_GLOBAL_UP -5. Disconnect interface -6. Check that interface indicates status NSAPI_STATUS_DISCONNECTED -7. Repeat connect and disconnect steps 1 to 6 four times +1. Connect interface. +2. Check that interface indicates status `NSAPI_STATUS_CONNECTING`. +3. Optional: Check that interface indicates status `NSAPI_STATUS_LOCAL_UP` (whether this is indicated depends on network interface type). +4. Check that interface indicates status `NSAPI_STATUS_GLOBAL_UP`. +5. Disconnect interface. +6. Check that interface indicates status `NSAPI_STATUS_DISCONNECTED`. +7. Repeat connect and disconnect steps 1 to 6 four times. **Expected result:** -Callback statuses are correct. `Connect()` and `disconnect()` calls return NSAPI_ERROR_OK. Note: network interface may indicate same status several times a row depending on network interface type. This will not fail the test. +Callback statuses are correct. `Connect()` and `disconnect()` calls return `NSAPI_ERROR_OK`. Note: The network interface may indicate the same status several times in a row, depending on the network interface type. This will not fail the test. ### NETWORKINTERFACE_STATUS_NONBLOCK **Description:** -Test `NetworkInterface::attach()` and status indications on non-blocking mode. +Test `NetworkInterface::attach()` and status indications on nonblocking mode. **Preconditions:** -1. Network interface is initialized -2. Status callback is attached to network interface -3. Network interface is on non-blocking mode +1. Network interface is initialized. +2. Status callback is attached to the network interface. +3. Network interface is on nonblocking mode. **Test steps:** -1. Connect interface -2. Check that interface indicates status NSAPI_STATUS_CONNECTING -3. Optional: check that interface indicates status NSAPI_STATUS_LOCAL_UP (whether this is indicated depends on network interface type) -4. Check that interface indicates status NSAPI_STATUS_GLOBAL_UP -5. Disconnect interface -6. Check that interface indicates status NSAPI_STATUS_DISCONNECTED -7. Repeat connect and disconnect steps 1 to 6 four times +1. Connect interface. +2. Check that interface indicates status `NSAPI_STATUS_CONNECTING`. +3. Optional: Check that interface indicates status `NSAPI_STATUS_LOCAL_UP` (whether this is indicated depends on network interface type). +4. Check that interface indicates status `NSAPI_STATUS_GLOBAL_UP`. +5. Disconnect interface. +6. Check that interface indicates status `NSAPI_STATUS_DISCONNECTED`. +7. Repeat connect and disconnect steps 1 to 6 four times. **Expected result:** -Callback statuses are correct. `Connect()` and `disconnect()` calls return NSAPI_ERROR_OK. Note: network interface may indicate same status several times a row depending on network interface type. This will not fail the test. +Callback statuses are correct. `Connect()` and `disconnect()` calls return `NSAPI_ERROR_OK`. Note: The network interface may indicate the same status several times in a row, depending on the network interface type. This will not fail the test. ### NETWORKINTERFACE_STATUS_GET @@ -149,18 +147,18 @@ Test `NetworkInterface::get_connection_status()`. **Preconditions:** -1. Network interface is initialized -2. Network interface is on blocking mode +1. Network interface is initialized. +2. Network interface is on blocking mode. **Test steps:** -1. Check that `get_connection_status()` returns status NSAPI_STATUS_DISCONNECTED -2. Connect interface -3. Poll the `get_connection_status()` until it returns status NSAPI_STATUS_GLOBAL_UP -4. Disconnect interface -5. Check that `get_connection_status()` returns status NSAPI_STATUS_DISCONNECTED -6. Repeat connect and disconnect steps 2 to 5 four times +1. Check that `get_connection_status()` returns status `NSAPI_STATUS_DISCONNECTED`. +2. Connect interface. +3. Poll the `get_connection_status()` until it returns status `NSAPI_STATUS_GLOBAL_UP`. +4. Disconnect interface. +5. Check that `get_connection_status()` returns status `NSAPI_STATUS_DISCONNECTED`. +6. Repeat connect and disconnect steps 2 to 5 four times. **Expected result:** -`Connect()` and `disconnect()` calls return NSAPI_ERROR_OK. Right status is returned by `get_connection_status()`. +`Connect()` and `disconnect()` calls return `NSAPI_ERROR_OK`. The right status is returned by `get_connection_status()`. From ae205566357b45963626ef4e33dc4848fad5e2fa Mon Sep 17 00:00:00 2001 From: Amanda Butler Date: Thu, 7 Feb 2019 16:10:38 -0600 Subject: [PATCH 4/4] Edit README.md Edit file, mostly for inclusion of articles. --- TESTS/network/emac/README.md | 97 +++++++++++++++++------------------- 1 file changed, 46 insertions(+), 51 deletions(-) diff --git a/TESTS/network/emac/README.md b/TESTS/network/emac/README.md index 254627c0b11..3bcab0f042d 100644 --- a/TESTS/network/emac/README.md +++ b/TESTS/network/emac/README.md @@ -8,7 +8,7 @@ To configure a device to be a CTP echo server, you need to enable the `echo-serv ## Other configuration options -Targets with connectivity set the target.network-default-interface-type configuration variable appropriately, either to their only interface or their most-commonly-used one. For targets that provide more than one type of connectivity, you may choose the default by overriding the target.network-default-interface-type configuration variable. +Targets with connectivity set the `target.network-default-interface-type` configuration variable appropriately, either to their only interface or their most commonly used one. For targets that provide more than one type of connectivity, you may choose the default by overriding the `target.network-default-interface-type` configuration variable. For Ethernet, if you want to overrride the default, set the `json` configuration to: @@ -21,7 +21,7 @@ For Ethernet, if you want to overrride the default, set the `json` configuration } ``` -For Wi-Fi set the `json` configuration to: +For Wi-Fi, set the `json` configuration to: ``` "target_overrides": { @@ -35,16 +35,13 @@ For Wi-Fi set the `json` configuration to: } ``` -For Wi-Fi you also need to configure Wi-Fi SSID and security options to the configuration file. +For Wi-Fi, you also need to configure Wi-Fi SSID and security options to the configuration file. Test case priorities -------------------- -Please refer to the following table for priorities of test cases. Priorities -are labeled as MUST and SHOULD. MUST means this is a requirement and -therefore mandatory to pass the test. SHOULD means it is recommended to -pass the test if the driver implements the feature in question. +Please refer to the following table for priorities of test cases. Priorities are labeled as MUST and SHOULD. MUST means this is a requirement and therefore mandatory to pass the test. SHOULD means it is recommended to pass the test if the driver implements the feature in question. | | Test case | Priority | |-----|-----------------------------------------|----------| @@ -129,101 +126,100 @@ To verify whether the echo server is receiving CTP Ethernet frames, enable `echo **Description:** -Test case initializes and connects the EMAC driver and the test network stack. +The test case initializes and connects the EMAC driver and the test network stack. -**Preconditions:** +**Precondition:** -1. Device is ready to be connected (Ethernet cable is plugged or WIFI base station is available). +The device is ready to be connected. (The ethernet cable is plugged in, or a Wi-Fi base station is available). **Test steps:** -1. Constructs the network interface - -2. Connects the network interface +1. Constructs the network interface. +2. Connects the network interface. **Expected result:** -Network interface is connected. +The network interface is connected. ### EMAC broadcast **Description:** -Test case tests basic broadcast functionality and resolves CTP echo server MAC address. +The test case tests basic broadcast functionality and resolves CTP echo server MAC address. -**Preconditions:** +**Precondition:** -1. Network interface is connected. +The network interface is connected. **Test steps:** -1. Sends three CTP broadcast messages (100 bytes each) -2. Waits for three seconds +1. Sends three CTP broadcast messages (100 bytes each). +2. Waits for three seconds. 3. Sends three CTP broadcast messages (60 bytes each). 4. Listens for the CTP echo server responses. 5. Stores the addresses of the echo servers if replies are received. **Expected result:** -Echo server replies to broadcast messages. For each sent broadcast message reply is waited for three seconds. In case reply is not received sending is repeated six times before failing the test. +The echo server replies to broadcast messages. For each sent broadcast message, a reply waits for three seconds. If the reply is not received, the reply is sent six more times before the test fails. ### EMAC unicast **Description:** -Test case tests basic unicast functionality and that the CTP echo server replies to unicast messages. +The test case tests basic unicast functionality, and the CTP echo server replies to unicast messages. -**Preconditions:** +**Precondition:** -1. Network interface is connected. +The network interface is connected. **Test steps:** 1. Sends three CTP unicast messages (100 bytes each) to the CTP echo server. -2. Verifies that all are replied. +2. Verifies all replies. **Expected result:** -Echo server replies to unicast messages. For each sent unicast message reply is waited for three seconds. In case reply is not received sending is repeated five times before failing the test. +The echo server replies to unicast messages. For each sent unicast message, a reply waits for three seconds. If the reply is not received, it is sent five more times before the test fails. ### EMAC unicast frame length **Description:** -Test case tests Ethernet frame lengths. +The test case tests Ethernet frame lengths. **Preconditions:** -1. Network interface is connected. +The network interface is connected. **Test steps:** -1. Sends CTP unicast messages with Ethernet message length from 100 bytes to the maximum defined by the MTU of the EMAC with 50 bytes increments. -2. Verifies that all are replied. +1. Sends CTP unicast messages with Ethernet message length from 100 bytes to the maximum defined by the MTU of the EMAC with 50-byte increments. +2. Verifies all replies. **Expected result:** -Echo server replies to messages that are sent to it. For each sent unicast message reply is waited for 500ms. In case reply is not received sending is repeated five times before failing the test. +The echo server replies to messages that are sent to it. For each sent unicast message, a reply waits for 500ms. If the reply is not received, it is sendt again five more times before the test fails. ### EMAC unicast burst **Description:** -Test case tests Ethernet echoing at full speed. +The test case tests Ethernet echoing at full speed. -**Preconditions:** +**Precondition:** -1. Network interface is connected. +The network interface is connected. **Test steps:** -1. Sends CTP unicast messages with Ethernet message length from 100 bytes to the maximum defined by the MTU of the EMAC with 50 bytes increments. +1. Sends CTP unicast messages with Ethernet message length from 100 bytes to the maximum defined by the MTU of the EMAC with 50-byte increments. 2. Repeats the sending 10 times. -3. Verifies that all are replied. +3. Verifies all replies. **Expected result:** -Echo server replies to messages that are sent to it. For each sent unicast message reply is waited for five seconds. In case reply is not received sending is repeated five times before failing the test. +The echo server replies to messages that are sent to it. For each sent unicast message, a reply waits for five seconds. If the reply is not received, it is sent again five more times before the test fails. ### EMAC unicast long @@ -233,27 +229,27 @@ Test case tests Ethernet echoing at full speed for an extended time. **Preconditions:** -1. Network interface is connected. +The network interface is connected. **Test steps:** 1. Sends CTP unicast messages with random Ethernet message length. -2. Repeats the sending 50000 times. -3. Verifies that all are replied. +2. Repeats the sending 50,000 times. +3. Verifies all replies. **Expected result:** -Echo server replies to messages that are sent to it. For each sent unicast message reply is waited for 350ms. In case reply is not received sending is repeated five times before failing the test. +The echo server replies to messages that are sent to it. For each sent unicast message, a reply waits for 350ms. If the reply is not received, it is sent five more times before the test fails. ### EMAC multicast filter **Description:** -Test case tests multicast filtering. Multicast filtering is an optional feature for the EMAC. The test does not fail if filtering is not implemented. +The test case tests multicast filtering. Multicast filtering is an optional feature for the EMAC. The test does not fail if filtering is not implemented. -**Preconditions:** +**Precondition:** -1. Network interface is connected. +The network interface is connected. **Test steps:** @@ -266,24 +262,24 @@ Test case tests multicast filtering. Multicast filtering is an optional feature **Expected result:** -Echo server replies to are received as expected by the test step. Supporting of the filtering will affect what messages are received. +The echo server replies are received as expected by the test step. Supporting of the filtering will affect what messages are received. ### EMAC memory **Description:** -Test case tests memory manager out-of-memory situations. The test case configures the test memory manager to reject memory buffer allocations made by the EMAC. Memory buffer allocations are divided into output and input memory allocations: +The test case tests memory manager out-of-memory situations. The test case configures the test memory manager to reject memory buffer allocations made by the EMAC. Memory buffer allocations are divided into output and input memory allocations: -- The output memory allocations are the ones made by the EMAC in the \`link\_out()\` function called by the network stack (test case). +- The output memory allocations are the ones made by the EMAC in the ``\`link\_out()\`` function called by the network stack (test case). - The input memory allocations are other memory allocations made by the EMAC. Depending on the EMAC implementation, it may or may not allocate memory manager buffers in the link output function. If the memory manager buffers are not allocated, disabling the link output memory allocations in the test does not affect the functionality. -In each test step, the test case sends CTP unicast messages with Ethernet message length from 100 bytes to the maximum defined by the MTU of the EMAC with 50 bytes increments. Memory buffers sent to the EMAC in the \`link\_out()\` function are forced to be non-aligned in this test case. +In each test step, the test case sends CTP unicast messages with Ethernet message length from 100 bytes to the maximum defined by the MTU of the EMAC with 50-byte increments. Memory buffers sent to the EMAC in the ``\`link\_out()\`` function are forced to be nonaligned in this test case. -**Preconditions:** +**Precondition:** -1. Network interface is connected. +The network interface is connected. **Test steps:** @@ -298,5 +294,4 @@ In each test step, the test case sends CTP unicast messages with Ethernet messag **Expected result:** -Echo server replies to messages that are sent to it based on whether the driver can allocate memory for frame or not. For each sent unicast message reply is waited for 500ms. In case reply is not received sending is repeated three times. If test expects that memory should be available and sending fails three times, test is failed. - +The echo server replies to messages that are sent to it based on whether the driver can allocate memory for the frame or not. For each sent unicast message, a reply waits for 500ms. If the reply is not received, it is sent another three times. If the test expects that memory should be available and sending fails three times, the test fails.