[greentea] RTL8195AM netsocket test failures #8124
Comments
|
IAR failures: |
|
@SeppoTakalo FYI.. |
|
@yennster Would you mind reconfirming that this is still an issue today? The Austin site had some network issues for most of the day yesterday, which created false failures with the netsocket test in CI. The issues seem to have resolved themselves, so I'm curious if this is still an issue as well. |
|
It looks something for @samchuarm to investigate and work with Realtek. |
|
Hi @ARMmbed/team-realtek @M-ichae-l , please help look into these GT netsocket test failures for RTL8195AM and address them as soon as you can. Thanks. |
|
By the way, do we confirm the failures are not caused by the network issue happened in Austin site as @cmonr mentioned? |
|
This is the result from our CI |
|
Thanks @SeppoTakalo . In that case, Hi @M-ichae-l , please help look into these failures. Thanks. |
|
PR #6292 is missing from new EMAC interface |
|
Since the netsocket tests were fully passed when Realtek delivered the EMAC interface, it seems these failures are due to the 5.10 updates. |
|
Hi, |
|
@samchuarm What makes you assume that all tests have passed by that time? During the EMAC work, only EMAC tests have been ran. I cannot see any evidence that these test have been ran before when this driver has been developed: |
|
@M-ichae-l Hi, can you provide the log from the test run? Also, are you running the full, extented, test set as instructed here: https://github.com/ARMmbed/mbed-os/blob/master/TESTS/netsocket/README.md For some of the testcases, we might have outdated configurations as there are now maintenance work being done in the test infra. Those will be resolved soon. This is the result from last night: Note: DNS_CACHE cases are known to fail, fix already submitted in #8171 |
|
@SeppoTakalo , I see your point. It was my misunderstanding. |
|
@M-ichae-l , Hi can you please address Seppo's question? |
|
@SeppoTakalo |
|
It should not be host OS specific. |
|
Hi @M-ichae-l , do you have update for running the tests? |
|
Hi @samchuarm, @AdamZhang0124 is currently taking over this issue. We have tested that it is not hosted OS specific. |
|
Thanks @M-ichae-l . @AdamZhang0124 , are you able to reproduce the issues now? |
|
@M-ichae-l, @samchuarm However, it seems like some test cases were not generated while I was testing. |
|
Do you see testcases in "skipped" state after a build? The reason for this is that if the testcases do not find configuration values they are expecting, they will skip the build. See example test configuration from: https://github.com/ARMmbed/mbed-os/blob/master/tools/test_configs/EthernetInterface.json Then examples, how test cases skip the build if values are not provided: |
|
Hi @AdamZhang0124 , thanks for the update. Is there a log that you can share so Seppo could help pinpoint where you might configure the tests wrong? |
|
Hi @samchuarm and @SeppoTakalo, dns_log.txt For the wifi test, there may have some setting errors that I am troubleshooting now. |
|
Hi @AdamZhang0124 , please follow @SeppoTakalo 's instruction to add the target override for MBED_CONF_LWIP_PBUF_POOL_SIZE. Hi @SeppoTakalo , does the test results for the UDP tests look ok to you? They seem all passed. |
|
Hi @samchuarm and @SeppoTakalo, I have identified the reason which causes udp test and tcp test failure. It is related to the thread stack size and packet buffer size. I need to do more tests to find suitable values. Other than that, the Network Wifi test still has problems, I will keep on working on that. |
This looks good now. |
|
@SeppoTakalo , thanks for the review. Hi @AdamZhang0124 , thanks for the work to fix these. Can you show us how you implement the fix for preview? Can you share your netsocket WiFi test results? Does it fail the same way as reported? Thanks. |
|
Hi @samchuarm , the failure of UDP test and TCP test is due to the small heap size. The default heap size is 1600. Only by changing the 'mem-size' value in mbed_lib.json to override the default value could fix this issue. The network WIFI test log is attached for your information: |
|
Thanks @AdamZhang0124 , that's quite many failures in the WiFi tests. Anything do you suspect that could cause these tests fail? Hi @SeppoTakalo and @0xc0170 , is it okay for Realtek to override the 'mem-size' value in mbed_lib.json for passing UDP/TCP netsocket tests ? |
|
Hi @samchuarm, I have found the problem. Some failures are caused because the unsecured AP was not set properly. This is the new test result and I am debugging the reminding failures. Thanks! |
|
@samchuarm and @AdamZhang0124 Yes. It is OK to override the Many platform already override the value. |
|
Hi @SeppoTakalo, I forgot to attach the full test log. |
|
Hi @AdamZhang0124 , any update on the WiFi tests? Thanks. |
|
Hi @samchuarm , so far I have fixed the WIFI-SCAN test case failure. This failure is due to the wrong expression of the 'wifi-secure-protocol' value in mbed_app.json file. There is a small typo that need to be noticed. After I change to WPA/WPA2 based on my wifi settings, this WIFI-SCAN test case is able to pass. |
|
Hi @SeppoTakalo , can you please review Adam's comments? Is this something needs a PR to address? Thanks. Hi @AdamZhang0124 , what about other WiFi test failures? Are you able to address them now? Thanks. |
|
I don't see the problem. The If you would use |
|
Hi @SeppoTakalo, In mbed_app.json, the "wifi-secure-protocol" refers to MBED_CONF_APP_WIFI_SECURE_PROTOCOL.
In Wifi_scan.cpp, the program checks the value of get_security().
In Get_security.cpp, the program compares the MBED_CONF_APP_WIFI_SECURE_PROTOCOL value with the preset char and then returns NSAPI_SECURITY_WPA_WPA2
Therefore, I believe the value of MBED_CONF_APP_WIFI_SECURE_PROTOCOL should follow the preset char naming which uses WPA/WPA2 to represent WPA_WPA2. Please correct me if I am wrong, thanks. |
|
Ah.. true.. you are correct. The The Looks like we need to unify those and use only the |
|
Hi @SeppoTakalo , thanks for the review. Then I guess WIFI-SCAN test failures are not Realtek platform specific. Hi @AdamZhang0124 , What are the remaining WiFi failures you're still working on? Any progress? |
|
Hi @samchuarm , |
|
Hi @AdamZhang0124 , this looks a lot better now. Thank you. |
|
Hi @samchuarm , I have fixed all the test case failures and start final checking the codes before submitting PR. |
|
Hi @AdamZhang0124 , that's great to know. Thanks for the work. Please submit PR to check in the fixes that you have now as soon as you could. Please associate the PR with this ticket. |
|
Hi @samchuarm and @SeppoTakalo , I have included the newly added test cases and the test is able to pass. Also, I have submitted the pull request #9084 , please help review. Btw, in the tcp test case under 'mbed-os\TESTS\netsocket\tcp', there is a missing comma in line 166 in main.cpp file which causes compile error. |
|
Hi @AdamZhang0124 , thanks for the new test results update and PR. Hi @SeppoTakalo , please help review the test results and PR changes. Thanks. |
This PR addresses the issue of ARMmbed#8124. It updates and enriches the wifi connection error type to adapt the Network Socket test plan requirement. In the meantime, it increases the heap size that allows the transmission of larger packet size. Description 1. Increase heap size in lwipstack\mbed_lib.json to fulfill bursty TCP and UDP transmission requirement. 2. Modify and enrich wifi connection error types in TARGET_AMEBA\RTWInterface.cpp to adapt the decision logic of the wifi test cases. 3. Add new static constants in TARGET_AMEBA\RTWInterface.h, including 'SSID_MAX_LENGTH', 'PASSPHRASE_MAX_LENGTH' and 'PASSPHRASE_MIN_LENGTH' to help verifying the validity of ssid and passphrase. Pull request type [x] Fix [ ] Refactor [ ] Target update [ ] Functionality change [ ] Docs update [ ] Test update [ ] Breaking change
This PR addresses the issue of #8124. It updates and enriches the wifi connection error type to adapt the Network Socket test plan requirement. In the meantime, it increases the heap size that allows the transmission of larger packet size. Description 1. Increase heap size in lwipstack\mbed_lib.json to fulfill bursty TCP and UDP transmission requirement. 2. Modify and enrich wifi connection error types in TARGET_AMEBA\RTWInterface.cpp to adapt the decision logic of the wifi test cases. 3. Add new static constants in TARGET_AMEBA\RTWInterface.h, including 'SSID_MAX_LENGTH', 'PASSPHRASE_MAX_LENGTH' and 'PASSPHRASE_MIN_LENGTH' to help verifying the validity of ssid and passphrase. Pull request type [x] Fix [ ] Refactor [ ] Target update [ ] Functionality change [ ] Docs update [ ] Test update [ ] Breaking change
Description
Environment: Windows 10
Mbed CLI version: 1.8.0
Python version: 2.7.14
IAR version: 7.80.1.11864
ARMC5 version: 5060750
Mbed OS tag: mbed-os-5.10.0-rc2
Summary of netsocket test failures with ARMC5 and the RTL8195AM board:
Issue request type
[ ] Question
[ ] Enhancement
[X] Bug
The text was updated successfully, but these errors were encountered: