Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PDMC test failed with ESP8266 #9088

Closed
ccli8 opened this issue Dec 13, 2018 · 8 comments
Closed

PDMC test failed with ESP8266 #9088

ccli8 opened this issue Dec 13, 2018 · 8 comments
Labels
type: bug

Comments

@ccli8
Copy link
Contributor

ccli8 commented Dec 13, 2018

Description

mbed-os
27e1bc3 Merge pull request #9073 from vervaekejonathan/master

simple-mbed-cloud-client
0a3221a (HEAD -> master, origin/master, origin/HEAD) Merge pull request #60 from ARMmbed/override-callbacks

mbed-cloud-client
62de4b8 (HEAD, tag: 2.0.1.1, origin/master, origin/HEAD) Merge pull request #20 from ARMmbed/release-2.0.1.1

NuMaker-mbed-SD-driver
9557d57 (HEAD -> master, origin/master, origin/HEAD) Fix user buffer is not SD DMA-compatible

Related issue or PR

#8927

I am running PDMC tests on NUMAKER_PFM_M487. On M487+Ethernet, tests are passed. On M487+ESP8266, almost simple-mbed-cloud-client-tests-basic* tests are failed. Per my check, these tests are failed in download_test() function (simple-mbed-cloud-client/TESTS/COMMON/download_test.cpp), which tries to download a large text file through HTTP. On M487+ESP8266, there are buffer overrun or packet dropped in these failed tests. As I know, ESP8266 driver can be seen with 3 level buffers:

  1. H/W UART FIFO
  2. MBED_CONF_DRIVERS_UART_SERIAL_RXBUF_SIZE (enlared from 256 to 1024) in mbed-os/drivers/UARTSerial.h(cpp)
  3. MBED_CONF_ESP8266_SOCKET_BUFSIZE (default to 8 KiB) in mbed-os/components/wifi/esp8266-driver

When M487+ESP8266 cannot consume received packets in time, these packets will be dropped when MBED_CONF_ESP8266_SOCKET_BUFSIZE limit is reached, or buffer overrun with MBED_CONF_DRIVERS_UART_SERIAL_RXBUF_SIZE. The best solution to it seems to be support for UART flow control (CTS/RTS). But for targets like M487 which cannot consume packets in time and no UART flow control ready, is there another solution?

M487+ARM+Ethernet

mbedgt: test case report:
+----------------------+------------------+----------------------------------------------------+--------------------------------------------+--------+--------+--------+--------------------+
| target               | platform_name    | test suite                                         | test case                                  | passed | failed | result | elapsed_time (sec) |
+----------------------+------------------+----------------------------------------------------+--------------------------------------------+--------+--------+--------+--------------------+
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT 1 file, buff     1                | 1      | 0      | OK     | 5.72               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT 1 file, buff     4                | 1      | 0      | OK     | 3.89               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT 1 file, buff    16                | 1      | 0      | OK     | 3.42               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT 1 file, buff    64                | 1      | 0      | OK     | 3.33               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT 1 file, buff   256                | 1      | 0      | OK     | 3.26               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT 1 file, buff  1024                | 1      | 0      | OK     | 2.89               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT 1 file, buff  4096                | 1      | 0      | OK     | 1.94               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT 1 file, buff 16384                | 1      | 0      | OK     | 1.47               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT format                            | 1      | 0      | OK     | 0.03               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-threaded   | NUSD+FAT 2 files, buff 4b/256b             | 1      | 0      | OK     | 7.18               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-threaded   | NUSD+FAT 3 files, buff 256b/1kb/4kb        | 1      | 0      | OK     | 8.33               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-threaded   | NUSD+FAT format                            | 1      | 0      | OK     | 0.02               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-net-single    | Ethernet   128 buffer                      | 1      | 0      | OK     | 9.59               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-net-single    | Ethernet   256 buffer                      | 1      | 0      | OK     | 6.23               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-net-single    | Ethernet  1024 buffer                      | 1      | 0      | OK     | 6.19               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-net-single    | Ethernet  4096 buffer                      | 1      | 0      | OK     | 6.42               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-net-single    | Ethernet network setup                     | 1      | 0      | OK     | 5.52               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-net-threaded  | Ethernet 1 thread                          | 1      | 0      | OK     | 7.05               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-net-threaded  | Ethernet 2 threads                         | 1      | 0      | OK     | 10.95              |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-net-threaded  | Ethernet network setup                     | 1      | 0      | OK     | 4.52               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-stress-net-fs | Ethernet network setup                     | 1      | 0      | OK     | 4.5                |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-stress-net-fs | NUSD+FAT format                            | 1      | 0      | OK     | 0.05               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-stress-net-fs | NUSD+FAT+Ethernet 1 thread, dl, file seq.  | 1      | 0      | OK     | 6.15               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-stress-net-fs | NUSD+FAT+Ethernet 2 threads, dl, 1kb       | 1      | 0      | OK     | 8.09               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-stress-net-fs | NUSD+FAT+Ethernet 3 threads, dl, 256b, 1kb | 1      | 0      | OK     | 10.19              |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-connect    | Connect to Ethernet                        | 2      | 0      | OK     | 4.55               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-connect    | Format FAT                                 | 1      | 0      | OK     | 0.08               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-connect    | Initialize NUSD+FAT                        | 2      | 0      | OK     | 0.03               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-connect    | Initialize Simple PDMC                     | 2      | 0      | OK     | 1.48               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-connect    | Pelion Bootstrap & Reg.                    | 1      | 0      | OK     | 9.18               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-connect    | Pelion Directory                           | 1      | 0      | OK     | 1.83               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-connect    | Pelion Re-register                         | 1      | 0      | OK     | 3.16               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-connect    | Post-reset Identity                        | 1      | 0      | OK     | 0.11               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-connect    | Resource LwM2M GET                         | 1      | 0      | OK     | 1.65               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-connect    | Resource LwM2M POST                        | 1      | 0      | OK     | 0.97               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-connect    | Resource LwM2M PUT                         | 1      | 0      | OK     | 1.34               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-connect    | Resource LwM2M SET                         | 1      | 0      | OK     | 0.73               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-update     | Connect to Ethernet                        | 2      | 0      | OK     | 4.54               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-update     | Format FAT                                 | 1      | 0      | OK     | 0.08               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-update     | Initialize NUSD+FAT                        | 2      | 0      | OK     | 0.03               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-update     | Initialize Simple PDMC                     | 2      | 0      | OK     | 1.47               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-update     | Pelion Bootstrap & Reg.                    | 1      | 0      | OK     | 8.84               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-update     | Pelion Directory                           | 1      | 0      | OK     | 1.36               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-update     | Pelion Firmware Download                   | 1      | 0      | OK     | 140.81             |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-update     | Pelion Firmware Prepare                    | 1      | 0      | OK     | 0.58               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-update     | Pelion Firmware Update                     | 1      | 0      | OK     | 20.3               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-update     | Pelion Re-register                         | 1      | 0      | OK     | 2.99               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-update     | Post-update Erase                          | 1      | 0      | OK     | 0.01               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-dev_mgmt-update     | Post-update Identity                       | 1      | 0      | OK     | 0.11               |
+----------------------+------------------+----------------------------------------------------+--------------------------------------------+--------+--------+--------+--------------------+
mbedgt: test case results: 49 OK

M487+ARM+ESP8266

mbedgt: test case report:
+----------------------+------------------+----------------------------------------------------+--------------------------------------------+--------+--------+---------+--------------------+
| target               | platform_name    | test suite                                         | test case                                  | passed | failed | result  | elapsed_time (sec) |
+----------------------+------------------+----------------------------------------------------+--------------------------------------------+--------+--------+---------+--------------------+
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT 1 file, buff     1                | 1      | 0      | OK      | 5.72               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT 1 file, buff     4                | 1      | 0      | OK      | 3.84               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT 1 file, buff    16                | 1      | 0      | OK      | 3.48               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT 1 file, buff    64                | 1      | 0      | OK      | 3.27               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT 1 file, buff   256                | 1      | 0      | OK      | 3.25               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT 1 file, buff  1024                | 1      | 0      | OK      | 2.9                |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT 1 file, buff  4096                | 1      | 0      | OK      | 1.94               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT 1 file, buff 16384                | 1      | 0      | OK      | 1.48               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-single     | NUSD+FAT format                            | 1      | 0      | OK      | 0.06               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-threaded   | NUSD+FAT 2 files, buff 4b/256b             | 1      | 0      | OK      | 7.2                |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-threaded   | NUSD+FAT 3 files, buff 256b/1kb/4kb        | 1      | 0      | OK      | 8.27               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-fs-threaded   | NUSD+FAT format                            | 1      | 0      | OK      | 0.01               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-net-single    | Ethernet   128 buffer                      | 0      | 1      | FAIL    | 3.17               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-net-single    | Ethernet   256 buffer                      | 0      | 0      | SKIPPED | 0.0                |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-net-single    | Ethernet  1024 buffer                      | 0      | 0      | SKIPPED | 0.0                |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-net-single    | Ethernet  4096 buffer                      | 0      | 0      | SKIPPED | 0.0                |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-net-single    | Ethernet network setup                     | 1      | 0      | OK      | 7.33               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-net-threaded  | Ethernet 1 thread                          | 0      | 1      | FAIL    | 2.05               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-net-threaded  | Ethernet 2 threads                         | 0      | 0      | SKIPPED | 0.0                |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-net-threaded  | Ethernet network setup                     | 1      | 0      | OK      | 7.32               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-stress-net-fs | Ethernet network setup                     | 1      | 0      | OK      | 7.33               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-stress-net-fs | NUSD+FAT format                            | 1      | 0      | OK      | 0.03               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-stress-net-fs | NUSD+FAT+Ethernet 1 thread, dl, file seq.  | 0      | 1      | FAIL    | 2.03               |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-stress-net-fs | NUSD+FAT+Ethernet 2 threads, dl, 1kb       | 0      | 0      | SKIPPED | 0.0                |
| NUMAKER_PFM_M487-ARM | NUMAKER_PFM_M487 | simple-mbed-cloud-client-tests-basic-stress-net-fs | NUSD+FAT+Ethernet 3 threads, dl, 256b, 1kb | 0      | 0      | SKIPPED | 0.0                |
+----------------------+------------------+----------------------------------------------------+--------------------------------------------+--------+--------+---------+--------------------+
mbedgt: test case results: 3 FAIL / 6 SKIPPED / 16 OK

Issue request type

[ ] Question
[ ] Enhancement
[X] Bug
@ciarmcom
Copy link
Member

Internal Jira reference: https://jira.arm.com/browse/MBOCUSTRIA-313

@screamerbg
Copy link
Contributor

FYI @samchuarm

@screamerbg
Copy link
Contributor

@VeijoPesonen

@VeijoPesonen
Copy link
Contributor

@ccli8 my recommendation would be that you should update ESP8266's firmware to version 1.7.0. With that firmware the driver doesn't allocate memory for TCP packets but uses application provided buffers directly instead. That would be the solution in case of TCP. For further details please read ARMmbed/mbed-os/master/components/wifi/esp8266-driver/README.md

@ccli8
Copy link
Contributor Author

ccli8 commented Dec 19, 2018

@VeijoPesonen Thanks for your information.
1.7.0 AT requires 16Mb or above flash, but M487 IoT on-board ESP8266 module just has 8Mb flash. Currently, we cannot upgrade the on-board ESP8266 module to 1.7.0 AT. Besides, we fly-wire another 32Mb ESP8266 module which upgrades to 1.7.0 AT for PDMC tests. It is passed. Currently, we are struggling for the upgrade...
@samchuarm

@0xc0170
Copy link
Contributor

0xc0170 commented Dec 19, 2018

1.7.0 AT requires 16Mb or above flash, but M487 IoT on-board ESP8266 module just has 8Mb flash.

cc @SeppoTakalo

@SeppoTakalo
Copy link
Contributor

Please run full set on Mbed OS socket tests for this platform as instructed in Network Socket test plan.

If it passes the Mbed OS tests, then you should continue testing with Client.

@SeppoTakalo
Copy link
Contributor

From: https://www.espressif.com/en/products/hardware/esp8266ex/resources

ESP8266 AT Bin V1.7.0
Based on ESP8266_NonOS_SDK V3.0.0.

Limited by the size of the AT bin file,  only `1024+1024 flash map` is supported by default.

We have test cases that proof the 1.6.2 and older to be not working properly, and therefore we cannot support those.
For any commercial platform, the minimum requirement would be that lates ESP AT firmware is installed and CTS, RTS and Reset for the ESP8266 as well.

And as we cannot support this, we need to close this as "Won't fix"

@ccli8 ccli8 mentioned this issue Dec 26, 2018
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type: bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@0xc0170 @SeppoTakalo @screamerbg @ciarmcom @ccli8 @VeijoPesonen