Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Memory leak with esp_http_client + CONFIG_MBEDTLS_DYNAMIC_BUFFER (IDFGH-4251) #6104

Closed
boarchuz opened this issue Nov 11, 2020 · 4 comments
Closed

Memory leak with esp_http_client + CONFIG_MBEDTLS_DYNAMIC_BUFFER (IDFGH-4251) #6104

boarchuz opened this issue Nov 11, 2020 · 4 comments

Comments

@boarchuz
Copy link
Contributor

Environment

  • Module or chip used: ESP32-WROOM-32D
  • IDF version: v4.3-dev-1720-g494a124d9
  • Compiler version: xtensa-esp32-elf-gcc (crosstool-NG esp-2020r3) 8.4.0

Problem Description

A small amount of memory (~16b) is leaked with each HTTPS connection using esp_https_client with CONFIG_MBEDTLS_DYNAMIC_BUFFER enabled.
No memory is leaked if CONFIG_MBEDTLS_DYNAMIC_BUFFER is disabled.
Heap trace suggests this is due to LwIP making a series of 4b allocations but I'm having trouble debugging further.

Steps to reproduce

Using esp_http_client_example as a base, replace http_test_task function with the following:

static void http_test_task(void *pvParameters)
{
    uint32_t loop_num = 0;
    uint32_t prev_free_heap = 0;
    int total_heap_diff = 0;
    for(;;)
    {
        https_with_url();
        uint32_t this_free_heap = esp_get_free_heap_size();
        if(loop_num > 0)
        {
            int this_heap_diff = this_free_heap - prev_free_heap;
            total_heap_diff += this_heap_diff;
            ESP_LOGI(TAG, "free: %6u\tchange: %6d\ttotal: %6d", this_free_heap, this_heap_diff, total_heap_diff);
        }
        ++loop_num;
        prev_free_heap = this_free_heap;
    }
}

Enable CONFIG_MBEDTLS_DYNAMIC_BUFFER

#
# mbedTLS
#
CONFIG_MBEDTLS_INTERNAL_MEM_ALLOC=y
# CONFIG_MBEDTLS_DEFAULT_MEM_ALLOC is not set
# CONFIG_MBEDTLS_CUSTOM_MEM_ALLOC is not set
CONFIG_MBEDTLS_ASYMMETRIC_CONTENT_LEN=y
CONFIG_MBEDTLS_SSL_IN_CONTENT_LEN=16384
CONFIG_MBEDTLS_SSL_OUT_CONTENT_LEN=4096
CONFIG_MBEDTLS_DYNAMIC_BUFFER=y
# CONFIG_MBEDTLS_DYNAMIC_FREE_PEER_CERT is not set
# CONFIG_MBEDTLS_DYNAMIC_FREE_CONFIG_DATA is not set
# CONFIG_MBEDTLS_DEBUG is not set

Debug Logs

CONFIG_MBEDTLS_DYNAMIC_BUFFER Disabled (no leak?)

I (9704) HTTP_CLIENT: free: 218256      change:      0  total:      0
I (13794) HTTP_CLIENT: free: 218264     change:      8  total:      8
I (17494) HTTP_CLIENT: free: 218264     change:      0  total:      8
I (20854) HTTP_CLIENT: free: 218268     change:      4  total:     12
I (24244) HTTP_CLIENT: free: 218268     change:      0  total:     12
I (27614) HTTP_CLIENT: free: 218268     change:      0  total:     12
I (30994) HTTP_CLIENT: free: 218268     change:      0  total:     12
I (34374) HTTP_CLIENT: free: 218268     change:      0  total:     12
I (38064) HTTP_CLIENT: free: 218264     change:     -4  total:      8
I (41444) HTTP_CLIENT: free: 218264     change:      0  total:      8
I (44824) HTTP_CLIENT: free: 218268     change:      4  total:     12
I (48204) HTTP_CLIENT: free: 218268     change:      0  total:     12
I (51574) HTTP_CLIENT: free: 218268     change:      0  total:     12
I (54964) HTTP_CLIENT: free: 218268     change:      0  total:     12
I (58334) HTTP_CLIENT: free: 218268     change:      0  total:     12
I (61714) HTTP_CLIENT: free: 218268     change:      0  total:     12
I (65094) HTTP_CLIENT: free: 218268     change:      0  total:     12
I (68474) HTTP_CLIENT: free: 218080     change:   -188  total:   -176
I (71854) HTTP_CLIENT: free: 218268     change:    188  total:     12
I (75234) HTTP_CLIENT: free: 218268     change:      0  total:     12
I (78614) HTTP_CLIENT: free: 218268     change:      0  total:     12
I (81994) HTTP_CLIENT: free: 218268     change:      0  total:     12
I (85374) HTTP_CLIENT: free: 218080     change:   -188  total:   -176
I (88764) HTTP_CLIENT: free: 218080     change:      0  total:   -176
I (92124) HTTP_CLIENT: free: 218268     change:    188  total:     12
I (95514) HTTP_CLIENT: free: 218268     change:      0  total:     12
I (98884) HTTP_CLIENT: free: 218268     change:      0  total:     12
I (102264) HTTP_CLIENT: free: 218268    change:      0  total:     12
I (105954) HTTP_CLIENT: free: 218268    change:      0  total:     12
I (109334) HTTP_CLIENT: free: 218268    change:      0  total:     12
I (112714) HTTP_CLIENT: free: 218080    change:   -188  total:   -176
I (116094) HTTP_CLIENT: free: 218268    change:    188  total:     12
I (119474) HTTP_CLIENT: free: 218268    change:      0  total:     12

CONFIG_MBEDTLS_DYNAMIC_BUFFER Enabled (~16b avg leak)

I (9525) HTTP_CLIENT: free: 218232      change:    -16  total:    -16
I (12905) HTTP_CLIENT: free: 218220     change:    -12  total:    -28
I (16275) HTTP_CLIENT: free: 218204     change:    -16  total:    -44
I (19965) HTTP_CLIENT: free: 218188     change:    -16  total:    -60
I (23345) HTTP_CLIENT: free: 218172     change:    -16  total:    -76
I (26725) HTTP_CLIENT: free: 218156     change:    -16  total:    -92
I (30105) HTTP_CLIENT: free: 218136     change:    -20  total:   -112
I (33485) HTTP_CLIENT: free: 218120     change:    -16  total:   -128
I (36865) HTTP_CLIENT: free: 218100     change:    -20  total:   -148
I (40245) HTTP_CLIENT: free: 218076     change:    -24  total:   -172
I (43625) HTTP_CLIENT: free: 218064     change:    -12  total:   -184
I (47005) HTTP_CLIENT: free: 218044     change:    -20  total:   -204
I (50375) HTTP_CLIENT: free: 217660     change:   -384  total:   -588
I (53755) HTTP_CLIENT: free: 218020     change:    360  total:   -228
I (57135) HTTP_CLIENT: free: 217996     change:    -24  total:   -252
I (60515) HTTP_CLIENT: free: 217988     change:     -8  total:   -260
I (63895) HTTP_CLIENT: free: 217964     change:    -24  total:   -284
I (67275) HTTP_CLIENT: free: 217944     change:    -20  total:   -304
I (70655) HTTP_CLIENT: free: 217928     change:    -16  total:   -320
I (74035) HTTP_CLIENT: free: 217904     change:    -24  total:   -344
I (77415) HTTP_CLIENT: free: 217884     change:    -20  total:   -364
I (80795) HTTP_CLIENT: free: 217880     change:     -4  total:   -368
I (84185) HTTP_CLIENT: free: 217848     change:    -32  total:   -400
I (87565) HTTP_CLIENT: free: 217836     change:    -12  total:   -412
I (90935) HTTP_CLIENT: free: 217816     change:    -20  total:   -432
I (94305) HTTP_CLIENT: free: 217804     change:    -12  total:   -444
I (97695) HTTP_CLIENT: free: 217788     change:    -16  total:   -460
I (101685) HTTP_CLIENT: free: 217772    change:    -16  total:   -476
I (105065) HTTP_CLIENT: free: 217756    change:    -16  total:   -492
I (108445) HTTP_CLIENT: free: 217740    change:    -16  total:   -508
I (111815) HTTP_CLIENT: free: 217720    change:    -20  total:   -528
I (115195) HTTP_CLIENT: free: 217704    change:    -16  total:   -544
I (118575) HTTP_CLIENT: free: 217688    change:    -16  total:   -560
I (121955) HTTP_CLIENT: free: 217672    change:    -16  total:   -576
I (125335) HTTP_CLIENT: free: 217656    change:    -16  total:   -592
I (128715) HTTP_CLIENT: free: 217636    change:    -20  total:   -612
I (132095) HTTP_CLIENT: free: 217616    change:    -20  total:   -632
I (135475) HTTP_CLIENT: free: 217600    change:    -16  total:   -648
I (138855) HTTP_CLIENT: free: 217584    change:    -16  total:   -664
I (142235) HTTP_CLIENT: free: 217564    change:    -20  total:   -684
I (145615) HTTP_CLIENT: free: 217548    change:    -16  total:   -700
@github-actions github-actions bot changed the title Memory leak with esp_http_client + CONFIG_MBEDTLS_DYNAMIC_BUFFER Memory leak with esp_http_client + CONFIG_MBEDTLS_DYNAMIC_BUFFER (IDFGH-4251) Nov 11, 2020
@Alvin1Zhang
Copy link
Collaborator

Thanks for reporting, we will look into.

@AxelLin
Copy link
Contributor

AxelLin commented Nov 12, 2020

@boarchuz

FYI, there is a fix in ESP8266_RTOS_SDK:
espressif/ESP8266_RTOS_SDK@7ffb082#diff-267dab0fdb0a487503dba92eb56e659f8889988d82c7608231db43604a5bbd79

This fix can be applied to esp-idf, you can check if it helps.

@boarchuz
Copy link
Contributor Author

@AxelLin
Thanks, that fixes it

@AxelLin
Copy link
Contributor

AxelLin commented Nov 17, 2020

BTW, here is another fix you might want to apply if you set CONFIG_MBEDTLS_DYNAMIC_FREE_PEER_CERT=y.
espressif/ESP8266_RTOS_SDK@b6e45c2

projectgus pushed a commit that referenced this issue May 21, 2021
@donghengqaz @espressif-bot
fix(mbedtls): fix mbedtls dynamic resource memory leaks and RSA cert …
176d6fa 
…drop earlier

RX process caches the session information in "ssl->in_ctr" not in "ssl->in_buf".
So when freeing the SSL, can't free the "ssl->in_ctr", because the "ssl->in_buf"
is empty.

Make the RX process like TX process, and cache the session information in
"ssl->in_buf", so that the cache buffer can be freed when freeing the SSL.

Closes #6104
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@AxelLin @Alvin1Zhang @boarchuz