Memory gets smaller and smaller

[tobewinner]

In my esp32, using esp-idf v2.1, there are socket with mbedtls and original sockets.
Those sockets will be close and resocket frequently.
a few time after the esp32 starts to run, I notice the memory gots smaller and smaller using

Code: Select all

esp_get_free_heap_size

then I wrap thoes func

Code: Select all

mallc

Code: Select all

calloc

Code: Select all

realloc

Code: Select all

free

with some log in my code, Trying to find the leak bug, hower every memory I got from [c-m-re]alloc, is freed.

Is there any other ways to debug the memory leak bug?

Besides I found a bug in mbedtls/port/net.c

Code: Select all

mbedtls_net_connect

Code: Select all

mbedtls_net_bind

Below is the original src of mbedtls_net_connect

Code: Select all

/*
 * Initiate a TCP connection with host:port and the given protocol
 */
int mbedtls_net_connect( mbedtls_net_context *ctx, const char *host, const char *port, int proto )
{
    int ret;
    struct addrinfo hints, *addr_list, *cur;

    if ( ( ret = net_prepare() ) != 0 ) {
        return ( ret );
    }

    /* Do name resolution with both IPv6 and IPv4 */
    memset( &hints, 0, sizeof( hints ) );
    hints.ai_family = AF_UNSPEC;
    hints.ai_socktype = proto == MBEDTLS_NET_PROTO_UDP ? SOCK_DGRAM : SOCK_STREAM;
    hints.ai_protocol = proto == MBEDTLS_NET_PROTO_UDP ? IPPROTO_UDP : IPPROTO_TCP;

    if ( getaddrinfo( host, port, &hints, &addr_list ) != 0 ) {
        return ( MBEDTLS_ERR_NET_UNKNOWN_HOST );
    }

    /* Try the sockaddrs until a connection succeeds */
    ret = MBEDTLS_ERR_NET_UNKNOWN_HOST;
    for ( cur = addr_list; cur != NULL; cur = cur->ai_next ) {
        ctx->fd = (int) socket( cur->ai_family, cur->ai_socktype,
                                cur->ai_protocol );
        if ( ctx->fd < 0 ) {
            ret = MBEDTLS_ERR_NET_SOCKET_FAILED;
            continue;
        }

        if ( connect( ctx->fd, cur->ai_addr, cur->ai_addrlen ) == 0 ) {
            ret = 0;
            break;
        }

        close( ctx->fd );
        ret = MBEDTLS_ERR_NET_CONNECT_FAILED;
    }

    freeaddrinfo( addr_list );

    return ( ret );
}

hower if mbedtls_net_connect failed for connect, then ctx->fd is not -1, the subsequent call to mbedtls_net_free will result to some bad influence(may close sockets using by other thread).

[kolban]

When you allocate resources such as TCP/IP sockets, these also consume RAM. Be sure that you are releasing those resources before allocating new ones. It has also been my experience that the ESP-IDF maintains "memory pools" that initially start with low size but will grow until a maximum is used and then stop growing. This would "appear" to the external viewer like a memory leak but actually isn't as there is an upper bound to the amount of memory used by those pools. It isn't surprising for me to see memory used going from:

0K -> 5K-> 10K -> 15K -> 20K -> 25K ... which would appear to be a memory leak, however if watch for a longer time I see:

30K -> 35K -> 40K -> 40K -> 40K -> 40K

(Note these numbers are made up to illustrate the notion of used memory increasing over time but reaching a stable state).

[ESP_Angus]

Those sockets will be close and resocket frequently.
a few time after the esp32 starts to run, I notice the memory gots smaller and smaller using

Code: Select all

esp_get_free_heap_size

To expand on kolban's answer, closed TCP sockets will appear to "leak" some memory because of the TIME_WAIT state. If you wait over a couple of minutes, you should find that this memory use stabilises as the TIME_WAIT sockets time out and are properly freed.

For incoming sockets, you can set use setsockopt to set SO_REUSEADDR on the socket, in order to mitigate this problem.

Is there any other ways to debug the memory leak bug?

There are some new memory leak tracing & checking features coming in IDF v3.0, but they're not available quite yet.

Besides I found a bug in mbedtls/port/net.c

Code: Select all

mbedtls_net_connect

Code: Select all

mbedtls_net_bind

Thanks for reporting this. A fix will be available in IDF master branch soon, for the forthcoming V3.0 release.

[tobewinner]

When you allocate resources such as TCP/IP sockets, these also consume RAM. Be sure that you are releasing those resources before allocating new ones. It has also been my experience that the ESP-IDF maintains "memory pools" that initially start with low size but will grow until a maximum is used and then stop growing. This would "appear" to the external viewer like a memory leak but actually isn't as there is an upper bound to the amount of memory used by those pools. It isn't surprising for me to see memory used going from:

0K -> 5K-> 10K -> 15K -> 20K -> 25K ... which would appear to be a memory leak, however if watch for a longer time I see:

30K -> 35K -> 40K -> 40K -> 40K -> 40K

(Note these numbers are made up to illustrate the notion of used memory increasing over time but reaching a stable state).

Thanks, After 30+ hours running, I found memory usage is on some stable state as you explained.
However 40K is a large amount of memory, it would be better if there is a way to turn the "memory pools" to low size.

[ESP_Angus]

However 40K is a large amount of memory, it would be better if there is a way to turn the "memory pools" to low size.

I think Kolban made up 40K in this case to illustrate the point. How much RAM are you seeing being used?

There are some configurating tuning options for both WiFi and LWIP (under the menuconfig menu) to reduce memory overhead, but they can have a negative performance impact. I wouldn't recommend modifying them unless you're experiencing low memory problems.

[tobewinner]

However 40K is a large amount of memory, it would be better if there is a way to turn the "memory pools" to low size.

I think Kolban made up 40K in this case to illustrate the point. How much RAM are you seeing being used?

There are some configurating tuning options for both WiFi and LWIP (under the menuconfig menu) to reduce memory overhead, but they can have a negative performance impact. I wouldn't recommend modifying them unless you're experiencing low memory problems.

I'm not sure, after 30+ hours running, nearly 20~30K memory is "lost" in my esp32.
I'm using mbedtls, one tls socket uses nearly 30~40K memory, and bt uses almost 40~50K.
If the sockets "memory pool" eats 40K again. The memory will get very tight for me.
So I have to pay more attention to the memory usage, if there is a better way to reduce the system(maybe lwip or mbedtls) memory usage(such as decreasing the max number of open sockets), it will be a good news for me.