ESP-IDF memory usage

[Oromis]

Hi,

I just upgraded our ESP32 firmware from 3.1.x to 3.2.x and then to 3.3.1. Unfortunately it seems that the RAM footprint increased dramatically:

• On 3.1.x we had about 51 kB of free RAM (as reported by esp_get_free_heap_size()). Our application is quite large.
• On 3.2.x it went down to 48 kB (which is not dramatic)
• On 3.3.1 we only have ~20 kB if free RAM, causing our app to crash occasionally during peak load when most RAM is needed. We need at least 25 kB of free RAM to be safe.

We don't need any of the new features introduced in the newer versions at the moment, so I disabled all of them and enabled any RAM-saving features I could find.

In our next hardware revision we'll try to add an external RAM chip but for all products in the field we need to support a version without the extra chip. I tried to enable support for the SPI RAM chip, but apparently that increases RAM consumption even more (even if the external RAM isn't initialized yet) so that our app fails to start up due to failed memory allocations.

How can I reduce the RAM footprint of ESP-IDF? Is there any way to debug which subsystem takes up the RAM to figure out if I can disable any more systems? Will ESP-IDF's RAM footprint keep increasing? Because at the moment it seems like we're stuck on 3.2.x with our app due to RAM shortage...

Thanks for any input!

[PeterR]

You will need to list which features/major components you are using.
You could start with:

Code: Select all

make -j8 size-components

and post the results.

Check if any .data sections are the result of missing a 'const'.
Ethernet/lwip RX/TX buffer settings might help (but loose some performance).

What are you doing to use 20KB+ heap?

[Oromis]

Thanks for the suggestions, here are the results for size-components:

Code: Select all

Total sizes:
 DRAM .data size:   14284 bytes
 DRAM .bss  size:   36056 bytes
Used static DRAM:   50340 bytes ( 130396 available, 27.9% used)
Used static IRAM:   89757 bytes (  41315 available, 68.5% used)
      Flash code:  914464 bytes
    Flash rodata:  527256 bytes
Total image size:~1545761 bytes (.bin may be padded larger)
Per-archive contributions to ELF file:
            Archive File DRAM .data & .bss   IRAM Flash code & rodata   Total
               libutil.a          0      5      0       2126   276711  278842
             libstdc++.a        149   5640      0     123292    72407  201488
           libnet80211.a        937   9042  11532     111824    21666  155001
            libmbedtls.a        100    264     30     105230    27961  133585
                  libc.a          0      4      0      96765     7811  104580
               liblwip.a         21   3856      0      78436    15256   97569
                 libpp.a       1317   5411  23928      38623     5096   74375
              liblogic.a          0     73      0      57188     9117   66378
             libdriver.a        147     79   5458      24906    17971   48561
         libperipheral.a          0     30      0      40569     6395   46994
                libphy.a       1604    930   6487      30450        0   39471
              libesp32.a       2342    464   7408      13514     8844   32572
                libwpa.a          0    686      0      21417     2577   24680
            libservice.a          4     91      0      14949     9451   24495
            libnetwork.a          0    523      0      16949     3310   20782
           libfreertos.a       4140    788  13138          0     1776   19842
       libnetwork_real.a        156   3763      0      10873     4870   19662
            libos_real.a          9   1094    170      11117     3886   16276
     libwpa_supplicant.a          0      0      0      12894     2318   15212
              libcloud.a          0     73      0      12318     2097   14488
          libnvs_flash.a          0     32      0      10799     3066   13897
             liblogger.a          0   1399      0       8948     2195   12542
             libmodbus.a          0     11      0       9123     1782   10916
                libsoc.a        160      4   4738        502     3955    9359
          libspi_flash.a         24    291   5590       1262      928    8095
           libethernet.a          8    279    322       5258     1649    7516
         libcloud_real.a          0      1      0       5429     1961    7391
                libgcc.a          4     20    104       5779      888    6795
               libheap.a       1143      4   3051        737      932    5867
 libbootloader_support.a          0      4    862       3074     1905    5845
                libvfs.a        240    103      0       3680      413    4436
               libmain.a          0      2      0       3126     1176    4304
         libapp_update.a          0     12     85       2443     1336    3876
        libesp_ringbuf.a          0      0   3234          0      582    3816
            libcoexist.a       1278     28      0       2227        0    3533
         libstatistics.a          0      5      0       2936      470    3411
      libtcpip_adapter.a          0    124      0       2571      370    3065
              libefuse.a         36      4      0        849     2052    2941
               libjson.a         12      8      0       2248       40    2308
                  libm.a          4      0      0       2157       11    2172
             libnewlib.a        152    272    774        787       96    2081
             libupdate.a          0      2      0       1629      417    2048
            libpthread.a         16     12    236        823      645    1732
          libutil_real.a          0      6      0        914      630    1550
                 libos.a          0     46      0       1269      168    1483
        libupdate_real.a          0      1      0        669      642    1312
                liblog.a          8    268    434        359      117    1186
               libcore.a          0     29      0        825      302    1156
                libcxx.a         12     16      0        681      443    1152
    libsmartconfig_ack.a          0      1      0        809      299    1109
                libhal.a          0      0    519          0       32     551
                librtc.a          0      0    484          0        0     484
             libBase64.a          0     25      0        383       73     481
               libmesh.a          1      1      0         43        0      45
               libwpa2.a          0      1      0          0        0       1
                libwps.a          0      1      0          0        0       1

My understanding of how to interpret this is rather limited to be honest. As far as I know, esp_get_free_heap_size() reports the number of free bytes in IRAM plus the number of free bytes in DRAM, right?

This line confuses me:

Code: Select all

Used static IRAM:   89757 bytes (  41315 available, 68.5% used)

If there's 41k of IRAM available and I am using 90k, how is that 68.5% usage? My math would suggest that I'm overusing at 217%..., even though we only have a few functions marked with IRAM_ATTR.

What are you doing to use 20KB+ heap?

Allocating stuff I guess Our app is quite complex - it's the firmware for a home battery controller with lots of custom components. We're doing TLS-Encrypted Websocket communication, Modbus RTU communication, UART communication, lots of internal calculations based on sensor readings, state of charge calculation, use of NVS for local buffering...

A big chunk of dynamically allocated memory are std::stringstreams used to generate JSON strings (multiple kB in size).

Ethernet/lwip RX/TX buffer settings might help (but loose some performance).

Sure, but thing is that these RAM issues only came up after updating ESP-IDF. On 3.1.x it was completely fine and 3.2.x seems okay as well. Did other people experience the same decrease of available RAM as we did after upgrading?

Thanks!

[PeterR]

Your DRAM looks about right. Mine is 3KB lower on IDF 4.1, but you may be using more components.

esp_get_free_heap_size()

I don't know. Use

Code: Select all

size_t heap_caps_get_total_size(uint32_t caps)

instead. You need memory which will hold variables. IRAM can if 32 bit aligned.

If there's 41k of IRAM available and I am using 90k, how is that 68.5% usage?

41K available i.e. left over to allocate. 90+41K total.

A big chunk of dynamically allocated memory are std::stringstreams used to generate JSON strings (multiple kB in size).

Well that would explain your need for 25KB dynamic heap. You need to be careful. Two or more tasks using may have bigger peaks (depends on free strategy).
Doesn't explain your initial starting figure though. Your starting figure may be stack & that is easy to measure and trim.

Sure, but thing is that these RAM issues only came up after updating ESP-IDF. On 3.1.x it was completely fine and 3.2.x seems okay as well.

I don't know but new features = more memory. The change log >3.2 suggests that more memory have been freed.

[Oromis]

Thanks for the reply!

You need to be careful. Two or more tasks using may have bigger peaks (depends on free strategy).

We are aware. We know about all locations in the code where dynamic memory is allocated and we know an upper bound for each of these allocations. There's also a mechanism to stop two sections using lots of memory from running simultaneously.

Your starting figure may be stack & that is easy to measure and trim.

I just did that and while our stack sizes were optimized already, we had more tasks than strictly necessary. I just merged a few tasks and saved ~10k of stack space, finally allowing my app to boot on 3.3.1. Thanks for the hint

Still, I really don't like the base libraries constantly consuming more and more RAM with each update ...