Trimming building unnecessary IDF modules

[eriksl]

This is my CMakeLists.txt:

Code: Select all

# The following lines of boilerplate have to be in your project's
# CMakeLists in this exact order for cmake to work correctly
cmake_minimum_required(VERSION 3.16)

include($ENV{IDF_PATH}/tools/cmake/project.cmake)

set(COMPONENTS main bt esp_psram wear_levelling)

project(esp32-s3-develop)

It has a minimal set of components. If I am leaving any of these out, I get compilation errors.

While building IDF/CMake decides these components are also required. But many of them aren't. I am not using mcpwm or C++. Is there a way to forcefully leave them out:

Code: Select all

[ 32%] Built target __idf_esp_driver_mcpwm
[ 33%] Built target __idf_esp_driver_gptimer
[ 34%] Built target __idf_esp_driver_pcnt
[ 38%] Built target __idf_cxx
[ 67%] Built target __idf_efuse
[ 71%] Built target mbedx509

Especially the C++ support I'd like to drop, I suspect it accounts for most of the bloat in the image.

Also is there a way to build without internet connectivity? It looks like CMake checks for new versions of some packages on EVERY build run.

[MicroController]

Is there a way to forcefully leave them out:

Nope.

I suspect it accounts for most of the bloat in the image

It doesn't. Things (i.e. symbols, i.e. functions and variables) that are built but never used/referenced by the application are removed during linking of the application binary.

[eriksl]

I am aware of that.

But

• building takes more time than necessary
• apparently this isn't completely true. The linker cannot always know whether a symbol is really unused.

The image now is ridiculously large. On my ESP8266 I had images that did a lot more than this and it all fitted in 500 kB. I now have a minimal functionality image and it takes almost 1000 kB! I have two OTA partitions of 1500 kB, so I fear for the future. On the ESP8266 I could have two complete OTA images in 2 MB flash and even then some space spare for fonts etc.\

I really think I should be able to disable all C++ support and that really would make a huge difference. libstdc++ isn't small and it gets linked in for a big part.

[MicroController]

Did you enable compiler optimizations in menuconfig?

Then, idf.py size-components may be helpful to see what's actually going on.

I really think I should be able to disable all C++ support and that really would make a huge difference.

Negative
The IDF itself has some C++ code (at least the NVS component, IIRC), and the use of C++ in an application has (almost?) no impact on the size of the binary. Enabling C++ exceptions may add a few percent though.

[eriksl]

Of course I selected "-Os" everywhere.

And yes, that's what I was afraid of. No way to entirely disable C++. Don't get me wrong, I am a great fan of C++. But not on a microcontroller with limited resources. A pity they decided this way.

[MicroController]

And yes, that's what I was afraid of. No way to entirely disable C++. Don't get me wrong, I am a great fan of C++. But not on a microcontroller with limited resources. A pity they decided this way.

"C++ consumes more resources" is quite a common mis-belief. It does not.
At least the language does not. Of course, your code can go haywire with Strings, Vectors, Lambdas and stuff if you choose, but you don't have to.
In fact, I found that C++'s template 'metaprogramming' is a very powerful tool to make the compiler do stuff you'd have to do at runtime in C, and thus it's especially suited to MCUs. Additionally, the 'modern C++' paradigm of defining (member) functions inline unlocks a lot of optimization possibilities for the compiler.

As I said above, libstdc++ is not what's hogging your flash.

[eriksl]

It's not the code, agreed. But why would you want to use C++ if you can't use exceptions and stl containers? For object oriented programming you don't really need C++.

And really, as soon as you start using templates, code gets duplicated.

As said I really see the advantages of using C++ (including all the beautiful stuff like real classes (which of course you can emulate in plain C too), STL, Boost, etc). But microcontrollers like ESP's are simply too lightweight for it. My proof is that for considerately more functionality, on almost the same architecture (ESP8266, C), I need half of the flash memory compared to ESP32, which includes parts written in C++ that clog up the memory footprint. And then my version on the ESP8266 also includes a correct (!) software, bit-banging I2C interface, a high resolution / high frequency PWM generator, both of which the ESP32 can do in hardware, plus a complete robust OTA implementation. ~35000 lines of C code, image is 480 kB. On the ESP32 I only get to do "hello world" for this. As soon as you start using wifi and bluetooth, the image already doubles in size. So if you want to do OTA and a bit of NVS, you need at least 4 MB flash! I think it's all hard to explain.

[MicroController]

But why would you want to use C++ if you can't use exceptions and stl containers?

Classes, member functions, visibilities, constructors, destructors (RAII), automatic type conversion, user-defined operators, namespaces, constexpr, references, exceptions, templates ("compile-time polymorphism") + concepts, atomics, user-defined literals, and a bunch of other 'goodies' from std (e.g. <array>, <chrono>, <atomic>, <string_view>,...)...

[eriksl]

All of these which are handy an nice if they're facilitated by the compiler. But they're not enabling something you can't do in C (with correct programming, something many peolpe struggle with).

And as said, I am a great fan of C++. It's my preference for anything not microcontroller based (when not using perl, when a text filtering interpreted language is more suitable).

Ok so let's rephrase my question.

Why does an image only containing printf("hello world") (and some very basic wifi) take almost 1 MB of space on the flash. I am quite convinced this can be improved one way or another. If Espressif can get 50% off without removing C++ support, even better! Maybe they should take the compiler/linker flags, like -fdata-sections/-fcode-sections and -Os versus -O3 for the libraries. Apparently GNU ld isn't that smart if you don't instruct it to. If you do not use any of the -f options, it will drag in all of the module when only when at least one symbol is used. Only with -f-*sections it will also filter on whether each symbol is actually used, within the module (by making each symbol a separate section).

Maybe they very well already do all of this. But then, they really should look harder. This is unacceptable! I am really worried for the future, when I will start adding some real functionality to my image. I now have 1.5 MB available for each image, because in total I have 4 MB flash, so now already on 2/3 of the available space. How long before this slot size will become too small? I will be screwed then.

[boarchuz]

Why does an image only containing printf("hello world") (and some very basic wifi) take almost 1 MB of space on the flash.

ESP-IDF'S default config generally favours development-friendly options over release optimisations. A project as you've described, aggressively optimised for size, should be almost half of that, in my experience.