ESP32 Forum

Posted: **Mon Oct 24, 2016 4:40 am**

The way I understand the story is that ESP-IDF provides a set of components. When we then download an application template and compile it, the default is that ALL the components in ESP-IDF are compiled and linked together along with the code in <apptemplate>/main.

However what if I only want a subset of all possible ESP-IDF components included in the compile/link steps? If I read the "build_system.rst" file found here:

https://github.com/espressif/esp-idf/bl ... system.rst

It seems that I can edit the <apptemplate>/Makefile and add a:

COMPONENTS=<component names>

entry to explicitly name the the components I wish to include (the default would otherwise be all that could be found). However when I try this (for example setting COMPONENTS=nvs_flash), I find that the compilation of my app template now fails:

Code: Select all

GENCONFIG
#
# configuration written to /home/kolban/esp32/esptest/apps/workspace/cpp/sdkconfig
#
make: *** No rule to make target '/home/kolban/esp32/esptest/apps/workspace/cpp/build/app-template.bin', needed by 'all_binaries'.  Stop.

Posted: **Mon Oct 24, 2016 10:09 pm**

Yes such feature will be great to speed up build time.

It will be very interesting to also check other compiler option a bit like on ARM CortexMx with GCC 4.8 or more with LTO option (-flto) which heavily reduce final bin/elf size (elimination of unused functions) and also optimize speed.
As for example 02_blink the generated bin is very big 238KB to just blink a LED (even if I know it use FreeRTOS ...)

Definition of LTO:
Link Time Optimization (LTO) gives GCC the capability of dumping its internal representation (GIMPLE) to disk, so that all the different compilation units that make up a single executable can be optimized as a single module

For more details see https://gcc.gnu.org/wiki/LinkTimeOptimizationFAQ

See some benchmarks here:
http://www.phoronix.com/scan.php?page=n ... px=MTI5ODE

I have tested -flto option on latest trunk esp/esp-idf/examples/02_blink (from today)
And there is no any change on final binary size
Modifications done in esp-idf/make/project.mk (diff result):

Code: Select all

diff --git "a/esp-idf/make/project.mk_flto" "b/esp-idf/make/project.mk"
index 1c52233..67e2e92 100644
--- "a//esp-idf/make/project.mk_flto"
+++ "b/esp-idf/make/project.mk"
@@ -86,7 +86,7 @@ COMPONENT_PATHS_BUILDABLE := $(foreach cp,$(COMPONENT_PATHS),$(if $(wildcard $(c
 # Assemble global list of include dirs (COMPONENT_INCLUDES), and
 # LDFLAGS args (COMPONENT_LDFLAGS) supplied by each component.
 COMPONENT_INCLUDES :=
-COMPONENT_LDFLAGS := -flto
+COMPONENT_LDFLAGS :=
 #
 # Also add any inter-component dependencies for each component.
 
@@ -185,7 +185,7 @@ OPTIMIZATION_FLAGS = -Og
 endif
 
 # Enable generation of debugging symbols
-OPTIMIZATION_FLAGS += -flto -ggdb
+OPTIMIZATION_FLAGS += -ggdb
 
 # List of flags to pass to C compiler
 # If any flags are defined in application Makefile, add them at the end.

It will be interesting to also build all blob/lib(*.a) provided by Espressif with same option -flto to have a real gain
I also suspect -flto is just ignored/not implemented by actual xtensa-esp32-elf-gcc (crosstool-NG crosstool-ng-1.22.0-59-g8d95cad) 4.8.5
Could we have confirmation ?

Posted: **Mon Oct 24, 2016 11:59 pm**

This is correct, 4.8.5 doesn't support LTO.
We are testing 5.2.0 for building libraries and esp-idf, and will announce as soon as tests are done.

Posted: **Tue Oct 25, 2016 12:11 am**

kolban wrote: entry to explicitly name the the components I wish to include (the default would otherwise be all that could be found). However when I try this (for example setting COMPONENTS=nvs_flash), I find that the compilation of my app template now fails:

This is possible, but it's a bit sub-optimal right now. The core list of COMPONENTS for a minimal build is quite large, and the failures can be counter-intuitive (to say the least). This Makefile works with the idf-template repository:

Code: Select all

PROJECT_NAME := app-template

COMPONENTS=freertos esp32 newlib esptool_py nvs_flash spi_flash log tcpip_adapter lwip main xtensa-debug-module driver bt

include $(IDF_PATH)/make/project.mk

If you get strange compile errors when changing the list of components, you may also need to do a 'make clean'.

(It's worth noting that the resulting binary is the exact same size whether the list of components is trimmed or not, the only difference at the moment is the number of files compiled and then never linked.)

When we get time, we plan to make this process easier and more configurable (and the minimal subset of components a lot smaller!).

hydrabus wrote:Yes such feature will be great to speed up build time.

At the moment the best things you can do to speed up builds is to enable ccache (either via symlink, or by setting the compiler prefix to "ccache xtensa-esp32-elf-gcc") and to use parallel builds (ie make -jN).

However I agree that in the future being able to not compile certain components will help a lot.

hydrabus wrote: I have tested -flto option on latest trunk esp/esp-idf/examples/02_blink (from today)

Adjusting the LDFLAGs is a bit tricky, that initial COMPONENT_LDFLAGS declaration is a false start unfortunately. You can do it like this:

Code: Select all

diff --git a/make/project.mk b/make/project.mk
index 67e2e92..60161e6 100644
--- a/make/project.mk
+++ b/make/project.mk
@@ -137,7 +137,7 @@ export PROJECT_PATH
 
 # Set default LDFLAGS
 
-LDFLAGS ?= -nostdlib \
+LDFLAGS ?= -nostdlib -flto \
        -L$(IDF_PATH)/lib \
        -L$(IDF_PATH)/ld \
        $(addprefix -L$(BUILD_DIR_BASE)/,$(COMPONENTS) $(SRCDIRS)) \
@@ -185,7 +185,7 @@ OPTIMIZATION_FLAGS = -Og
 endif
 
 # Enable generation of debugging symbols
-OPTIMIZATION_FLAGS += -ggdb
+OPTIMIZATION_FLAGS += -flto -ggdb
 
 # List of flags to pass to C compiler
 # If any flags are defined in application Makefile, add them at the end.

... however the binary will fail to link because of "call target out of range" errors.

As far as I know, this is a limitation of gcc xtensa support that will need to be fixed in the toolchain.

. LTO does work on very simple binaries with xtensa, for example I use it when building the stub flasher program for esptool.py (and it shaves off a substantial amount of the binary size as well.) It'd be great to be able to use it with esp-idf.

Posted: **Thu Mar 02, 2017 1:34 am**

Hey @ESP_Angus,

Was there any work done during the upgrade to gcc 5.2.0 to include the ability to use LTO?

If not, is it still a case of the xtensa tool chain not supporting this correctly yet?

Thanks in advance!

Posted: **Wed Mar 08, 2017 11:13 pm**

Lucas.Hutchinson wrote: Was there any work done during the upgrade to gcc 5.2.0 to include the ability to use LTO?

If not, is it still a case of the xtensa tool chain not supporting this correctly yet?

Not yet, but it's on my list to look into.

I think it won't work without some modifications to the toolchain, I don't know how major they will need to be!

Posted: **Thu Mar 09, 2017 12:53 am**

Ok no worries. Glad to hear it is still on the to-do list.

Hopefully the mods are not tooo major!

Posted: **Tue May 01, 2018 10:30 am**

Hi,

I am struggling with my .bin image size. It is more than 1 MB. Even simple programming with enable C++ exception using C++ Http and WebSocket is about 750 kb.

How Can I reduce my mage size. how to tell gcc compiler to include code only which is used. I think It is bringing in all unused code as well.

-flto linker option does not make different and serial upload compress option as well makes no difference in output .bin size.

Thanks,
Naeem

Posted: **Tue May 01, 2018 1:56 pm**

Having an executable over 1MByte is not uncommon. The vast majority is the ESP-IDF which simply has to be that big by its nature. A normal ESP32 has 4MBytes of flash ... meaning you are not short on flash. The only time size would be an issue is if you needed OTA services and had to split the flash in half for two simultaneous images.

Posted: **Tue May 01, 2018 4:29 pm**

Yes I need OTA services and had to split the flash in half for two simultaneous images.

What are options?
Can I use 2 MB partition each for both images? how to setup more than 1 MB?

I also need 2 MB for web files such as css. js. I guess they can go into sd card flash.

ESP32 Forum

[Answered] How to build a subset of the possible components in ESP-IDF?

[Answered] How to build a subset of the possible components in ESP-IDF?

Re: How to build a subset of the possible components in ESP-IDF?

Re: How to build a subset of the possible components in ESP-IDF?

Re: How to build a subset of the possible components in ESP-IDF?

Re: [Answered] How to build a subset of the possible components in ESP-IDF?

Re: [Answered] How to build a subset of the possible components in ESP-IDF?

Re: [Answered] How to build a subset of the possible components in ESP-IDF?

Re: [Answered] How to build a subset of the possible components in ESP-IDF?

Re: [Answered] How to build a subset of the possible components in ESP-IDF?

Re: [Answered] How to build a subset of the possible components in ESP-IDF?