Compatibility between "normal CMake" and ESP-IDF

[ESP_Angus]

Generic CMake build support pushed to GitHub master branch a few hours ago (commit).

Docs:
https://docs.espressif.com/projects/esp ... components

New examples:
https://github.com/espressif/esp-idf/tr ... stem/cmake

[permal]

Generic CMake build support pushed to GitHub master branch a few hours ago (commit).

Oh, I can't wait to get home to try this out!

[Oleg Endo]

Nice! Thanks a lot!

[chodzikman]

Hi!

Thank you for the updated build system, however - I do experience some issues:

Using almost exactly your example:

Code: Select all

cmake_minimum_required(VERSION 3.5)
project(idf_as_lib C)

# The source file main.c contains app_main() definition
add_executable(${CMAKE_PROJECT_NAME}.elf application.h application.c)

# Provides idf_import_components() and idf_link_components()
include(esp-idf/tools/cmake/idf_functions.cmake)

# Create artifacts used for flashing the project to target chip
set(IDF_BUILD_ARTIFACTS ON)
set(IDF_PROJECT_EXECUTABLE ${CMAKE_PROJECT_NAME}.elf)
set(IDF_BUILD_ARTIFACTS_DIR ${CMAKE_BINARY_DIR})

# Trim down components included in the build. Although freertos and spi_flash are the ones needed by the application
# itself, the bootloader and esptool_py components are also needed in order to create the artifacts to be used
# for flashing to the target chip
set(IDF_COMPONENTS freertos spi_flash bootloader esptool_py)

# Wraps add_subdirectory() to create library targets for components, and then return them using the specified variable
idf_import_components(components esp-idf esp-idf)

# Wraps target_link_libraries() to link processed components by idf_import_components to target
idf_link_components(${CMAKE_PROJECT_NAME}.elf "${components}")

I get this:

Code: Select all

-- The C compiler identification is GNU 8.2.1
-- Check for working C compiler: /bin/cc
-- Check for working C compiler: /bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
CMake Error at esp-idf/tools/cmake/idf_functions.cmake:16 (include):
  include could not find load file:

    utilities
Call Stack (most recent call first):
  CMakeLists.txt:11 (include)


CMake Error at esp-idf/tools/cmake/idf_functions.cmake:17 (include):
  include could not find load file:

    components
Call Stack (most recent call first):
  CMakeLists.txt:11 (include)


CMake Error at esp-idf/tools/cmake/idf_functions.cmake:18 (include):
  include could not find load file:

    kconfig
Call Stack (most recent call first):
  CMakeLists.txt:11 (include)


CMake Error at esp-idf/tools/cmake/idf_functions.cmake:19 (include):
  include could not find load file:

    targets
Call Stack (most recent call first):
  CMakeLists.txt:11 (include)


CMake Error at esp-idf/tools/cmake/idf_functions.cmake:20 (include):
  include could not find load file:

    git_submodules
Call Stack (most recent call first):
  CMakeLists.txt:11 (include)


CMake Error at esp-idf/tools/cmake/idf_functions.cmake:21 (include):
  include could not find load file:

    GetGitRevisionDescription
Call Stack (most recent call first):
  CMakeLists.txt:11 (include)


CMake Error at esp-idf/tools/cmake/idf_functions.cmake:22 (include):
  include could not find load file:

    crosstool_version_check
Call Stack (most recent call first):
  CMakeLists.txt:11 (include)


CMake Error at esp-idf/tools/cmake/idf_functions.cmake:23 (include):
  include could not find load file:

    ldgen
Call Stack (most recent call first):
  CMakeLists.txt:11 (include)


CMake Error at esp-idf/tools/cmake/idf_functions.cmake:25 (set_default):
  Unknown CMake command "set_default".
Call Stack (most recent call first):
  CMakeLists.txt:11 (include)


-- Configuring incomplete, errors occurred!
See also "/home/mchodzikiewicz/CLionProjects/beerbob_fw_esp32/cmake-build-debug/CMakeFiles/CMakeOutput.log".

[Finished]

Any advice?

[permal]

Menu config doesn't work anymore. This is what I get when running "make menuconfig" on the idf-as-lib example (the project itself builds). Same thing running ~/esp/esp-idf/tools/idf.py -C .. menuconfig

Code: Select all

Scanning dependencies of target mconf-idf
[  0%] Creating directories for 'mconf-idf'
[  0%] No download step for 'mconf-idf'
[  0%] No patch step for 'mconf-idf'
[100%] No update step for 'mconf-idf'
[100%] No configure step for 'mconf-idf'
[100%] Performing build step for 'mconf-idf'
[100%] No install step for 'mconf-idf'
[100%] Completed 'mconf-idf'
[100%] Built target mconf-idf
Scanning dependencies of target menuconfig
/home/permal/esp/esp-idf/Kconfig:15: warning: 'option env="IDF_TARGET"' on symbol IDF_TARGET_ENV has no effect, because the environment variable IDF_TARGET is not set
No such file or directory
esp-idf/CMakeFiles/menuconfig.dir/build.make:57: recipe for target 'esp-idf/CMakeFiles/menuconfig' failed
make[3]: *** [esp-idf/CMakeFiles/menuconfig] Error 1
CMakeFiles/Makefile2:442: recipe for target 'esp-idf/CMakeFiles/menuconfig.dir/all' failed
make[2]: *** [esp-idf/CMakeFiles/menuconfig.dir/all] Error 2
CMakeFiles/Makefile2:449: recipe for target 'esp-idf/CMakeFiles/menuconfig.dir/rule' failed
make[1]: *** [esp-idf/CMakeFiles/menuconfig.dir/rule] Error 2
Makefile:255: recipe for target 'menuconfig' failed
make: *** [menuconfig] Error 2

Also, how are include directories from IDF-components supposed to be published? I've had to add lines like these so that #includes can be found (I'm still converting from the previous CMake system so I might be to blame here):

Code: Select all

include_directories(/home/permal/esp/esp-idf/components/lwip/port/esp32/include
                    /home/permal/esp/esp-idf/components/lwip/lwip/src/include
                    /home/permal/esp/esp-idf/components/vfs/include
                    /home/permal/esp/esp-idf/components/newlib/platform_include
                    /home/permal/esp/esp-idf/components/esp32/include
        )

Edit; Also, sdkconfig.h seeming isn’t generated during the build so it currently stops when trying to include it.

[permal]

I've not been able to make any more progress with this, not even with the included example (idf_as_lib).

My questions are still:

- How do I run menuconfig for a project like idf_as_lib to generate my sdkconfig and subsequent sdkconfig.h file?
- How are include paths for components supposed to be propagated to the higher-level component (e.g. esp-idf/components/esp32/include)?

I would love some feedback on this so that I can complete the update of my framework during the weekend.

[ESP_Angus]

Hi permal,

(For benefit of anyone skim reading the replies, this post is about Generic CMake projects including ESP-IDF as a series of libraries, not the other way around.)

- How do I run menuconfig for a project like idf_as_lib to generate my sdkconfig and subsequent sdkconfig.h file?

I can reproduce the menuconfig target failing as well, will fix.

Normal (non-menuconfig) generation of sdkconfig.h happens during the build. Note that if building a project with IDF as a library, CMake needs to invoked with some additional arguments (because this isn't an ESP-IDF project CMakeLists.txt, there is less "magic"). The necessary arguments are shown in the README:
https://github.com/espressif/esp-idf/tr ... is-example

idf.py won't work for generic CMake projects, as it doesn't pass these arguments (and there's no way for idf.py to know how a generic CMake project is structured). Will see if we can make that a clearer error.

- How are include paths for components supposed to be propagated to the higher-level component (e.g. esp-idf/components/esp32/include)?

The idf_import_components() step adds all of the components as libraries to the build and idf_link_components() adds them as dependencies of the target executable.

Similar to any other component library target in CMake, the dependent libraries' public headers are added to the executable's include paths automatically (this is how idf_as_lib project's main.c file can include FreeRTOS headers, etc).

If you have other library subprojects added to the top level CMake project that need to access IDF components' headers, you'll need to import those headers in the same way you would for a normal CMake project which contains multiple static libraries. Calling idf_import_components() and idf_link_components() creates library targets of the form idf_component_COMPONENTNAME for all components, so these targets can be used in other CMake code.

For example, if you have some subdirectory "mylib" that calls add_library(mylib STATIC ...) then you can do something like this:

Code: Select all

add_subdirectory(mylib)
target_link_libraries(${CMAKE_PROJECT_NAME}.elf mylib)

target_include_directories(mylib PRIVATE
    $<TARGET_PROPERTY:idf_component_esp32,INCLUDE_DIRECTORIES>)

(The INCLUDE_DIRECTORIES property includes the esp32 component's public directories and all its dependent public header directories.)

Will add some of these details to the documentation.

I'll also see if there's a way we can make an IDF_INCLUDE_DIRECTORIES variable available in the parent CMake project, similar to how this variable can be used when including a CMake subproject into an ESP-IDF project - as I can see that this would make this step a bit simpler. But the above pure CMake approach will work.
EDIT: I misunderstood the role of this variable, generator expressions are probably the easiest method for adding dependencies to external libraries.

[ESP_Angus]

Hi chodzikman,

Thank you for the updated build system, however - I do experience some issues:

Thanks for the detailed report. My colleague tells me they've reproduced this failure and a fix is on the way.

Angus

[permal]

...

It was all my fault; it all compiles now which is one step closer

However, I can't get the linker finalize, it exists with the following error. To my eyes this looks like low-level stuff, i.e. things from the binary blobs that are shipped with IDF. I'm using the latest xtensa gcc (xtensa-esp32-elf-linux64-1.22.0-80-g6c4433a-5.2.0) and have updated all sub modules of the master branch (1cffc90e58c84a7a4daa801c5864110210c4d767). Searching for some of the missing references, such as rtc_slp_prep only gives a match in .map-files, and no exact hit.

Is there a newer tool chain available or is this once again something on my side?

Code: Select all

[ 98%] Linking CXX executable main
/home/permal/esp/esp-idf/components/esp32/lib/librtc.a(pm.o): In function `pm_set_sleep_cycles':
/home/qgu/git_tree/chip7.1_rtc/board_code/app_test/pp/rtc/pm.c:60: undefined reference to `rtc_slowck_cali'
/home/permal/esp/esp-idf/components/esp32/lib/librtc.a(pm.o): In function `pm_rtc_clock_cali':
/home/qgu/git_tree/chip7.1_rtc/board_code/app_test/pp/rtc/pm.c:68: undefined reference to `rtc_slowck_cali'
/home/permal/esp/esp-idf/components/esp32/lib/librtc.a(pm.o):(.text.pm_set_sleep_mode_full+0x10): undefined reference to `rtc_slp_prep'
/home/permal/esp/esp-idf/components/esp32/lib/librtc.a(pm.o): In function `pm_set_sleep_mode_full':
/home/qgu/git_tree/chip7.1_rtc/board_code/app_test/pp/rtc/pm.c:237: undefined reference to `rtc_slp_prep'
/home/qgu/git_tree/chip7.1_rtc/board_code/app_test/pp/rtc/pm.c:265: undefined reference to `rtc_slp_prep'
/home/qgu/git_tree/chip7.1_rtc/board_code/app_test/pp/rtc/pm.c:308: undefined reference to `rtc_slp_prep'
/home/qgu/git_tree/chip7.1_rtc/board_code/app_test/pp/rtc/pm.c:358: undefined reference to `rtc_slp_prep'
/home/permal/esp/esp-idf/components/esp32/lib/librtc.a(pm.o):/home/qgu/git_tree/chip7.1_rtc/board_code/app_test/pp/rtc/pm.c:370: more undefined references to `rtc_slp_prep' follow
/home/permal/esp/esp-idf/components/esp32/lib/librtc.a(rtc.o): In function `rtc_soc_clk_ck12m':
/home/qgu/git_tree/chip7.1_rtc/board_code/app_test/pp/rtc/rtc.c:230: undefined reference to `rtc_init_clk'
/home/permal/esp/esp-idf/components/esp32/lib/librtc.a(rtc.o): In function `rtc_init_full':
/home/qgu/git_tree/chip7.1_rtc/board_code/app_test/pp/rtc/rtc.c:333: undefined reference to `rtc_init_clk'
/home/permal/esp/esp-idf/components/esp32/lib/librtc.a(rtc.o): In function `rtc_slp_prep_lite_12M':
/home/qgu/git_tree/chip7.1_rtc/board_code/app_test/pp/rtc/rtc.c:448: undefined reference to `rtc_slp_prep'
/home/permal/esp/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/lib/libstdc++.a(thread.o):(.literal._ZNSt6thread20hardware_concurrencyEv+0x0): undefined reference to `sysconf'
/home/permal/esp/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/lib/libstdc++.a(thread.o): In function `std::thread::hardware_concurrency()':
/builds/idf/crosstool-NG/.build/src/gcc-5.2.0/libstdc++-v3/src/c++11/thread.cc:160: undefined reference to `sysconf'
collect2: error: ld returned 1 exit status
test/main/CMakeFiles/main.dir/build.make:143: recipe for target 'test/main/main' failed
make[2]: *** [test/main/main] Error 1
CMakeFiles/Makefile2:258: recipe for target 'test/main/CMakeFiles/main.dir/all' failed
make[1]: *** [test/main/CMakeFiles/main.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2

[ESP_Angus]

Hi permal,

So you don't see any CMake warnings about git submodules when CMake runs? (git submodulate update --init --recursive should be all up to date).

Can you please run "make VERBOSE=1" and post the full command line used for linking?

Angus