ESP-TLS memory quirck

[kebner]

Hi

I'm quite new to ESP32 and stumbled upon a problem that kept me up all day/night now: I try to get esp_tls library working and tried to followed the examples but found that something is messing wirth the memory:
I want to connect to a https server and used the example code like_
tls = esp_tls_conn_http_new(WEB_URL, &cfg);
But after that, the code crashed due to illegal addresses for the write/read callbacks. That made me to add some line to esp_tls.c like

Code: Select all

    /* Connect to host */
    if (esp_tls_conn_new_sync(&url[u.field_data[UF_HOST].off], u.field_data[UF_HOST].len,
                     get_port(url, &u), cfg, tls) == 1) {
printf( "DEEP UNDER 3. tls = %p, read = %p, write = %p\n", tls, tls->read, tls->write );			
		return tls;						 
    }
    return NULL;

The tls pointer was the same after the function call, but the content of the structure was completly overwritten:

Code: Select all

DEEP UNDER 3. tls = 0x3ffba908, read = 0x400d29f8, write = 0x400d2998

And upon return from calling tls = esp_tls_conn_http_new(WEB_URL, &cfg);

Code: Select all

A. tls struct. tls = 0x3ffba908, read = 0x0, write = 0x0

What on earth or ESP32 could overwrite the struct with 0 again? These two printf were literally called 2 statements apart!

I added calls to heap_caps_check_integrity_all all over with no prevail. I've created the task with a lot of memory:

xTaskCreate(&wpa2, "wpa2", 4096*4, NULL, 5, NULL);

I then added a funtion to esp_tls.c/esp_tls.h to see if that would help clarif the issue, but it got worse:
Added to esp_tls.c

Code: Select all

struct esp_tls * t1( struct esp_tls *tls )
{
    printf( "t1 tls = %p, read = %p\n", tls, tls->read );
    tls->read = 0xababab;
    printf( "t1 tls = %p, read = %p\n", tls, tls->read );

    return tls;
}

and in my calling function:

Code: Select all

    struct esp_tls t_, *t;
    memset( &t_, 0, sizeof(esp_tls_t));
    t = &t_;
heap_caps_check_integrity_all( true );
    t = t1( t );
heap_caps_check_integrity_all( true );
    printf( "After t1 tls = %p\n", t );
    printf( "After t1 tls = %p, read = %p\n", t, t->read );
   

The terminal showed the puzzling

Code: Select all

t1 tls = 0x3ffc9c50, read = 0x0
t1 tls = 0x3ffc9c50, read = 0xababab
After t1 tls = 0x3ffc9c50
After t1 tls = 0x3ffc9c50, [b]read = 0x0[/b]

What is happening to "my" memory here? Why are values set in a library get overwritten when returning to main?

Any ideas? I'm lost.

Thanks
Klaus

[ESP_Angus]

Hi Klaus,

Thanks for all these details. Could you please post the full function that exhibits this problem, as well? (If it's part of a more complex function, is it possible to create a minimal example function that still does this?)

Are you sure that all fields in the "cfg" structure are being initialized before it's passed into esp_tls_conn_http_new? This is usually done with a C99 initializer or by calling bzero/memset on the structure.

Also, please let us know which ESP-IDF version you have.

[kebner]

Hi Angus

thanks for reaching out! This is the most basic example I could create. The error seems to be that libraries "somehow" got their own version of memory and that returning back into the project, these two memories differ. This sounds absolutely crazy, I know. Therefore, I must have made some mistakes...

As mentioned, I've added a small function to esp_tls.c (and a prototype of that to esp_tls.h):

Code: Select all

struct esp_tls * t1( struct esp_tls *tls )
{
    printf( "t1 tls = %p, read = %p\n", tls, tls->read );
    tls->read = 0xababab;
    printf( "t1 tls = %p, read = %p\n", tls, tls->read );
    return tls;
}

It really easy (sorry for omitting any error checks - just for illustration): Print out the pointer address and the content of read. Change that to some arbitrary value and return that.

This is the part where I call it:

Code: Select all

void app_main()
{

    struct esp_tls t_, *t;
    memset( &t_, 0, sizeof(esp_tls_t));
    t = &t_;

    t = t1( t );

    printf( "After t1 tls = %p\n", t );
    printf( "After t1 tls = %p, read = %p (t_.read = %p)\n", t, t->read, t_.read );
...

You see, I created the struct on my heap, set it to 0 and then call teh above shown function t1...

And this is the log output:

Code: Select all

t1 tls = 0x3ffb46d0, read = 0x0
t1 tls = 0x3ffb46d0, read = 0xababab
After t1 tls = 0x3ffb46d0
After t1 tls = 0x3ffb46d0, read = 0x0 (t_.read = 0x0)

It is the same pointer in app_main and in t1, t1 changed the read pointer - BUT: in app_main, the read pointer is 0.

I'm lost.

sdkconfig is attached, CMakeList.txt (trivial):

Code: Select all

# Warning! This code was automatically generated for projects
# without default 'CMakeLists.txt' file.

set(app_sources
	"main.c"
)

idf_component_register(SRCS ${app_sources})

I use ESP-IDF with PlatformIO/VSCode. AFAIK, this should be 4.0.0 When booting, it's shown as "cpu_start: ESP-IDF: HEAD-HASH-NOTFOUND"...

Thanks,
Klaus

[pikkio]

Hi Klaus, I'm in your exact situation and I'm going crazy about it. Have you managed to solve the issue? I don't know what else I can try to figure it out.

Thanks
Simone

[kebner]

Hi Simone

actually: No, I've abandoned hope in 2020. But...
I've just recompiled my code using
* PlatformIO 5.2.5
* Espressif 3.3.2 (Yes, I know, it's outdated)
and: It works.

Just don't ask me why.

Kind regards
Klaus

[pikkio]

Hi Klaus, a few hours after I posted here, I managed to understand the issue: the structure esp_tls_t was being declared differently between the translation unit of my code and the one of mbedtls, so mbedtls allocated, say 1700 bytes for the structure, while on the outside the code thought it was 2100 bytes, with a different layout. including the header mbedtls/esp_config.h inside esp_tls.h, before the others mbedtls includes, resolved the issue. This is imho a bug in esp-idf.

Now on to the next issue: understanding why closing a connection wastes 5 seconds...

Regards
Simone

[ESP_Mahavir]

including the header mbedtls/esp_config.h inside esp_tls.h, before the others mbedtls includes, resolved the issue. This is imho a bug in esp-idf.

Inclusion of "esp_config.h" happens through https://github.com/espressif/esp-idf/bl ... s.txt#L181

`MBEDTLS_CONFIG_FILE` is part of pretty much all public headers in mbedTLS and hence I do not see this as a bug.

Could you please help to confirm this? Or maybe share some application that shows this problem?

Thanks.

[gmdriscoll]

CONFIRMED:
Adding the include to esp_tls.h fixes the error.
#ifdef CONFIG_ESP_TLS_USING_MBEDTLS
+ #include "mbedtls/esp_config.h"
#include "mbedtls/platform.h"

I am using platformio and esp idf ver 4.3.2.
- framework-espidf @ 3.40302.0 (4.3.2)
- tool-cmake @ 3.16.4
- tool-esptoolpy @ 1.30300.0 (3.3.0)
- tool-idf @ 1.0.1
- tool-mconf @ 1.4060000.20190628 (406.0.0)
- tool-ninja @ 1.9.0
- toolchain-esp32ulp @ 1.22851.191205 (2.28.51)
- toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch3

The application is the espressif provided https_request_example_main.c with no changes except adding in the wifi connection code and changing out the pem file.

struct esp_tls *tls = esp_tls_conn_http_new(WEB_URL, &cfg);
ret = esp_tls_conn_write(tls,
REQUEST + written_bytes,
sizeof(REQUEST) - written_bytes);

The connection is made, but the write fails with this error:
Guru Meditation Error: Core 0 panic'ed (InstrFetchProhibited). Exception was unhandled.

[emp_tamarin]

Experienced this esp-tls bug today! Establishing the connection shows no errors but once esp_tls_conn_write(...) is called it crashes. Adding #include "mbedtls/esp_config.h" at the top of the .c file, as suggested by previous replies, makes it work.

I'm using PlatformIO with ESP-IDF v4.4.3.