Not able to perform OTA using ESP32 with AWS S3

[rahulbari]

hi,

I am trying to run the https://github.com/espressif/esp-idf/tr ... _https_ota sample code to test the OTA over HTTPS, instead of web server I am using S3 bucket.

When i configure like below

ESP_LOGI(TAG, "Starting Advanced OTA example");

esp_err_t ota_finish_err = ESP_OK;
esp_http_client_config_t config = {
.url = CONFIG_EXAMPLE_FIRMWARE_UPGRADE_URL,
.cert_pem = NULL,
.timeout_ms = CONFIG_EXAMPLE_OTA_RECV_TIMEOUT,
.keep_alive_enable = true,
};

i am getting following error output on console

W (6711) esp_https_ota: Continuing with insecure option because CONFIG_ESP_HTTPS_OTA_ALLOW_HTTP is set.
I (6711) advanced_https_ota_example: OTA started
E (6941) esp-tls-mbedtls: No server verification option set in esp_tls_cfg_t structure. Check esp_tls API reference
E (6941) esp-tls-mbedtls: Failed to set client configurations, returned [0x8017] (ESP_ERR_MBEDTLS_SSL_SETUP_FAILED)

Need correct procedure to make it work the example code, using the S3 bucket. I need to use secured manner to perform the OTA.
Please enlight with any experience you have,

Thankyou,
Rahul B.

[fanmen1]

Can you provide more info on how your S3 bucket is configured?
error "No server verification option set in esp_tls_cfg_t structure" suggests that you require a SSL/TLS certificate to make the connection.
you have set the .cert_pem = NULL, pointing to no certificate.
try using the default esp_crt_bundle and check the logs, you can enable mbedtls verbose logging for more clear debug logs

[rahulbari]

First of all, thank you so much for your reply.

I've configured the S3 bucket on AWS with public access. Inside this bucket, I've stored the "hello-world.bin" file for performing OTA updates.

and heres the output after verbose on showing no file

I (13645) advanced_https_ota_example: Starting Advanced OTA example
W (13645) esp_https_ota: Continuing with insecure option because CONFIG_OTA_ALLOW_HTTP is set.
D (13665) HTTP_CLIENT: Begin connect to: https://rahulawstestbucket1.s3.ap-south ... ws.com:443
D (13665) esp-tls: host:rahulawstestbucket1.s3.ap-south-1.amazo�aws.com: strlen 47
W (13855) wifi:<ba-add>idx:0 (ifx:0, 8a:e1:0b4a:85), tid:0, ssn:4, winSize:64
V (13865) wifi:sta recv dup seq=64 tid=0, discard
D (13865) esp-tls: [sock=54] Resolved IPv4 address: 3.5.208.135
D (13865) esp-tls: [sock=k4] Connecting to server. HOST: rahulawstestbucket1.s3.ap-south-1.amazonaws.com, Port: 443
D (13965) esp-tls: handshake in progress...
D (15115) HTTP_CLIENT: Write header[3]: GET /hello-world.bin HTTP/1.1
User-Agent: ESP32 HTTP Client/1.0
Host: rahulawstestbucket1.s3.ap-south-1.amazonaws.com
Content-Length: 0

D (15275) HTTP_CLIENT: on_message_begin
D (15275) HTTP_CLIENT: HEADER=x-amz-request-id:9EJSVY16G778E79D
D (15275) HTTP_CLIENT: HEADER=x-amz-id-2:S7rP9YedZr4hlGYTbr+oSbzzhOYZnvopByE+aj28xKWYsMfOAEwECExxOPBcMc/l92JLnMBxJKXjOuXsgBO6mw==
D (15285) HTTP_CLIENT: HEADER=Content-Type:application/xml
D (15295) HTTP_CLIENT: HEADER=Transfer-Encoding:chunked
D (15295) HTTP_CLIENT: HEADER=Date:Fri, 26 Apr 2024 12:19:25 GMT
D (15305) HTTP_CLIENT: HEADER=Server:AmazonS3
D (15305) HTTP_CLIENT: http_on_headers_complete, status=403, offset=278, nread=278
D (15315) HTTP_CLIENT: content_length = -1
E (15315) esp_https_ota: File not found(403)
E (15325) esp_https_ota: Failed to establish HTTP connection
E (15335) advanced_https_ota_example: ESP HTTPS OTA Begin failed

if any other alternate way to perform OTA action please let me know

Thanks & Regards
Rahul B.

[chegewara]

https://www.google.com/search?q=File+no ... e&ie=UTF-8

[rahulbari]

thank you so much for your reply.
now after debuging code I have got this error message,

I (6409) ota: Starting OTA
W (6410) esp_https_ota: Continuing with insecure option because CONFIG_OTA_ALLOW_HTTP is set.
E (6414) esp-tls: couldn't get hostname for :c6poednjo7.execute-api.ap-south-1.amazonaws.com: getaddrinfo() returns 202, addrinfo=0x0
E (6424) esp-tls: Failed to open new connection
E (6429) TRANSPORT_BASE: Failed to open a new connection
E (6436) HTTP_CLIENT: Connection failed, sock < 0
E (6441) esp_https_ota: Failed to open HTTP connection: ESP_ERR_HTTP_CONNECT
E (6448) esp_https_ota: Failed to establish HTTP connection
E (6455) ota: ESP HTTPS OTA Begin failed ESP_ERR_HTTP_CONNECT

is that any one knows about this issue please let me know
Thanks & Regards
Rahul B.

[liaifat85]

Did you ever try to modify your code like this?

Code: Select all

esp_http_client_config_t config = {
    .url = CONFIG_EXAMPLE_FIRMWARE_UPGRADE_URL,
    .cert_pem = "/path/to/s3_certificate.pem", // Path to the PEM certificate file
    .timeout_ms = CONFIG_EXAMPLE_OTA_RECV_TIMEOUT,
    .keep_alive_enable = true,
};

// Ensure server verification is enabled
esp_tls_cfg_t tls_cfg = {
    .cacert_buf = s3_server_certificate, // Provide the S3 server's certificate here
    .cacert_bytes = sizeof(s3_server_certificate),
    // Other fields such as client_cert_buf, client_cert_bytes, client_key_buf, client_key_bytes if required
};

// Set the TLS configuration
config.tls_cfg = &tls_cfg;

// Create and configure the HTTPS client with the provided configuration
esp_http_client_handle_t client = esp_http_client_init(&config);

[chegewara]

thank you so much for your reply.
now after debuging code I have got this error message,

I (6409) ota: Starting OTA
W (6410) esp_https_ota: Continuing with insecure option because CONFIG_OTA_ALLOW_HTTP is set.
E (6414) esp-tls: couldn't get hostname for :c6poednjo7.execute-api.ap-south-1.amazonaws.com: getaddrinfo() returns 202, addrinfo=0x0
E (6424) esp-tls: Failed to open new connection
E (6429) TRANSPORT_BASE: Failed to open a new connection
E (6436) HTTP_CLIENT: Connection failed, sock < 0
E (6441) esp_https_ota: Failed to open HTTP connection: ESP_ERR_HTTP_CONNECT
E (6448) esp_https_ota: Failed to establish HTTP connection
E (6455) ota: ESP HTTPS OTA Begin failed ESP_ERR_HTTP_CONNECT

is that any one knows about this issue please let me know
Thanks & Regards
Rahul B.

I dont see issue here. I believe you have misconfiguration, since

Code: Select all

couldn't get hostname for :

As for previous "issue", you just have to configure AWS-S3 bucket properly, so the file can be downloaded.

[rahulbari]

"
I dont see issue here. I believe you have misconfiguration, since
Code: Select all
couldn't get hostname for :
As for previous "issue", you just have to configure AWS-S3 bucket properly, so the file can be downloaded.
"
no it is not misconfiguration in aws s3 bucket because I have tested that api with thunder client and its showing status 200 OK means
api is working fine, so is that any other possible reason ?

[rahulbari]

Did you ever try to modify your code like this?

Code: Select all

esp_http_client_config_t config = {
    .url = CONFIG_EXAMPLE_FIRMWARE_UPGRADE_URL,
    .cert_pem = "/path/to/s3_certificate.pem", // Path to the PEM certificate file
    .timeout_ms = CONFIG_EXAMPLE_OTA_RECV_TIMEOUT,
    .keep_alive_enable = true,
};

// Ensure server verification is enabled
esp_tls_cfg_t tls_cfg = {
    .cacert_buf = s3_server_certificate, // Provide the S3 server's certificate here
    .cacert_bytes = sizeof(s3_server_certificate),
    // Other fields such as client_cert_buf, client_cert_bytes, client_key_buf, client_key_bytes if required
};

// Set the TLS configuration
config.tls_cfg = &tls_cfg;

// Create and configure the HTTPS client with the provided configuration
esp_http_client_handle_t client = esp_http_client_init(&config);

@liaifat85
first of all, Thank you for your reply, I appreciate your effort.
genrally iam using

.url = CONFIG_EXAMPLE_FIRMWARE_UPGRADE_URL, // here I have create GET api for fetch the code with bucket name and .bin file
.cert_pem = NULL, // becaused i dont used any server cirtificate here.

so i will try this code sniffet as you provide and let you know, Thanks alot

Thanks & regards,
Rahul B.

[rahulbari]

thank you so much for your reply.
now after debuging code I have got this error message,

I (6409) ota: Starting OTA
W (6410) esp_https_ota: Continuing with insecure option because CONFIG_OTA_ALLOW_HTTP is set.
E (6414) esp-tls: couldn't get hostname for :c6poednjo7.execute-api.ap-south-1.amazonaws.com: getaddrinfo() returns 202, addrinfo=0x0
E (6424) esp-tls: Failed to open new connection
E (6429) TRANSPORT_BASE: Failed to open a new connection
E (6436) HTTP_CLIENT: Connection failed, sock < 0
E (6441) esp_https_ota: Failed to open HTTP connection: ESP_ERR_HTTP_CONNECT
E (6448) esp_https_ota: Failed to establish HTTP connection
E (6455) ota: ESP HTTPS OTA Begin failed ESP_ERR_HTTP_CONNECT

is that any one knows about this issue please let me know
Thanks & Regards
Rahul B.

I dont see issue here. I believe you have misconfiguration, since

Code: Select all

couldn't get hostname for :

As for previous "issue", you just have to configure AWS-S3 bucket properly, so the file can be downloaded.

@chegewara ,
I wanted to reach out to inform you of the progress, as well as a challenge I've encountered that I'm hoping you could assist me with.

So far, I have completed the following activities:

1) Created S3 buckets with public access and enabled bucket versioning to facilitate OTA updates and in that bucket store hello_world,bin file.
2) Developed REST APIs for GET operation, to access that file.
3) I am using stadard "advanced_https_ota" for my testing below are relevent code for this

ESP_LOGI(TAG, "Starting Advanced OTA example");

esp_err_t ota_finish_err = ESP_OK;
esp_http_client_config_t config = {
.url = CONFIG_EXAMPLE_FIRMWARE_UPGRADE_URL,
.cert_pem = NULL,
.crt_bundle_attach = esp_crt_bundle_attach,
};

4) .cert_pem = NULL, because i dont want to used any cirtificate here
5) I have also try openssl rsa cirtifcate attached here but its showing error as

I (6409) ota: Starting OTA
W (6410) esp_https_ota: Continuing with insecure option because CONFIG_OTA_ALLOW_HTTP is set.
E (6414) esp-tls: couldn't get hostname for :c6poednjo7.execute-api.ap-south-1.amazonaws.com: getaddrinfo() returns 202, addrinfo=0x0
E (6424) esp-tls: Failed to open new connection
E (6429) TRANSPORT_BASE: Failed to open a new connection
E (6436) HTTP_CLIENT: Connection failed, sock < 0
E (6441) esp_https_ota: Failed to open HTTP connection: ESP_ERR_HTTP_CONNECT
E (6448) esp_https_ota: Failed to establish HTTP connection
E (6455) ota: ESP HTTPS OTA Begin failed ESP_ERR_HTTP_CONNECT

but when i am trying to do in menuconfig section select
component config ⇒ ESP- TLS ⇒
[*] Allow potentially insecure options
[*] Skip server certificate verification by default (WARNING: ONLY FOR TESTI

This will works fine.
But its insecure connection and i want secure connection with OTA .

At the moment, I don't know where to start or is this possible to do?! Do you have any ideas for this?
Please helppp !!! if any details required please let me know

Technical points:
- Tested with ESP-IDF v4.4.3
- ESP-WROOM-32E
- aws-S3
- aws-api-gateway

Thanks & Regards
Rahul B.