HTTP Get file on amazon "S3"

[Arnaud_Girard]

Hello,

I'm using ESP32-C3 with ESP-AT firmware (v2.4...)

Another part has created account on Amazon. Then, We generate some device with certificate.
I convert these certificate (ca, cert and key) with the provided python script AtPKI.py.

Then I flash these files to the ESP32Flash on partition mqtt_ca, mqtt_key and mqtt_cert with the command AT+SYSFLASH.

Then I'm able to connect to the amazon server on MQTT with commands:
- AT+CIPSNTPCFG (set time)
- AT+MQTTUSERCFG
- AT + MQTTCONN..

After that I can publish message and subscribe to a topic.
This work properly (Tested with the MQTT client test on Amazon web interface)


Now I want be able to get a file.
To do that, I upload my file on "amazon S3" module.
This module allow me to upload a file and generate an url (for some couple of hour).
The generated URL is like this:

https://<mysite>-sandbox-ota.s3.eu-west-1.amazonaws.com/<myrep>/<myfile>.xxx?response-content-disposition=inline&X-Amz-Security-Token=IQoJb3JpZ2luX2V[....]Amz-Date=20221026T151421Z&X-Amz-SignedHeaders=host&X-Amz-Expires=43200&X-Amz-Credential=ASIAWXYV5C2B6FEAXLEB%2F2022[....]

If I copy/paste this URL into my web browser, the file is downloaded. Then the link is good.

Now I want download this file with the ESP32.
Like the URL is > 256 Byte, I write the follwoing command:
- AT+HTTPURLCFG=1668
- >
- <myURL>
- AT+HTTPURLCFG? -> return my URL.

Follwing is my terminal view with different try:
17:42:13.407 [TX] - AT+HTTPGETSIZE=""
17:42:13.422 [RX] - AT+HTTPGETSIZE=""
17:42:13.998 [RX] -
ERROR

17:45:13.470 [RX] - AT+HTTPGETSIZE="https://<mysite>-ota.s3.eu-west-1.amazonaws.com/<myrep>/<myfile>.xxx"
17:45:14.046 [RX] - +HTTPGETSIZE:0
OK


17:42:40.587 [TX] - AT+HTTPCLIENT=2,0,"","https://<mysite>-sandbox-ota.s3.eu-west-1.amazonaws.com","/<myrep>",2
17:42:40.600 [RX] - AT+HTTPCLIENT=2,0,"","https://<mysite>-sandbox-ota.s3.eu-west-1.amazonaws.com","/<myrep>",2
17:42:41.228 [RX] - ERR CODE:0x010a7190
ERROR

17:43:24.569 [TX] - AT+HTTPCLIENT=2,0,"",,,2
17:43:24.585 [RX] - AT+HTTPCLIENT=2,0,"",,,2
17:43:25.128 [RX] - ERR CODE:0x010a7000
ERROR


Now if I create a local HTTP server on my computer, and just use AT+HTTPURLCFG=xx then My local url.
Then I get the size and then the file with following commands:
AT+HTTPCLIENT=2,0,"",,,2
AT+HTTPGETSIZE=""

All work properly.

In other word, I get error only with amazon file..

I think there are certificate problem?? But the url contains the key...
Where is my mistake?
Are they exemple to get HTTP file on amazon s3 server?

Regards

[esp-at]

i don't think it's a certificate issue, but a buffer size issue due to long url.

could you please compile esp-at firmware with latest master branch, and change DEFAULT_HTTP_BUF_SIZE to 4096 in esp-idf/components/esp_http_client/include/esp_http_client.h, and see if this issue still exists.

~

[esp-at]

Could you please use AT+HTTPCGET command instead.
like:
AT+HTTPURLCFG=<length>
AT+HTTPCGET="",<4096>,<2048>

[Arnaud_Girard]

Hello

Thanl for your help.

The command AT+HTTPCGET work properly.

Regards

[Arnaud_Girard]

Hello

I missed to check again the command AT+HTTPGETSIZE=""

So to get the file, I used the command AT+HTTPCGET, but before that, I would get the file size to erase the amount of neessary flash before receive the data flow.

Then I call the command
AT+HTTPCGET
But return "ERROR"

[esp-at]

AT+HTTPGETSIZE returns ERROR, because of long url. you can change DEFAULT_HTTP_BUF_SIZE to 4096 in esp-idf/components/esp_http_client/include/esp_http_client.h.

i would like to add a tx parameter for AT+HTTPGETSIZE to support long url.
~

[Arnaud_Girard]

Hello

I havn't install all IDE to compile the source code.
I get directly the oficial release of ESP-AT, And I hope to keep this because is simplier for further update to only need to use the command AT+CIUPDATE...

Do you think is possible to add this option to make GETSIZE compatible with long URL?

[esp-at]

plase test AT+HTTPGETSIZE=<"url">[,<tx_size>][,<rx_size>][,<timeout_ms>] feature with the firmware in the attachment.

such as AT+HTTPGETSIZE="https://blabla",2048,2048,5000

This feature will sync to GitHub within next two weeks.
~

[Arnaud_Girard]

Ok,

THese two follocing command run correctly:

19:53:11.919 [TX] - AT+HTTPGETSIZE="",2048,2048,5000

19:53:11.927 [RX] - AT+HTTPGETSIZE="",2048,2048,5000

19:53:12.790 [RX] - +HTTPGETSIZE:28298

OK

19:53:19.230 [TX] - AT+HTTPGETSIZE=""

19:53:19.233 [RX] - AT+HTTPGETSIZE=""

19:53:20.480 [RX] - +HTTPGETSIZE:28298

OK

So I deduce you increase the default buffer size?

You means the release v2.5.0.0 which include this new functionality will be released within 2 weeks?


Thank for this additional function.

[esp-at]

please download latest esp-at firmware from GitHub Actions: https://github.com/espressif/esp-at/act ... 3368936467

See https://docs.espressif.com/projects/esp ... ithub.html for more details.