I'm currently working on a project using ESP32-CAM to capture AVI files and upload them to an S3 bucket. Right now everything works, but the upload speed to my S3 bucket is ridiculously slow: I'm currently getting 25 kB/s!!
I tested my board using the iperf example and was able to achieve an average speed of 10 Mbit/sec (around 1.3 MB/sec), so the problem is not the board. Also, since I'm reading the video files from and SD card, I tested the read speed and got around 803 kB/sec using 512 kB chunks, so no issue there.
I'm currently adapting the esp_http_client example (https://github.com/espressif/esp-idf/tr ... ttp_client) so I can test if the bottleneck comes from my own code or the library, but I still think 25 kB/s is simply too slow for anything practical.
A code snippet so you can understand what I'm trying to do:
Code: Select all
#include "esp_log.h"
#include "nvs_flash.h"
#include <string.h>
#include <stdio.h>
#define BUFF_SIZE 524288
/* Anonymous namespace */
namespace {
const constexpr char *TAG{"CAM Initial Setup"};
EXT_RAM_BSS_ATTR static char buffer[BUFF_SIZE] = {0};
char response_buffer[CONFIG_MAX_HTTP_OUTPUT_BUFFER + 1] = {0};
}
extern "C" void app_main(void){
/************************
* Initialize NVS flash *
*************************/
esp_err_t ret = nvs_flash_init();
if (ret == ESP_ERR_NVS_NO_FREE_PAGES || ret == ESP_ERR_NVS_NEW_VERSION_FOUND) {
ESP_ERROR_CHECK(nvs_flash_erase());
ret = nvs_flash_init();
}
ESP_ERROR_CHECK(ret);
/**********
* Tests *
**********/
setvbuf(stdout, NULL, _IONBF, 0); // Don't buffer stdout
std::fstream file;
size_t filesize;
// (declaration of this functions is not shown, but they're not important for this topic)
storage.open_video_file("333565712.avi", &file); // Wrapper function for opening a file
storage.get_video_size("333565712.avi", &filesize); // Wrapper function for querying file size
// 1) Reading from SD Card
ESP_LOGI(TAG, "%d/%d = %d", filesize, BUFF_SIZE, filesize/BUFF_SIZE);
uint32_t start = (uint32_t)time(NULL);
for (int i=0; i<filesize; i+=BUFF_SIZE) {
file.read(buffer, BUFF_SIZE);
printf("%03d/%03d\r", i/BUFF_SIZE, filesize/BUFF_SIZE);
}
printf("\n");
uint32_t end = (uint32_t)time(NULL);
ESP_LOGI(TAG, "Read %d Bytes in %lus --> %lu B/s ~ %lu kB/s", filesize, end-start, filesize/(end-start), (filesize/(end-start)) >> 10);
// 2) Writing to S3 bucket (WiFi is connected elsewhere, not important for this topic i think)
file.close(); // Ensure file handle is "clean"
esp_http_client_handle_t http_client;
/*==========================================================================
Here I obtain a presigned URL. This is done by using esp_http_client_perform and
querying a custom endpoint. This is also not important for this topic, but I thought it
was worth mentioning that my esp_http_client instance is used in a GET request before
starting the upload. Assume the result is stored in a cJSON object:
==========================================================================*/
std::string s3_bucket_url = cJSON_GetObjectItem(json, "url")->valuestring;
esp_http_client_set_url(http_client, s3_bucket_url.c_str());
esp_http_client_set_method( http_client, HTTP_METHOD_PUT);
esp_http_client_set_header(http_client, "Content-Length", std::to_string(filesize).c_str());
int bytes_sent = 0;
storage.open_video_file("333565712.avi", &file);
start = (uint32_t)time(NULL);
uint32_t then = start;
uint32_t now;
esp_http_client_open(http_client, filesize);
ESP_LOGI(TAG, "filesize = %d", filesize);
for (int i=0; i<filesize; i+=BUFF_SIZE) {
file.read(buffer, BUFF_SIZE);
bytes_sent = esp_http_client_write(http_client, buffer, file.gcount());
now = (uint32_t)time(NULL);
printf("%03d/%03d\t%ld kB/s\t%lds between chunks\r",
i/BUFF_SIZE, filesize/BUFF_SIZE, (bytes_sent >> 10)/(now - then), now - then);
then = now;
}
end = (uint32_t)time(NULL);
printf("\n");
ESP_LOGI(TAG, "Sent %d Bytes in %lus --> %lu B/s ~ %lu kB/s", filesize, end-start, filesize/(end-start), (filesize/(end-start)) >> 10);
esp_http_client_fetch_headers(http_client);
ESP_LOGI(TAG, "Got status code %d", esp_http_client_get_status_code(http_client));
esp_http_client_close(http_client);
esp_http_client_cleanup(http_client);
}
Any help would be greatly appreciated, so thanks in advance
