Problem using a ESP32 to measure WiFi internet speed

[ssesp32]

I'm trying to program my ESP32 to monitor my home WiFi internet speed, but I am getting strange results and could use your help. (I understand that my ESP32 will be unable to measure my WiFi at speeds much over 20Mbps, but that's fine as long as I can record speeds up to around that number.)

I'm testing the time it takes to do a HTTP.GET() request and fully retrieve a large (600KB-15MB) JSON file I have hosted on Amazon S3. To get the full JSON file, I am using the HTTP Stream example from https://github.com/espressif/arduino-es ... Client.ino. I am simply reading the data into a small memory buffer, piece by piece, and doing nothing with it. When I get to the last piece of data, I stop the timer.

However, on a plenty-fast WiFi connection, I am recording a download speed of only about 0.29 Mbps. My understanding is that in real-world practical tests of the ESP32 people are seeing more like 10-30Mbps. This makes me believe I am doing something wrong. Perhaps the method I have chosen to measure the download speed is bottlenecked by the computing operations I have chosen. Perhaps I should be testing the speed using UDP? Am I currently doing it via TCP/IP? This area is not my strength.

Here's the snippet of relevant code:

Code: Select all

        
        HTTPClient http;
        
        http.begin("http://aws-example.s3.com/file.json");

        int httpCode = http.GET();
        
        if(httpCode > 0) {
            if(httpCode == HTTP_CODE_OK) {
            	
                int remainingFileSize = http.getSize();

                // create buffer for read
                uint8_t buff[1460] = { 0 };

                // get tcp stream
                WiFiClient * stream = http.getStreamPtr();

                // read all data from server
                while(http.connected() && (remainingFileSize > 0 || remainingFileSize == -1)) {
                    // get available data size
                    size_t size = stream->available();

		    // start timer
          	    unsigned long startMillis = millis();

		    // iteratively read all the data into the buffer, and do nothing with it.
                    if(size) { 
                        // read up to 1460 byte
                        int c = stream->readBytes(buff, ((size > sizeof(buff)) ? sizeof(buff) : size));

			// do nothing with the data

                        if(remainingFileSize > 0) {
                            remainingFileSize -= c;
                        }
                    }
                }

		// end timer
          	unsigned long endMillis = millis();
            }
         
            
          float download_sec = (float) (endMillis - startMillis) / 1000;
          int get_size = http.getSize();
          float payload_kbytes = (float) get_size / 1000;
          float payload_mbits = payload_kbytes*8 / 1000;
          float download_mbps = payload_mbits/download_sec;
        
          Debug.print(DBG_VERBOSE, "Payload (bytes): %gKB", payload_kbytes);
          Debug.print(DBG_VERBOSE, "Payload (bits): %gMb", payload_mbits );
          Debug.print(DBG_VERBOSE, "Payload Delivery Time: %gs", download_sec);
          Debug.print(DBG_INFO, "Download Mbps: %g", download_mbps);
       }
       

I have run this code retrieving various size JSON files, here are the results:

// SMALL FILE
09:44:27.842 -> Payload (bytes): 651.09KB
09:44:27.842 -> Payload (bits): 5.20872Mb
09:44:27.842 -> Payload Delivery Time: 17.847s
09:44:27.842 -> Download Mbps: 0.291854


// MED FILE
09:47:35.855 -> Payload (bytes): 3672.16KB
09:47:35.855 -> Payload (bits): 29.3773Mb
09:47:35.855 -> Payload Delivery Time: 104.666s
09:47:35.855 -> Download Mbps: 0.280677


// LARGE FILE
09:56:22.549 -> Payload (bytes): 14688.6KB
09:56:22.549 -> Payload (bits): 117.509Mb
09:56:22.549 -> Payload Delivery Time: 407.169s
09:56:22.549 -> Download Mbps: 0.288601


I see examples of people seeing much higher speeds on their ESP32 projects:
10-16 Mbps: https://github.com/martin-ger/esp32_nat ... erformance (homepage: https://serverkernel.com/how-to-turn-es ... -repeater/)
20-30 Mbps: https://www.reddit.com/r/esp32/comments ... le_seeing/

Any tips on what I'm doing wrong? And how I can accurately write a script to measure the WiFi speed?

[ESP_Sprite]

I don't exactly know what's going on under the hood in Arduino land, but have you tried using a larger buffer to receive data in?

Also, it's possible the other examples you saw used an ESP-IDF configuration optimized for throughput (you can set a balance between high speed and memory usage using the WiFi config options). I think the default Arduino config is more balanced for general use cases.

[bobo1974]

@ssesp32:
I have struggled with arduino download speed myself for a while.
I was just giving it another round with the latest 2.0.0 board manager version.
I get to 700kbps in average.

There is a way to increase by recompiling arduinos library but it is significant work I can't do right now. I tried it unsuccessfully already once.

Have you reached some improvements on these?

@ESP_SPRITE: obviously with a standard setup we don't aim for stars, but 200kbps is way too small. Who could be contacted to send a change request? MANY users have the same issue.

An btw I see no influence from the buffer.

My results:
GET: http://speedtest.tele2.net/1MB.zip
12:32:22.274 -> starting...200 Buff Size: 50
12:32:34.367 -> Time: 11352 Bytes: 1048576 Throughput : 736 kbps736
12:32:39.371 -> starting...200 Buff Size: 500
12:32:51.714 -> Time: 12231 Bytes: 1048576 Throughput : 680 kbps680
12:32:56.756 -> starting...200 Buff Size: 1000
12:33:12.129 -> Time: 12540 Bytes: 1048576 Throughput : 664 kbps664
12:33:17.165 -> starting...200 Buff Size: 2000
12:33:29.909 -> Time: 12658 Bytes: 1048576 Throughput : 656 kbps656

GET: https://proof.ovh.net/files/1Mb.dat
12:29:40.654 -> starting...200 Buff Size: 50
12:29:51.735 -> Time: 9957 Bytes: 1048576 Throughput : 840 kbps840
12:29:56.745 -> starting...200 Buff Size: 500
12:30:07.681 -> Time: 9869 Bytes: 1048576 Throughput : 848 kbps848
12:30:12.693 -> starting...200 Buff Size: 1000
12:30:23.984 -> Time: 10084 Bytes: 1048576 Throughput : 824 kbps824
12:30:28.964 -> starting...200 Buff Size: 2000
12:30:40.288 -> Time: 10114 Bytes: 1048576 Throughput : 824 kbps824

Code:

Code: Select all

#include <WiFi.h>
#include <HTTPClient.h>

const char* ssid = "XXXXXXXX";
const char* password =  "XXXXXXXX";

const char* root_ca = \
                      "-----BEGIN CERTIFICATE-----\n" \
               
                      "-----END CERTIFICATE-----\n";

void setup() {
  Serial.begin(115200);
  delay(1000);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Connecting to WiFi..");
  }
  Serial.println("Connected to the WiFi network");
}

long speedtest(int buf_size) {
  unsigned long start;
  long total = 0;
  if ((WiFi.status() == WL_CONNECTED)) { //Check the current connection status
    
    HTTPClient http;
    Serial.print("starting...");
    //http.begin("https://proof.ovh.net/files/1Mb.dat", root_ca); //Specify the URL and certificate
    http.begin("http://speedtest.tele2.net/1MB.zip"); //Specify the URL and certificate
    int httpCode = http.GET();                                                  //Make the request

    if (httpCode > 0) { //Check for the returning code
      Serial.print(httpCode); Serial.print("           ");
      if (httpCode == HTTP_CODE_OK) {
        // get length of document (is -1 when Server sends no Content-Length header)
        total = http.getSize();
        uint8_t buff[buf_size] = { 0 };
        int buf_size_actual = sizeof(buff);
        Serial.print("\t\t Buff Size: "); Serial.println(buf_size_actual);
        WiFiClient * stream = http.getStreamPtr();
        http.setReuse(true);
        start = millis();
        size_t size = 1;
        long remaining = total;
        while (http.connected() && (remaining > 0 || remaining == -1)) {
          if (stream->available()) {
            remaining -= stream->readBytes(buff, buf_size);
          }
        }
      }
    }
    else {
      Serial.println("Error on HTTP request");
    }

    http.end();
    Serial.print("\t\tTime: "); Serial.print(millis() - start);
    Serial.print("\t\tBytes: "); Serial.print(total);
    Serial.print("\t\tThroughput :"); Serial.print("\t"); Serial.print(total / (millis() - start) * 8); Serial.print("\tkbps");
    return total / (millis() - start) * 8;
  }
}

void loop() {
  Serial.println(speedtest(50));
  delay(5000);
  Serial.println(speedtest(500));
  delay(5000);
  Serial.println(speedtest(1000));
  delay(5000);
  Serial.println(speedtest(2000));
  delay(5000);
}