Split firmware.

[nikolo]

esp32 when you turn on the WIFI/BT includes the library file in the firmware itself. But the stack itself does not change compared to the logic of the program. IDF also has a partition system (ota/filesystem/... ). Considering the code is copied to RAM and run, i have a question is how to modify the firmware so as to separate the WIFI/BT stack from the firmware and place the library file on the partition? as a result, get a smaller firmware file.

[MicroController]

This is basically the idea of shared libraries/DLLs, which has been proposed for the ESP by others and which was determined to be (close to) impossible to realize.

A seemingly more realistic idea just popped up in my head, and that is to use some kind of LZW compression for a new firmware version, initializing the compression's dictionary with the previous version of the firmware. This would give you a pretty optimum encoding of the differences between the old and the new firmware which the ESP could then "decompress" to compute the new firmware from its present firmware and the update. The LZW encoding efficiently takes care of code pieces which did not change but got moved to a different location by the linker.

[ESP_Mahavir]

Fyi, we do have a component for the delta compressed OTA updates feature: https://github.com/espressif/idf-extra- ... _delta_ota.

[MicroController]

Nice! Didn't know that...

[nikolo]

Delta compressed OTA it's good, but it doesn't solve goal.

Example use wifi 50%, application 30%, free 20%. 2 OTA.

Code: Select all

[BOOT|NVS|#####%%%__#####%%%__] like now    
[BOOT|NVS|#####%%%__%%%_______] as it could be

left for 3 OTA partitions, or can be distributed among existing.

[phatpaul]

Wow, this looks interesting. I currently have a 1.6MB firmware OTA update, and it takes 20~40s to upload from a mobile app to ESP32 via WiFi. I wish it was faster and sometimes the Wifi connection drops in the middle of the process.

I wonder how much delta compression would help in my case. I have Wifi, Ethernet, Bluedroid BLE, libesphttpd, and an espfs image that doesn't change too much between releases.

My understanding is that much of the OTA update delay (and issues with the Wifi conn dropping) is due to the SPI flash erasing and writing (the application and network drivers are mostly blocked during flash erase/write chunks). So the bottleneck isn't the file transfer over the network in my case.

Thoughts?

[nikolo]

In your case, the acceleration will be significant, cut the time at least in half, the only critical point is that you need to have the previous firmware or the base one in order to make a patch. And this is already the need to introduce additional systems for identifying the firmware.

[MicroController]

left for 3 OTA partitions, or can be distributed among existing.

I believe I understand what you're saying. As of now, for OTA you indeed need, at least temporarily, enough flash memory for two complete copies of the firmware including all libraries.
Maybe someone experienced with LD can advise if it's possible to link static libraries in a way to reproducibly end up at the same location for every build. (But then those libraries would have to be built&linked without the space-saving optimization of discarding unused functions.)

[ESP_Mahavir]

Wow, this looks interesting. I currently have a 1.6MB firmware OTA update, and it takes 20~40s to upload from a mobile app to ESP32 via WiFi. I wish it was faster and sometimes the Wifi connection drops in the middle of the process.

There are some timing numbers in the PR here: https://github.com/espressif/idf-extra- ... 1606391908. Please check out, looks like it could save some time for your use-case.