Replacing encrypted app, causes "No MD5" partition error: why?!

[ESP_Mahavir]

Hello,

For ESP-IDF, default config options are set through Kconfig files. Please see https://github.com/espressif/esp-idf/bl ... #L123-L124 as pointer for partition table MD5 default setting (set as yes).

ESP-IDF v4.4 uses esptool from its v3 branch (https://github.com/espressif/esptool/tree/release/v3). We ensure that all fixes are available in esptool v3 branch too. Latest esptool (v4.x) shall be available in IDF v5.0 release, through PyPI package manager. We recommend using the tool that is bundled with the corresponding release (either ESP-IDF or Arduino).

Hope this helps.

[HighVoltage]

Since I have been struggling getting the build-controlled encryption system to cooperate with my Arduino-IDE-generated bin, I decided to look at building my application under IDF instead. I thought getting a non-trivial application with 3rd party library dependencies to work would require more effort. But it turned out that was the easier route: I got the build done in a couple of days, and didn't require any source changes. I'll just continue to develop in Arduino IDE, and when I want to encrypt, build under IDF.

I may revisit the arduino-based-bin workflow when I have more time, as I know others have reported it to work. But I need to get back to real work on my project for now.

[espnoob]

Sorry to hijack the thread, but there is an interesting question that I did not see an explicit answer for.

This "encryption-aware" concept really does not make any sense. You are suggesting if you have a working "plain text" app installed, and then you turn on the built-in encryption process, the working "plain text" app can stop working when it gets auto-encrypted because it may have been built in an environment that is not encryption aware?!

Can you point to where this is documented? Everything I've read implies a working plain-text app will just be encoded by AES and decoded automatically. Nothing suggests that a plain-text app that works is at risk of not working after it gets encrypted.

Is it as described? Do I have to `grep` through the components directory to discover the places where "encryption-awareness" is created?

Thanks!

[greddy]

I went through exactly the same process as you:

1) original code in Arduino ... ran into lack of encryption support
2) ported code to ESPIDF using arduino as a component
3) MD5 checksum error when trying to load Arduino firmware via OTA (on an encrypted device)
4) compiled the OTA firmware using ESPIDF
5) firmware updates work again.

that was for a "development mode" flash encrypted ESP32.

Next steps
1) to try adding in secure boot v2 in "reflashable" mode
2) switch secure boot and flash encryption to "release modes" / fully locked down

and hope not to run out of modules along the way....

[ESP_Mahavir]

This "encryption-aware" concept really does not make any sense. You are suggesting if you have a working "plain text" app installed, and then you turn on the built-in encryption process, the working "plain text" app can stop working when it gets auto-encrypted because it may have been built in an environment that is not encryption aware?!

This is not quite true. Whole encryption/decryption process has no binding to the environment where application is built and hence application shall execute in same way as it was before the flash encryption was enabled on the device.

Everything I've read implies a working plain-text app will just be encoded by AES and decoded automatically. Nothing suggests that a plain-text app that works is at risk of not working after it gets encrypted

Right, I can confirm this and its also clearly highlighted in our docs too.

[hacksta]

I went through exactly the same process as you:

I am attempting to go through encryption on an Arduino build as well - were you successful in your attempts to enable release mode encryption and secure boot?

[willgeco]

I was attempting the same thing recently and wanted to share what I found if anyone is still struggling with this. I found that the reason this error is happening is actually because of a mismatch between the bootloader and application partition table offset. When encryption is enabled, you end up needing to update the offset in sdkconfig because the bootloader increases in size. The default value is 0x8000.

Code: Select all

CONFIG_PARTITION_TABLE_OFFSET=0x8000

Once this value is changed to 0xa000 (or whatever address you set based on bootloader settings) in ESP-IDF, the bootloader and application are recompiled with this new address, and the flash tool flashes the partition table to 0xa000.

The problem with Arduino builds is that it is built on top of pre-compiled ESP-IDF binaries that were built with the default value of 0x8000. So when you flash this build to the application partition, it has the wrong address for the partition table. When the app gets loaded, there's a bunch of code that runs before your application code executes which initializes peripherals, checks eFuses, and loads the partition table (and checks that the MD5 is correct since that is enabled in the pre-compiled libraries). It loads the memory from the address 0x8000 which is not the partition table and so the MD5 obviously doesn't match and the firmware bails assuming it's corrupted.

What really threw me off was when I tried only updating the partition table address without enabling encryption and then flashed the arduino app - everything worked perfectly!! Took me a bit to figure out - when I reflashed the new bootloader and partition table, the flash tool only erases as much flash as it needs, in the case of the default bootloader it erases from 0x1000 to 0x7fff - leaving the original partition table untouched at 0x8000. On boot the bootloader finds the partition table at 0xa000 and loads fine, then the app launches and finds the originally flashed partition table at 0x8000 and since they're the same everything runs happily. Once I enabled flash encryption the larger bootloader overwrites the memory at 0x8000 and the app crashes with the original error again.

I believe the only way to get the correct partition table address set is to recompile the esp32 core binaries, which can either be done by re-compiling using ESP-IDF with Arduino as a component or recompiling using Arduino Lib Builder with the correct offset, then building with Arduino IDE on top of that new core.