Flash encryption without secure boot

[user4_esp32]

Hello All--a few questions about using flash encryption without secure boot:

If I serial flash a binary with flash encryption enabled (i.e. "in make menuconfig, navigate to “Security Features” and select “Yes” for “Enable flash encryption on boot”."), and then write-protect the FLASH_CRYPT_CNT efuse via:

Code: Select all

espefuse.py --port PORT write_protect_efuse FLASH_CRYPT_CNT

(https://docs.espressif.com/projects/esp ... encryption)

1. Is it then impossible to disable flash encryption per the steps at https://docs.espressif.com/projects/esp ... encryption?

2. Will I still have three more serial flashes available? (https://docs.espressif.com/projects/esp ... l-flashing)

3. Assuming a partition setup of factory [plaintext], ota_0 [empty], and ota_1 [empty], if I OTA a binary built with flash encryption enabled to the ota_0 partition, and then set the device to boot from the ota_0 partition, will the device be able to execute the binary at ota_0?

[ESP_Angus]

Hi user4,

1. Is it then impossible to disable flash encryption per the steps at https://docs.espressif.com/projects/esp ... encryption?

Yes, if FLASH_CRYPT_CNT is write protected then this is impossible.

2. Will I still have three more serial flashes available? (https://docs.espressif.com/projects/esp ... l-flashing)

If FLASH_CRYPT_CNT is write protected then no more serial flashes with plaintext are possible (if you keep a copy of the flash encryption key, you can still flash pre-encrypted images).

The reason for the recommendation to write-protected FLASH_CRYPT_CNT is that, without secure boot enabled, an attacker can disable flash encryption and then flash their own small bootloader which re-enables flash encryption itself and then dumps out the real firmware.

If you're still in development phase and you don't need the device to be secure than you can keep FLASH_CRYPT_CNT writable until you need to deploy.

3. Assuming a partition setup of factory [plaintext], ota_0 [empty], and ota_1 [empty], if I OTA a binary built with flash encryption enabled to the ota_0 partition, and then set the device to boot from the ota_0 partition, will the device be able to execute the binary at ota_0?

When you say "OTA a binary built with flash encryption enabled", you mean an app built with flash encryption enabled in the project configuration - yes? This answer is given on that assumption.

The bootloader also needs to also have flash encryption enabled in the project configuration. Initially, it's flashed in plaintext as well (same as the factory app partition).

Then, on first boot the device encrypts itself (bootloader, partition table, app) and enables flash encryption by burning the FLASH_CRYPT_CNT efuse. Now the factory app partition is encrypted and the device will boot from this after the next reset.

If the factory partition OTA updates with a new app binary then it will transparently encrypt that binary as it's written to flash in the ota_0 partition. This makes it eligible to boot from, if the OTA API is used to change the boot partition.

[user4_esp32]

Thanks very much, ESP_Angus! To follow-on:

1.

If FLASH_CRYPT_CNT is write protected then no more serial flashes with plaintext are possible (if you keep a copy of the flash encryption key, you can still flash pre-encrypted images)

This only works if I pre-generate the flash encryption key (https://docs.espressif.com/projects/esp ... yption-key), right?

2.

When you say "OTA a binary built with flash encryption enabled", you mean an app built with flash encryption enabled in the project configuration - yes? This answer is given on that assumption.

Yes to building an app with flash encryption enabled in the project configuration. My question stems from this situation: if I have a remote unit already deployed with a partition setup of:

• bootloader [ plaintext]

• factory [ plaintext]

• ota_0 [empty]

• ota_1 [empty]

Is it possible to OTA that unit in someway so that it will be "flash encrypted" in the future? I gather from your response that I would need to OTA a bootloader that was built with flash encryption enabled in the project configuration.

3. For production devices deployed with a partition setup of:

• bootloader [ flash encryption enabled]

• factory [ flash encryption enabled]

• ota_0 [empty]

• ota_1 [empty]

When I build the binaries to be flashed via OTA into ota_0 or ota_1, do I need to build with flash encryption enabled in the sdkconfig, or can the binaries be plaintext (and will be encrypted by the bootloader on first boot into ota_0 or ota_1)?

[user4_esp32]

Hello,

Could someone please answer #3 from above? Perhaps @ESP_Angus?

3. For production devices deployed with a partition setup of:
bootloader [ flash encryption enabled]
factory [ flash encryption enabled]
ota_0 [empty]
ota_1 [empty]
When I build the binaries to be flashed via OTA into ota_0 or ota_1, do I need to build with flash encryption enabled in the sdkconfig, or can the binaries be plaintext (and will be encrypted by the bootloader on first boot into ota_0 or ota_1)?

[ESP_Angus]

Hi,

Sorry, I somehow missed this follow-up questions the first time around.

Thanks very much, ESP_Angus! To follow-on:

1.

If FLASH_CRYPT_CNT is write protected then no more serial flashes with plaintext are possible (if you keep a copy of the flash encryption key, you can still flash pre-encrypted images)

This only works if I pre-generate the flash encryption key (https://docs.espressif.com/projects/esp ... yption-key), right?

Yes, without modifying the bootloader the only way to have a copy of the key is to have pre-generated and burned it first.

If you modify the bootloader code (by making a copy of the bootloader component in your project) then you could do something else if you wanted, like generate the key and also output a copy on the serial console or in some other place.

2.

When you say "OTA a binary built with flash encryption enabled", you mean an app built with flash encryption enabled in the project configuration - yes? This answer is given on that assumption.

Yes to building an app with flash encryption enabled in the project configuration. My question stems from this situation: if I have a remote unit already deployed with a partition setup of:

• bootloader [ plaintext]

• factory [ plaintext]

• ota_0 [empty]

• ota_1 [empty]

Is it possible to OTA that unit in someway so that it will be "flash encrypted" in the future? I gather from your response that I would need to OTA a bootloader that was built with flash encryption enabled in the project configuration.

We don't have any supported workflow for this. The difficulty is that enabling flash encryption requires encrypting all partitions in-place first, and a failure during this process may require a serial reflash (due to the flash containing a mixture of ciphertext and plaintext).

It's assumed that on a first boot in a factory you can provide some reliable power source for this to happen (and you can also easily reflash a device from serial if something does go wrong). In an OTA situation where the device might be deployed in the field, if the initial encryption pass is interrupted in any way then there is no other recovery path.

For the same basic reason we don't support OTA updating the bootloader (if rewriting the bootloader is interrupted due to a power failure or a crash, there's no recovery mechanism apart from serial reflashing.)

If you know for sure that your devices will have stable power and a stable update environment, you could:

1. OTA update an app with flash encryption support enabled in sdkconfig (as there are some changes on the app side to support flash encryption). This OTA update happens 100% in plaintext.
2. Reset to running the new app.
3. A new, flash encryption enabled, bootloader image can be embedded in the updated app as binary data. Have the app erase the bootloader region in flash and write out with the new bootloader. A power failure or crash at this point will require serial reflashing. Suggest doing this with minimal other code running, ie have the first check in app_main() be whether the bootloader is the old bootloader, with flash encryption disabled, and if these two conditions are true then it immediately writes the new bootloader without starting any other tasks which might interfere with the update process.
4. Reset again, and the new bootloader runs and will follow the "first boot" process to encrypt all the partitions and enable flash encryption. A power failure at this point will require serial reflashing.
5. Reset again and the same app will boot as step (2), but encrypted this time.

This is not supported or recommended though, for the reasons mentioned.

3. For production devices deployed with a partition setup of:

• bootloader [ flash encryption enabled]

• factory [ flash encryption enabled]

• ota_0 [empty]

• ota_1 [empty]

When I build the binaries to be flashed via OTA into ota_0 or ota_1, do I need to build with flash encryption enabled in the sdkconfig, or can the binaries be plaintext (and will be encrypted by the bootloader on first boot into ota_0 or ota_1)?

Flash encryption should be enabled in the sdkconfig of both the app and the bootloader, if you want to use flash encryption. In the case of the app, it makes sure that support functions to work with encrypted flash are compiled in.

The app .bin files, both for initial serial flashing and for OTA should be plaintext. The factory app will be encrypted in-place by the bootloader on first boot. Any OTA updated binaries will be encrypted by the device when it writes them to flash.

(This raises the question of protecting the .bin files when transferring them from the server. The recommended way to do this is to apply some other encryption scheme to the files over the network, perhaps by using HTTPS and having the server authenticate the device when it requests the .bin file. We're planning a standard solution for this problem but we don't have an ETA for it to be available, yet.)

[user4_esp32]

Thanks very much for the thorough reply, @ESP_Angus, and for clarifying.

It would be helpful to add a note to the Docs to mention that for binaries meant for OTA to a device with encryption-enabled-in-the-bootloader, one should build the binary with flash encryption support enabled in sdkconfig--perhaps here: https://docs.espressif.com/projects/esp ... ta-updates

[FiwiDev]

My apologies in advance if this is posted in the wrong forum--but searching the forum is extremely difficult for me. After about two searches (which can take 10 seconds to return), the ESP32 website starts returning a completely blank page. Not sure what's going on--I'm usually better off searching Google and clicking on the results there.

Anyway, I have figured out how to use the Arduino IDE (please don't lose me right there ) to flash an ESP32 with an encrypted bootloader. No problems there, it works quite well, and my OTA stuff is working fine. This is encryption WITHOUT secure boot, same as the thread here.
I am using the "min_spiffs" partition scheme on Arduino, which is as follows:

Code: Select all

Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x1E0000,
app1, app, ota_1, 0x1F0000,0x1E0000,
spiffs, data, spiffs, 0x3D0000,0x30000,

The issue I ran across is that in the initial boot (after uploading the sketch with a replaced bootloader with "encrypt on boot" enabled, Release mode, etc.), after successfully encrypting everything else, the bootloader threw a few errors regarding the 2nd OTA partition, which is empty. Here's the end of the output:

[0;31mE (24725) esp_image: image at 0x1f0000 has invalid magic byte[0m
[0;33mW (24725) esp_image: image at 0x1f0000 has invalid SPI mode 255[0m
[0;33mW (24727) esp_image: image at 0x1f0000 has invalid SPI size 15[0m
[0;33mW (24733) flash_encrypt: Not disabling FLASH_CRYPT_CNT - plaintext flashing is still possible[0m
[0;32mI (24754) flash_encrypt: Flash encryption completed[0m
[0;32mI (24754) boot: Resetting with flash encryption enabled...[0m

I will note again that the "ota_1" partition is empty, thus the 255 and 15 numbers make sense--but I'm at a loss as to why the bootloader is complaining about that. Or if there's anything I can do about it. I will also note that OTA works perfectly fine with and without encryption enabled. I'm not having any problems at all, just this little "bump."

This is not a terminal issue for me by any means, as I can easily write some code for the ESP32 to blow the FLASH_CRYPT_CNT protection fuse if it's running encrypted. I'm just curious as to what went on before I FLASH a dozen boards for production.
The ESP examples for encryption use a "factory" partition and no OTA partitions; I have not found an example given where there were blank "app" partitions.

@ESP_Angus...any suggestions as to what I might have gotten wrong? I'd provide the complete bootloader output...except that I'd wear my fingers down to the 3rd knuckle typing it all out (Arduino IDE bug preventing me from copying the output in the monitor, so I have to type it down from a photo I took. Archaic, I know!)
If anyone would like, you can read my GitHub comment here: https://github.com/espressif/arduino-es ... -592134519

[ESP_Angus]

Hi FiwiDev,

Impressed you got this working. To be clear, we don't support using these features in Arduino. But I'll do my best to try and answer your question.

The issue I ran across is that in the initial boot (after uploading the sketch with a replaced bootloader with "encrypt on boot" enabled, Release mode, etc.), after successfully encrypting everything else, the bootloader threw a few errors regarding the 2nd OTA partition, which is empty. Here's the end of the output:

[0;31mE (24725) esp_image: image at 0x1f0000 has invalid magic byte[0m
[0;33mW (24725) esp_image: image at 0x1f0000 has invalid SPI mode 255[0m
[0;33mW (24727) esp_image: image at 0x1f0000 has invalid SPI size 15[0m
[0;33mW (24733) flash_encrypt: Not disabling FLASH_CRYPT_CNT - plaintext flashing is still possible[0m
[0;32mI (24754) flash_encrypt: Flash encryption completed[0m
[0;32mI (24754) boot: Resetting with flash encryption enabled...[0m

I will note again that the "ota_1" partition is empty, thus the 255 and 15 numbers make sense--but I'm at a loss as to why the bootloader is complaining about that. Or if there's anything I can do about it. I will also note that OTA works perfectly fine with and without encryption enabled. I'm not having any problems at all, just this little "bump."

Guessing from context, this is the part where the bootloader is trying to encrypt all the app partitions on first boot. It does this by looking for a valid app in each partition slot.

It probably shouldn't be logging an error here, but it's harmless - it's just saying "there isn't an app here, so I'm not encrypting it". The fact it logs here may be because there's no factory partition, I'm not sure - what IDF bootloader version are you using here?

This is not a terminal issue for me by any means, as I can easily write some code for the ESP32 to blow the FLASH_CRYPT_CNT protection fuse if it's running encrypted.

The lines about " flash_encrypt: Not disabling FLASH_CRYPT_CNT - plaintext flashing is still possible" are not related to the lines above.

They're logged because this config item is set to "N" by default if Secure Boot is off. You can set it and this value will be write protected on first boot:
https://docs.espressif.com/projects/esp ... -plaintext

[FiwiDev]

The fact it logs here may be because there's no factory partition, I'm not sure - what IDF bootloader version are you using here?

Well, that's just the problem. After spending multiple hours beating my head against the wall with ESP-IDF this morning, here's the answer: I have absolutely no idea. Unfortunately, I didn't write down the steps I took to get where I was...and after struggling with it all morning, I have no idea how to get back there again (terminal history doesn't help, either). I have the entire "esp-idf" folder that I used to generate the bootloader (with the bootloader in it!), but has GIT decided that it isn't a valid repository--and because of that, ESP-IDF will not initialize, so I can't get a version out of it that way, either.

And yes, I did forget to check the "disable plaintext flashing on boot" option in the initial bootloader. So close to perfection...thank you for pointing that out.

So I've tried going back to the beginning with a fresh IDF install:

Code: Select all

git clone -b v3.3.1 --recursive https://github.com/espressif/esp-idf.git

SORT of works. Because it's larger than whatever version I initially used, I have to turn off bootloader logging in order to get it to fit in the 0x7000 partition. While it threw the same partition errors--and also complained about "invalid CPU version"--it did lock FLASH_CRYPT_CNT. According to FLASH_CRYPT_CNT, it is running encrypted...but it boots with a startup error:

ets Jun 8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371
ets Jun 8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:5232
ho 0 tail 12 room 4
load:0x40078000,len:12792
ho 0 tail 12 room 4
load:0x40080400,len:5960
entry 0x400806a4

...though my program starts and runs just fine. (The lack of any encryption messages is because I had to disable bootloader output in order to get it to fit.) This may be a residual error from the three different bootloaders I've had on this ESP, with one accidentally interrupted and two complete encryptions...as I got this exact boot sequence when reverting back to the Arduino bootloader after interrupting the first encryption cycle.

I also tried "release/v3.3", which has been updated only a couple of days ago, far more recently than v3.1.1. To my great surprise, the entire "Security Options" menu item is missing. Wha??!

One other thing I tried was to initialize v3.1.1, and then use it to try to "menuconfig" the original project...but that too fails.

@ESP_Angus: Is there ANY way to figure out what version of ESP_IDF is in the GIT-cloned repository that GIT now disowns?

P.S. I understand that cross-posting is greatly frowned on; however, I intended it to be two separate sides of the same subject. On GitHub I was trying to show how it could be done...but here, asking for clarity on some issues.

[FiwiDev]

So I can report back that I have successfully programmed and encrypted 18 ESP32 project boards using Arduino IDE 1.8.11 with a custom bootloader.

Basically, I GIT'ted ESP-IDF v3.3.1, configured the bootloader as desired (encrypt on start, disable plaintext flashing, release mode, QIO, 80MHz, 4MB), though I did have to set the bootloader output to "Error" in order to get the bootloader to fit in the default Arduino partition. Obviously, this results in no feedback during encryption, but it does not affect the functionality in any way.

After replacing the corresponding Arduino IDE bootloader file with the new one, I just upload the sketch to the ESP32, cycle power, and wait for my application to initialize an LCD screen and run. No issues.

Encryption boot results in the following output:

Code: Select all

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:5232
ho 0 tail 12 room 4
load:0x40078000,len:12792
ho 0 tail 12 room 4
load:0x40080400,len:5960
entry 0x400806a4
[0;31mE (27445) esp_image: image at 0x1f0000 has invalid magic byte[0m
[0;31mE (27445) boot_comm: mismatch chip ID, expected 0, found 18770[0m
ets Jun  8 2016 00:22:57

rst:0x3 (SW_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:5232
ho 0 tail 12 room 4
load:0x40078000,len:12792
ho 0 tail 12 room 4
load:0x40080400,len:5960
entry 0x400806a4

And it runs encrypted, with the FLASH_CRYPT_CNT write disable bit set.
I still can't figure out for the life of me what version of ESP-IDF I used that was able to fit the bootloader into the 0x7000 partition without disabling bootloader output. If it means anything, it has the shell scripts ("install.sh"), which are not present in 3.2.3 or 3.2.1 (both of which I've tried, but the resulting bootloaders there are still too big).
It might be v3.3...but I thought I tried that, and couldn't find the Security settings. Who knows, I dunno what I'm doing...