Write RAM using bootloader serial protocol

[JayLogue]

I'm looking for a way to write a chunk of data from a host machine directly into RAM on an ESP32 module and then have the system reboot such that the data in RAM remains intact and is readable by an application.

The backstory here is that I'm building a system for provisioning information onto a device at manufacturing time. I have an existing python-based tool that uses this approach for another SoC and I’d like to adapt it to the ESP32.

One obvious way to achieve this would be to use JTAG. In the case of the ESP32, however, I wonder if the ESP32 bootloader serial protocol could be used instead. From reading the wiki, I see the MEM_BEGIN, MEM_DATA and MEM_END commands, which seem appropriate to the task. However there’s little in the way of detailed documentation outside of the esptool code.

I'm sure I can reverse engineer the code. But does it seem reasonable to use the MEM_* commands for this purpose? If so, must I reimplement the protocol myself, or is there an implementation outside of esptool that I can reuse? Perhaps I can just import esptool as a module and reuse the classes it contains; has anyone done this?

[WiFive]

It's probably doable with some mods to esptool but you have to choose a ram area that the app is not going to use or init. Why not have the app check if it has been provisioned and if not, reopen the serial connection to get the provisioning data from the programmer? Or do it via WiFi so you have a built in self test for wifi.

[JayLogue]

Thanks for your reply.

Writing a custom serial protocol would be possible, but makes the on-device code much more ESP32-specific. Given that none of the other supported SoCs require a custom protocol, I'd like to avoid that. If that ends up being my only option I will likely opt for a solution using JTAG instead.

WiFi isn't an option for security reasons. Without some sort of per-device key already installed on the device, there's no way to establish a private/authenticated channel between the provisioning machine and the device (well, short of using an isolation chamber, which is logistically difficult).

I have begun investigating whether the code in the esptool can be reused or extended.

[WiFive]

Well the additional on device code could just be (open uart, write data to memory location). The rest of the code could be the same.

So the attackers will be able to sniff WiFi at the factory and crack wpa2 psk or eap but not have the ability to compromise the provisioning machine?

With esptool by default it will tell the chip to execute what you just copied to ram so you have to change that and also make sure the ram you select is unused and preserved until your app loads.

[JayLogue]

Well the additional on device code could just be (open uart, write data to memory location). The rest of the code could be the same.

Certainly worth considering, although if I can avoid having the firmware operate modally like this it would be logistically more convenient.

So the attackers will be able to sniff WiFi at the factory and crack wpa2 psk or eap but not have the ability to compromise the provisioning machine?

The machines doing the provisioning are physically and network isolated from the rest of the factory. Since one of the purposes of the provisioning step is to place initial secrets on the device, by definition there is no per-device PSK available to secure a WiFi connection. Using a PSK hardcoded in the firmware image isn't acceptable since it is notoriously difficult to regulate who gets access to firmware.

I'm now looking at the esptool's load_ram command, which if I format my data as a firmware image, appears to allow me to place a chunk of data anywhere in DRAM without needing to alter the esptool or create my own version of the code. The image format seems straightforward, but I need to consider what to do with the load address (the address the bootloader jumps to after loading the image).

[WiFive]

So how do you reboot into your app while preserving the memory? Esptool would normally do a hard reset. Put a call to a rom restart function at the beginning of the binary and let it execute? Use rtc ram and mark it as noinit in your app?

[JayLogue]

So how do you reboot into your app while preserving the memory? Esptool would normally do a hard reset. Put a call to a rom restart function at the beginning of the binary and let it execute? Use rtc ram and mark it as noinit in your app?

This is the crux of the issue. I've set the entry point for my RAM image to the software_reset function in the ESP32 ROM (0x4000824c). At first this appeared to do nothing. But then I modified esptool to immediately dump the serial port to stdout after invoking the load_ram operation. This revealed that the system was going straight into flashing mode. E.g.:

Code: Select all

ets Jun  8 2016 00:22:57

rst:0x3 (SW_RESET),boot:0x3 (DOWNLOAD_BOOT(UART0/UART1/SDIO_REI_REO_V2))
waiting for download

I tried pulling GPIO0 high by setting the serial port DTR to low, but that didn't change the behavior.

My guess is that the stapping pings are sampled once on hard-boot and then never again. This would seem to imply that there's no way to use the ROM software_reset function to provoke a normal boot after booting in flashing mode. Since I don't have the source for software_reset I can't see if there are other means of controlling its behavior.

Guidance from Espressif folks would be valuable here.

[ESP_igrr]

No, indeed the strapping pins are sampled only power-on reset and RTC watchdog reset.

Sorry if I missed this in your previous posts, but why do you require the information to be specifically in RAM? The secrets can be saved on the host computer into an NVS partition binary. This binary can be flashed by esptool to the device. Upon startup, the device can read data from the NVS.

[JayLogue]

The project I’m working on is in support of Nest’s OpenWeave application framework (main code here, esp32 example here). We are developing a tool for factory and developer provisioning of devices that works across multiple platforms. A common strategy used by the tool is to inject a bundle of data into RAM, which the device firmware subsequently locates on boot, validates, and then stores in the appropriate way for the particular platform. This approach leverages common code in both the host and the device. It also makes the manufacturing process more consistent across products, which simplifies things for manufacturing partners.

So I’m looking to see if I can apply this same approach to the ESP32. My reading of the code indicates that RAM contents can be retained across a soft reboot provided the RAM location is chosen appropriately (e.g. the high end of a heap or the far top of a stack). So at this point it seems doable using the load_ram command with the caveat of how to initiate the soft reset.

If there was a way to convince software_reset ROM function to perform a flash boot that would be ideal. Alternatively I could place a small amount of code in the RAM app that directly orchestrates the soft reset. Since I don’t have the source for software_reset, I’m unclear on exactly what that code would do.

Is there another example of a low-level soft reset that I can look at?

Thanks for your help.

[WiFive]

inject a bundle of data into RAM

this has to be platform specific on the tool side

stores in the appropriate way for the particular platform

this has to be platform specific on the device side

So is it really just sleight of hand to force the process to appear consistent across platforms?