Meditation Error - device freezing, not restarting, due to low baud rate on UART0

[warwick]

Hello,

We are working on a project in Arduino version 1.0.4 /ESP IDF 3.2.2, where we ran into an unexpected problem of the ESP32 device freezing, rather than restarting, after Guru Meditation Errors, even though Panic Mode is set to CONFIG_ESP32_PANIC_PRINT_REBOOT=y as default via the Arduino sdkconfig file.

After quite some time trying to work out why the device was not restarting, we found that it was due to the baud rate we had configured for UART0 = 4,800.

It seems the low baud rate for UART0 does not allow all the registers and back trace to be printed - perhaps the ESP32 is limiting the time for this in some way - and the result is that the ESP32 prints only part of the register data and then freezes rather than restarting.

12:49:21.827 -> Guru Meditation Error: Core 1 panic'ed (StoreProhibited). Exception was unhandled.
12:49:22.015 -> Core 1 register dump:
12:49:22.062 -> PC : 0x4000c3f5 PS : 0x00060630 A0 : 0x800f9ead A1 : 0x3ffb1eb0
12:49:22.299 -> A2 : 0x000100aa A3 : 0x3ffdff40 A4 : 0x00000019 A5 : 0x3ffb80c0
12:49:22.487 -> A6 : 0x000007b2 A7 : 0x00000208 A8 : 0x00000000 A9 : 0x000100aa
12:49:22.674 -> A10 : 0x000⸮

If baud rate for UART0 is increased to 115,000, then the register dump and back trace data is printed completely and the device restarts, so it seems this issue is definitely related to the low baud rate on UART0 for some reason.

Of course we need to fix the root cause of this particular Guru Meditation Error, but all WDT resets, Brownout resets, etc also result in the device freezing rather than restarting, which is less than ideal for production devices which will be deployed remotely in unmanned environments.

In our particular application we need two bi-directional UARTs (one at 600 baud, the other at 9,600 baud) and another UART to receive only (at 4,800 baud), so need to use all three of the available UARTs in the ESP32. Baud rates are fixed by the external devices we need to communicate with, so we are not able to change the baud rates.

Our first approach was to try to select CONFIG_ESP32_PANIC_SILENT_REBOOT=y in the hopes that this will allow the device to be restarted (assuming it is the time taken to print the debug output causing the problem), but this is compiled as part of the Arduino sdkconfig defaults and seems not easy to change without making some quite big changes to the way our project is configured - such as by using Arduino as a component, which we have no experience of doing and which does not seem that straightforward.

Another possibility we are considering is to use one soft UART for the RXD (and keep UART0 at a high baud rate just for the debug output so the device can restart in case of Guru Meditation Errors), but we are worried the soft UART may cause some other code delays and problems in our real time application.

Does anyone have any ideas/insight as to why the low baud rate on UART0 is stopping the device from being able to restart and how to overcome this issue?

Thank you
Warwick

[emulamer]

Also using Arduino on an ESP32 I have unintentionally duplicated this with a particularly long backtrace at 9600 baud. Thanks for this post or I never would have found the issue. Fortunately I can just bump the baud, but this is particularly nasty- not even the watchdog timer reboots it when it gets stuck.

[naikwadisd]

Seems the problem is still not solved on latest Arduino version 2.0.4/ IDF 4.4.2

Is someone looking into this? I am posting a sample test code tried on the latest version.

-------------------------------------------------------------------------------------------------------------------------
#include <esp_int_wdt.h>
#include <esp_task_wdt.h>
void setup()
{
esp_int_wdt_init();
esp_task_wdt_init(3, true); //enable panic so ESP32 restarts
esp_task_wdt_add(xTaskGetIdleTaskHandleForCPU(0));
esp_task_wdt_add(xTaskGetIdleTaskHandleForCPU(1));
esp_task_wdt_add(NULL); //add current thread to WDT watch*/

Serial1.begin(9600);
Serial2.begin(600);
Serial.begin(4800);
}

void loop()
{
float value= 1000/0;
float value2= value/1000;
Serial.println(value);
Serial.println(value2);
delay(1);
esp_task_wdt_reset(); // Watch dog timer reset
}

[warwick]

Hello,

I am wondering if anyone from Espressif can help on this please? This seems to be a bug in the ESP32 hardware, if reset is not possible when the UART 0 has been set to a low baud rate. I think all types of reset - WDT, brown out, etc are not working as the device simply freezes part way through printing the reset debug output and never resets.

We have this problem on a commercial product - we have deployed over 1,000 devices already and are finding some devices freezing in the field after days or months of operation and the only way to recover the device is a power recycle, which can require a few hours of travelling to get to each device as they are deployed on remote sites.

Any work around? Is there some way to Silent the debug output at reset (maybe a register we can set or clear during runtime) so that the device is able to reset even though UART 0 is running at a low baud rate? We would like to avoid making big changes to the framework (like making Arduino as a component) we are using, as our application is quite complex and we are worried about introducing new bugs into our code.

Thank you.

[ESP_Sprite]

If you think this is a bug in Arduino, you'd generally be better off raising an issue on that repo.

[naikwadisd]

The issue raised for this bug can be tracked here : https://github.com/espressif/arduino-esp32/issues/7092