Help with Interrupt WDT Timeout Resets

Post Reply
jwwest
Posts: 6
Joined: Sat Dec 28, 2019 8:31 pm

Help with Interrupt WDT Timeout Resets

Postby jwwest » Fri Jun 09, 2023 9:47 pm

My project is experiencing intermittent resets due to the Interrupt WDT Timeout. I have a number of Guru meditation register dumps and backtraces as well as the corresponding core-dumps. Below I have inserted copies of a guru dump and the core dump from a single crash occurrence.

These crashes consistently have calls to the i2c_isr_handler_default in current thread and Core 0 always seems to be in ISR context.
There are 2 ISR functions that get printed, uart_hal_write_txfifo and i2c_isr_handler_default... but the EPCx registers in the guru dump don't exactly match what the core dump produced.

IDF version is 4.2.2.

Ultimately what I would like answered is how to make sense of the guru dump and the core dump... understanding the meaning of the different registers, especially the Exception registers, and the call stacks. Also, how is the stack managed when in ISR context?

The core dump has some issues, which I suspect may be due to heap corruption, or maybe stack overflow, but I've increased the stack sizes and still experience the problem as similar rates. There are 2 threads that tend to get stomped on in the core dump, and the threads info section lists a bunch of "New process nnnnn" messages at the end, which I don't understand.


Guru dump...
Code: [Select all] [Expand/Collapse]
  1. Guru Meditation Error: Core  0 panic'ed (Interrupt wdt timeout on CPU0).
  2.  
  3. Core  0 register dump:
  4. PC      : 0x400848e3  PS      : 0x00050434  A0      : 0x400826c8  A1      : 0x3ffb19d0
  5. 0x400848e3: i2c_isr_handler_default at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/driver/i2c.c:463
  6.  
  7. 0x400826c8: _xt_lowint1 at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/xtensa/xtensa_vectors.S:1105
  8.  
  9. A2      : 0x3ffc4e6c  A3      : 0x00000000  A4      : 0x00000000  A5      : 0x00000000
  10. A6      : 0x3ffb67f8  A7      : 0x00000000  A8      : 0x00000000  A9      : 0x3ffb19a0
  11. A10     : 0x3ffc4f54  A11     : 0x00060421  A12     : 0x3ffb19d8  A13     : 0x00000000
  12. A14     : 0x3ffc4f54  A15     : 0x3ffc4fa8  SAR     : 0x0000001e  EXCCAUSE: 0x00000005
  13. EXCVADDR: 0x00000000  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0xffffffff
  14. Core  0 was running in ISR context:
  15. EPC1    : 0x401982eb  EPC2    : 0x4000bff0  EPC3    : 0x00000000  EPC4    : 0x400848e3
  16. 0x401982eb: uart_hal_write_txfifo at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/hal/uart_hal_iram.c:35
  17.  
  18. 0x400848e3: i2c_isr_handler_default at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/driver/i2c.c:463
  19.  
  20.  
  21. Backtrace:0x400848e0:0x3ffb19d0 0x400826c5:0x3ffb1a00 0x400848f3:0x3ffd0770 0x4008ced5:0x3ffd0780 0x4008c583:0x3ffd07a0 0x400f67c0:0x3ffd07c0
  22. 0x400848e0: i2c_isr_handler_default at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/driver/i2c.c:460
  23.  
  24. 0x400826c5: _xt_lowint1 at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/xtensa/xtensa_vectors.S:1105
  25.  
  26. 0x400848f3: i2c_isr_handler_default at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/driver/i2c.c:488
  27.  
  28. 0x4008ced5: vPortExitCritical at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/xtensa/port.c:457
  29.  
  30. 0x4008c583: ulTaskNotifyTake at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/tasks.c:4651
  31.  
  32. 0x400f67c0: zeroCrossingTask at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/triac-driver/zero-crossing.c:92 (discriminator 1)
  33.  
  34. Core  1 register dump:
  35. PC      : 0x4008cec1  PS      : 0x00060a34  A0      : 0x8008dce6  A1      : 0x3ffcb2e0
  36. 0x4008cec1: vPortExitCritical at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/xtensa/port.c:448
  37.  
  38. A2      : 0x3ffb67fc  A3      : 0x3ffcb350  A4      : 0x000005b6  A5      : 0x3f437df0
  39. A6      : 0x00000000  A7      : 0x3ffc9d80  A8      : 0x00000004  A9      : 0x00060a23
  40. A10     : 0x0000abab  A11     : 0x3ffc4fb4  A12     : 0x00000004  A13     : 0x3ffcb354
  41. A14     : 0x00000000  A15     : 0x00000000  SAR     : 0x00000019  EXCCAUSE: 0x00000005
  42. EXCVADDR: 0x00000000  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0xffffffff
  43.  
  44. Backtrace:0x4008cebe:0x3ffcb2e0 0x4008dce3:0x3ffcb300 0x40109afb:0x3ffcb340 0x40143b96:0x3ffcb380 0x4014362a:0x3ffcb3a0 0x40143a6e:0x3ffcb3e0 0x40100179:0x3ffcb400 0x400fff39:0x3ffcb420 0x400ffab7:0x3ffcb440
  45. 0x4008cebe: vPortExitCritical at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/xtensa/port.c:448
  46.  
  47. 0x4008dce3: xQueueGenericReceive at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/queue.c:1556
  48.  
  49. 0x40109afb: i2c_master_cmd_begin at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/driver/i2c.c:1176
  50.  
  51. 0x40143b96: oven_i2c_write at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/led-driver/oven-i2c.c:117
  52.  
  53. 0x4014362a: writeWholeBuffer at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/led-driver/ledDriver.c:146
  54.  (inlined by) writeWholeBuffer at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/led-driver/ledDriver.c:129
  55.  
  56. 0x40143a6e: drawLedFlush at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/led-driver/ledDriver.c:434 (discriminator 1)
  57.  
  58. 0x40100179: __ui_instance_flush at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/UI/ui_instance_hw1.c:366
  59.  
  60. 0x400fff39: __ui_instance_wrapper_flush at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/UI/ui_instance_core.c:211
  61.  
  62. 0x400ffab7: uiThread at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/UI/ui.c:1915
  63.  
  64.  
  65.  
  66. ELF file SHA256: b904f1e13b0fb91b
GeSHi © Codebox Plus Extension

Coredump...
Code: [Select all] [Expand/Collapse]
  1.  
  2. ===============================================================
  3. ==================== ESP32 CORE DUMP START ====================
  4.  
  5. Crashed task handle: 0x3ffd0884, name: 'zeroCrossingTas', GDB name: 'process 1073547396'
  6.  
  7. ================== CURRENT THREAD REGISTERS ===================
  8. exccause       0x5 (AllocaCause)
  9. excvaddr       0x0
  10. epc1           0x4000bff0
  11. epc2           0x0
  12. epc3           0x400848e3
  13. epc4           0x0
  14. epc5           0x0
  15. epc6           0x0
  16. epc7           0x0
  17. eps2           0x60421
  18. eps3           0x0
  19. eps4           0x50421
  20. eps5           0x0
  21. eps6           0x0
  22. eps7           0x40089aae
  23. pc             0x400848e3   0x400848e3 <i2c_isr_handler_default+143>
  24. lbeg           0x4000c2e0   1073791712
  25. lend           0x4000c2f6   1073791734
  26. lcount         0xffffffff   4294967295
  27. sar            0x1e 30
  28. ps             0x50424  328740
  29. threadptr      <unavailable>
  30. br             <unavailable>
  31. scompare1      <unavailable>
  32. acclo          <unavailable>
  33. acchi          <unavailable>
  34. m0             <unavailable>
  35. m1             <unavailable>
  36. m2             <unavailable>
  37. m3             <unavailable>
  38. expstate       <unavailable>
  39. f64r_lo        <unavailable>
  40. f64r_hi        <unavailable>
  41. f64s           <unavailable>
  42. fcr            <unavailable>
  43. fsr            <unavailable>
  44. a0             0x400826c8   1074276040
  45. a1             0x3ffb19d0   1073420752
  46. a2             0x3ffc4e6c   1073499756
  47. a3             0x0  0
  48. a4             0x0  0
  49. a5             0x0  0
  50. a6             0x3ffb67f8   1073440760
  51. a7             0x0  0
  52. a8             0x0  0
  53. a9             0x3ffb19a0   1073420704
  54. a10            0x3ffc4f54   1073499988
  55. a11            0x60421  394273
  56. a12            0x3ffb19d8   1073420760
  57. a13            0x0  0
  58. a14            0x3ffc4f54   1073499988
  59. a15            0x3ffc4fa8   1073500072
  60.  
  61. ==================== CURRENT THREAD STACK =====================
  62. #0  0x400848e3 in i2c_isr_handler_default (arg=0x3ffc4e6c) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/driver/i2c.c:460
  63. #1  0x400826c8 in _xt_lowint1 () at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/xtensa/xtensa_vectors.S:1105
  64. #2  0x400848f6 in i2c_isr_handler_default (arg=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/driver/i2c.c:488
  65. #3  0x4008ced8 in vPortExitCritical (mux=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/xtensa/port.c:457
  66. #4  0x4008c586 in ulTaskNotifyTake (xClearCountOnExit=1, xTicksToWait=4294967295) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/tasks.c:4651
  67. #5  0x400f67c3 in zeroCrossingTask (pvParameters=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/triac-driver/zero-crossing.c:92
  68.  
  69. ======================== THREADS INFO =========================
  70.   Id   Target Id         Frame
  71. * 1    process 1073547396 0x400848e3 in i2c_isr_handler_default (arg=0x3ffc4e6c) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/driver/i2c.c:460
  72.   2    process 1073532180 0x4000bff0 in _xtos_set_intlevel ()
  73.   3    process 1073549588 0x4008dd41 in xQueueGenericReceive (xQueue=0x3ffd1290, pvBuffer=0x3ffd1050, xTicksToWait=<optimized out>, xJustPeeking=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  74.   4    process 1073526148 0x00000000 in ?? ()
  75.   5    process 1073610168 spin_task (arg=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/oven/main/stats.c:142
  76.   6    process 1073609788 0x400d5552 in spin_task (arg=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/oven/main/stats.c:141
  77.   7    process 1073479616 0x40193956 in esp_pm_impl_waiti () at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/esp32/pm_esp32.c:501
  78.   8    process 1073481552 0x40193956 in esp_pm_impl_waiti () at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/esp32/pm_esp32.c:501
  79.   9    process 1073509092 vTaskDelay (xTicksToDelay=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/tasks.c:1489
  80.   10   process 1073545716 vTaskDelayUntil (pxPreviousWakeTime=<optimized out>, xTimeIncrement=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/tasks.c:1423
  81.   11   process 1073592948 0x4008dd41 in xQueueGenericReceive (xQueue=0x3ffdabf0, pvBuffer=0x0, xTicksToWait=<optimized out>, xJustPeeking=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  82.   12   process 1073473980 0x4008dd41 in xQueueGenericReceive (xQueue=0x3ffbeb80, pvBuffer=0x0, xTicksToWait=<optimized out>, xJustPeeking=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  83.   13   process 1073598780 vTaskDelay (xTicksToDelay=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/tasks.c:1489
  84.   14   process 1073485324 0x4008cbb3 in prvProcessTimerOrBlockTask (xListWasEmpty=<optimized out>, xNextExpireTime=994448) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  85.   15   process 1073609296 0x4008dd41 in xQueueGenericReceive (/builds/idf/crosstool-NG/.build/HOST-x86_64-apple-darwin12/xtensa-esp32-elf/src/gdb/gdb/inline-frame.c:167: internal-error: void inline_frame_this_id(frame_info*, void**, frame_id*): Assertion `frame_id_p (*this_id)' failed.
  86. A problem internal to GDB has been detected,
  87. further debugging may prove unreliable.
  88. Quit this debugging session? (y or n) [answered Y; input not from terminal]
  89. /builds/idf/crosstool-NG/.build/HOST-x86_64-apple-darwin12/xtensa-esp32-elf/src/gdb/gdb/inline-frame.c:167: internal-error: void inline_frame_this_id(frame_info*, void**, frame_id*): Assertion `frame_id_p (*this_id)' failed.
  90. A problem internal to GDB has been detected,
  91. further debugging may prove unreliable.
  92. Create a core file of GDB? (y or n) [answered Y; input not from terminal]
  93. ERROR: GDB exited (None / )!
  94. ERROR: Problem occured! GDB exited, restart it.
  95. Reading symbols from build/oven-controller.elf...done.
  96. [New process 1073547396]
  97. [New process 1073532180]
  98. [New process 1073549588]
  99. [New process 1073526148]
  100. [New process 1073610168]
  101. [New process 1073609788]
  102. [New process 1073479616]
  103. [New process 1073481552]
  104. [New process 1073509092]
  105. [New process 1073545716]
  106. [New process 1073592948]
  107. [New process 1073473980]
  108. [New process 1073598780]
  109. [New process 1073485324]
  110. [New process 1073609296]
  111. [New process 1073555680]
  112. [New process 1073519604]
  113. [New process 1073617060]
  114. [New process 1073469452]
  115. [New process 1073471212]
  116. [New process 1073566312]
  117. [New process 1073412956]
  118. [New process 1073505624]
  119. [New process 1073488772]
  120. [Current thread is 1 (process 1073547396)]
  121.  
  122. ==================== THREAD 1 (TCB: 0x3ffd0884, name: 'zeroCrossingTas') =====================
  123. #0  0x400848e3 in i2c_isr_handler_default (arg=0x3ffc4e6c) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/driver/i2c.c:460
  124. #1  0x400826c8 in _xt_lowint1 () at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/xtensa/xtensa_vectors.S:1105
  125. #2  0x400848f6 in i2c_isr_handler_default (arg=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/driver/i2c.c:488
  126. #3  0x4008ced8 in vPortExitCritical (mux=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/xtensa/port.c:457
  127. #4  0x4008c586 in ulTaskNotifyTake (xClearCountOnExit=1, xTicksToWait=4294967295) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/tasks.c:4651
  128. #5  0x400f67c3 in zeroCrossingTask (pvParameters=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/triac-driver/zero-crossing.c:92
  129.  
  130. ==================== THREAD 2 (TCB: 0x3ffccd14, name: 'ocbThread') =====================
  131. #0  0x4000bff0 in ?? ()
  132. #1  0x4008beb6 in xTaskGetSchedulerState () at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/tasks.c:4019
  133. #2  0x4008d99c in xQueueGenericSend (xQueue=0x3ffcdd3c, pvItemToQueue=0x0, xTicksToWait=<optimized out>, xCopyPosition=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/queue.c:738
  134. #3  0x4010f288 in uart_tx_all (uart_num=<optimized out>, src=<optimized out>, size=<optimized out>, brk_en=<optimized out>, brk_len=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/driver/uart.c:1112
  135. #4  0x4010f7a8 in uart_tx_all (brk_len=0, brk_en=false, size=1, src=0x3ffcc5c0 \"[\", uart_num=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/driver/uart.c:1121
  136. #5  uart_write_bytes (uart_num=0, src=0x3ffcc5c0 \"[\", size=1) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/driver/uart.c:1121
  137. #6  0x400f6f2e in uart_tx_char_via_driver (fd=0, c=91) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/vfs/vfs_uart.c:177
  138. #7  0x400f7150 in uart_write (fd=0, data=<optimized out>, size=128) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/vfs/vfs_uart.c:221
  139. #8  0x400f7810 in esp_vfs_write (r=<optimized out>, fd=1, data=0x3ffbe568, size=128) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/vfs/vfs.c:420
  140. #9  0x4000bd86 in ?? ()
  141. #10 0x40001180 in ?? ()
  142. #11 0x40059301 in ?? ()
  143. #12 0x4005937d in ?? ()
  144. #13 0x40058b6c in ?? ()
  145. #14 0x400da496 in __sprint_r (uio=0x3ffcc944, fp=0x3ffae98c, ptr=0x3ffccd80) at /builds/idf/crosstool-NG/.build/HOST-x86_64-apple-darwin12/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/vfprintf.c:433
  146. #15 __sprint_r (ptr=0x3ffccd80, fp=0x3ffae98c, uio=0x3ffcc944) at /builds/idf/crosstool-NG/.build/HOST-x86_64-apple-darwin12/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/vfprintf.c:403
  147. #16 0x400dfa62 in _vfprintf_r (data=<optimized out>, fp=<optimized out>, fmt0=<optimized out>, ap=...) at /builds/idf/crosstool-NG/.build/HOST-x86_64-apple-darwin12/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/vfprintf.c:1781
  148. #17 0x400dfbd8 in vprintf (fmt=0x3f406ff0 \"  ---  \", ap=...) at /builds/idf/crosstool-NG/.build/HOST-x86_64-apple-darwin12/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/vprintf.c:34
  149. #18 0x400f4e64 in tcpLogPrint (msg=0x3f406ff0 \"  ---  \", args=...) at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/tcp/tcp.c:220
  150. #19 0x400f4f00 in tcpPrintfRedef (format=0x3f406ff0 \"  ---  \") at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/tcp/tcp.c:49
  151. #20 0x400eb4db in ovenMainCook (msg=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/oven-control-block/oven-control.c:1751
  152. #21 0x400eaae0 in ocbThread (pvParameters=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/oven-control-block/oven-control.c:444
  153.  
  154. ==================== THREAD 3 (TCB: 0x3ffd1114, name: 'powerConsumptio') =====================
  155. #0  0x4008dd41 in xQueueGenericReceive (xQueue=0x3ffd1290, pvBuffer=0x3ffd1050, xTicksToWait=<optimized out>, xJustPeeking=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  156. #1  0x400f63b0 in powerConsumption (pvParameters=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/triac-driver/triac.c:557
  157.  
  158. ERROR: GDB/MI command failed (error / msg="Cannot access memory at address 0x60a17")!
  159. WARNING: Unable to switch to thread 4
  160.  
  161. ==================== THREAD 5 (TCB: 0x3ffdfdb8, name: 'spin1') =====================
  162. #0  spin_task (arg=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/oven/main/stats.c:142
  163.  
  164. ==================== THREAD 6 (TCB: 0x3ffdfc3c, name: 'spin0') =====================
  165. #0  0x400d5552 in spin_task (arg=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/oven/main/stats.c:141
  166.  
  167. ==================== THREAD 7 (TCB: 0x3ffbffc0, name: 'IDLE0') =====================
  168. #0  0x40193956 in esp_pm_impl_waiti () at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/esp32/pm_esp32.c:501
  169. #1  0x400d4b1c in esp_vApplicationIdleHook () at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/esp_common/src/freertos_hooks.c:63
  170. #2  0x4008ad77 in prvIdleTask (pvParameters=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/tasks.c:3386
  171.  
  172. ==================== THREAD 8 (TCB: 0x3ffc0750, name: 'IDLE1') =====================
  173. #0  0x40193956 in esp_pm_impl_waiti () at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/esp32/pm_esp32.c:501
  174. #1  0x400d4b1c in esp_vApplicationIdleHook () at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/esp_common/src/freertos_hooks.c:63
  175. #2  0x4008ad77 in prvIdleTask (pvParameters=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/tasks.c:3386
  176.  
  177. ==================== THREAD 9 (TCB: 0x3ffc72e4, name: 'SDM Main Task') =====================
  178. #0  vTaskDelay (xTicksToDelay=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/tasks.c:1489
  179. #1  0x400f347d in sdm_acquire_temp_afe_one (channel=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/sensor-data-manager/sdm_main.c:1501
  180. #2  sdm_read_TEMPn (TEMPn=<optimized out>, slot=<optimized out>, slot_max=5) at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/sensor-data-manager/sdm_main.c:1595
  181. #3  0x400f35bf in sdm_main_task (pvParameters=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/sensor-data-manager/sdm_main.c:415
  182.  
  183. ==================== THREAD 10 (TCB: 0x3ffd01f4, name: 'sensorMonitorTh') =====================
  184. #0  vTaskDelayUntil (pxPreviousWakeTime=<optimized out>, xTimeIncrement=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/tasks.c:1423
  185. #1  0x40140be2 in RunOnInterval (task=0x400f3b90 <monitors>, milliSeconds=<optimized out>, args=0x3ffb6944 <threadArgs$11511>) at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/helpers/helpers.c:17
  186. #2  0x400f3f06 in sensorMonitorThread (arg=0x3ffb6944 <threadArgs$11511>) at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/sensor-monitor/sensor-monitor-thread.c:261
  187.  
  188. ==================== THREAD 11 (TCB: 0x3ffdba74, name: 'touchSvc') =====================
  189. #0  0x4008dd41 in xQueueGenericReceive (xQueue=0x3ffdabf0, pvBuffer=0x0, xTicksToWait=<optimized out>, xJustPeeking=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  190. #1  0x4010295e in touch_pads_reader_task (arg=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/UI/touch_pad_service.c:424
  191.  
  192. ==================== THREAD 12 (TCB: 0x3ffbe9bc, name: 'websocket_task') =====================
  193. #0  0x4008dd41 in xQueueGenericReceive (xQueue=0x3ffbeb80, pvBuffer=0x0, xTicksToWait=<optimized out>, xJustPeeking=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  194. #1  0x40157591 in sys_arch_sem_wait (sem=0x3ffbeb68, timeout=1000) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/lwip/port/esp32/freertos/sys_arch.c:184
  195. #2  0x40146809 in lwip_select (maxfdp1=55, readset=0x3ffe3898, writeset=0x0, exceptset=0x3ffe3890, timeout=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/lwip/lwip/src/api/sockets.c:2153
  196. #3  0x400f7e70 in esp_vfs_select (nfds=55, readfds=0x3ffe3898, writefds=0x0, errorfds=0x3ffe3890, timeout=0x3ffe38a0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/vfs/vfs.c:985
  197. #4  0x401843f5 in base_poll_read (t=0x3ffbf048, timeout_ms=1000) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/tcp_transport/transport_ssl.c:150
  198. #5  0x40195caa in esp_transport_poll_read (t=<optimized out>, timeout_ms=1000) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/tcp_transport/transport.c:156
  199. #6  0x401923aa in ws_poll_read (t=0x3ffbf12c, timeout_ms=1000) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/tcp_transport/transport_ws.c:528
  200. #7  0x40195caa in esp_transport_poll_read (t=<optimized out>, timeout_ms=1000) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/tcp_transport/transport.c:156
  201. #8  0x4018f168 in esp_websocket_client_task (pv=0x3ffd568c) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/esp_websocket_client/esp_websocket_client.c:689
  202.  
  203. ==================== THREAD 13 (TCB: 0x3ffdd13c, name: 'otaCheckTask') =====================
  204. #0  vTaskDelay (xTicksToDelay=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/tasks.c:1489
  205. #1  0x400e974e in otaCheckTask (pvParameter=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/ota/ota.c:766
  206.  
  207. ==================== THREAD 14 (TCB: 0x3ffc160c, name: 'Tmr Svc') =====================
  208. #0  0x4008cbb3 in prvProcessTimerOrBlockTask (xListWasEmpty=<optimized out>, xNextExpireTime=994448) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  209. #1  prvTimerTask (pvParameters=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/freertos/timers.c:544
  210.  
  211. ERROR: GDB exited (None / )!
  212. ERROR: Problem occured! GDB exited, restart it.
  213. ERROR: GDB exited (None / )!
  214. ERROR: Problem occured! GDB exited, restart it.
  215. WARNING: Unable to switch to thread 15
  216.  
  217. ==================== THREAD 16 (TCB: 0x3ffd28e0, name: 'tiT') =====================
  218. #0  0x4008dd41 in xQueueGenericReceive (xQueue=0x3ffd16a4, pvBuffer=0x3ffd2630, xTicksToWait=<optimized out>, xJustPeeking=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  219. #1  0x4015767c in sys_arch_mbox_fetch (mbox=<optimized out>, msg=0x3ffd2630, timeout=13) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/lwip/port/esp32/freertos/sys_arch.c:325
  220. #2  0x40146bc2 in tcpip_timeouts_mbox_fetch (mbox=<optimized out>, msg=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/lwip/lwip/src/api/tcpip.c:110
  221. #3  tcpip_thread (arg=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/lwip/lwip/src/api/tcpip.c:148
  222.  
  223. ==================== THREAD 17 (TCB: 0x3ffc9bf4, name: 'cliTask') =====================
  224. #0  0x4008dd41 in xQueueGenericReceive (xQueue=0x3ffcdef0, pvBuffer=0x0, xTicksToWait=<optimized out>, xJustPeeking=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  225. #1  0x40088800 in prvReceiveGeneric (pxRingbuffer=0x3ffcde58, pvItem1=0x3ffc98c4, pvItem2=0x0, xItemSize1=0x3ffc98c0, xItemSize2=0x0, xMaxSize=0, xTicksToWait=xTicksToWait@entry=4294967295) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/esp_ringbuf/ringbuf.c:748
  226. #2  0x40088c5c in xRingbufferReceive (xRingbuffer=0x3ffcde58, pxItemSize=0x3ffc9900, xTicksToWait=4294967295) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/esp_ringbuf/ringbuf.c:1104
  227. #3  0x4010f865 in uart_read_bytes (uart_num=0, buf=0x3ffc9940 \"\\004\", length=1, ticks_to_wait=4294967295) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/driver/uart.c:1164
  228. #4  0x400f6f10 in uart_rx_char_via_driver (fd=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/vfs/vfs_uart.c:197
  229. #5  0x400f686f in uart_read_char (fd=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/vfs/vfs_uart.c:239
  230. #6  0x400f71a6 in uart_read (size=1, data=0x3ffae96b, fd=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/vfs/vfs_uart.c:256
  231. #7  uart_read (fd=0, data=0x3ffae96b, size=1) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/vfs/vfs_uart.c:249
  232. #8  0x400f78a4 in esp_vfs_read (r=<optimized out>, fd=0, dst=0x3ffae96b, size=1) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/vfs/vfs.c:446
  233. #9  0x4000bdbe in ?? ()
  234. #10 0x40001128 in ?? ()
  235. #11 0x400594ec in ?? ()
  236. #12 0x40170fde in __srget_r (ptr=0x3ffc9c60, fp=0x3ffae924) at /builds/idf/crosstool-NG/.build/HOST-x86_64-apple-darwin12/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/rget.c:42
  237. #13 0x40170a44 in _fgetc_r (ptr=0x3ffc9c60, fp=0x3ffae924) at /builds/idf/crosstool-NG/.build/HOST-x86_64-apple-darwin12/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/fgetc.c:94
  238. #14 0x40170a82 in fgetc (fp=0x3ffae924) at /builds/idf/crosstool-NG/.build/HOST-x86_64-apple-darwin12/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/fgetc.c:114
  239. #15 0x4010717c in linenoiseDumb (buflen=4096, prompt=<optimized out>, buf=0x3ffce1cc <error: Cannot access memory at address 0x3ffce1cc>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/console/linenoise/linenoise.c:946
  240. #16 linenoise (prompt=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/console/linenoise/linenoise.c:988
  241. #17 0x40104ceb in cliTask (pvParameters=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/anova-cli/anovaCLImain.c:2270
  242.  
  243. ==================== THREAD 18 (TCB: 0x3ffe18a4, name: 'stats') =====================
  244. #0  0x4008c12a in vTaskSuspend (xTaskToSuspend=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  245. #1  0x400d55b0 in stats_task (arg=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/oven/main/stats.c:165
  246.  
  247. ==================== THREAD 19 (TCB: 0x3ffbd80c, name: 'ipc0') =====================
  248. #0  0x4008dd41 in xQueueGenericReceive (xQueue=0x3ffbd388, pvBuffer=0x0, xTicksToWait=<optimized out>, xJustPeeking=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  249. #1  0x40081e62 in ipc_task (arg=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/esp_ipc/ipc.c:51
  250.  
  251. ==================== THREAD 20 (TCB: 0x3ffbdeec, name: 'ipc1') =====================
  252. #0  0x4008dd41 in xQueueGenericReceive (xQueue=0x3ffbda68, pvBuffer=0x0, xTicksToWait=<optimized out>, xJustPeeking=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  253. #1  0x40081e62 in ipc_task (arg=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/esp_ipc/ipc.c:51
  254.  
  255. ==================== THREAD 21 (TCB: 0x3ffd5268, name: 'wifi') =====================
  256. #0  0x4008dd41 in xQueueGenericReceive (xQueue=0x3ffd30f0, pvBuffer=0x3ffd5170, xTicksToWait=<optimized out>, xJustPeeking=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  257. #1  0x40116351 in queue_recv_wrapper (queue=0x3ffd30f0, item=0x3ffd5170, block_time_tick=4294967295) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/esp_wifi/esp32/esp_adapter.c:352
  258. #2  0x40093314 in ppTask ()
  259.  
  260. ==================== THREAD 22 (TCB: 0x3ffafb5c, name: 'esp_timer') =====================
  261. #0  0x4008dd41 in xQueueGenericReceive (xQueue=0x3ffaead8, pvBuffer=0x0, xTicksToWait=<optimized out>, xJustPeeking=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  262. #1  0x400d4d39 in timer_task (arg=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/esp_timer/src/esp_timer.c:329
  263.  
  264. ==================== THREAD 23 (TCB: 0x3ffc6558, name: 'wavTaskLoop') =====================
  265. #0  0x4008dd41 in xQueueGenericReceive (xQueue=0x3ffc66d4, pvBuffer=0x3ffc64b0, xTicksToWait=<optimized out>, xJustPeeking=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  266. #1  0x401053b3 in wavTaskLoop (pvParameter=<optimized out>) at /Users/Shared/xxxx/idf-4-2-2-apo/app/components/audio/audio.c:141
  267.  
  268. ==================== THREAD 24 (TCB: 0x3ffc2384, name: 'sys_evt') =====================
  269. #0  0x4008dd41 in xQueueGenericReceive (xQueue=0x3ffbe174, pvBuffer=0x3ffc22a0, xTicksToWait=<optimized out>, xJustPeeking=0) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/soc/src/esp32/include/hal/cpu_ll.h:39
  270. #1  0x40197814 in esp_event_loop_run (event_loop=0x3ffbe148, ticks_to_run=4294967295) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/esp_event/esp_event.c:624
  271. #2  0x40197987 in esp_event_loop_run_task (args=0x3ffbe148) at /Users/Shared/xxxx/idf-4-2-2-apo/esp-idf/components/esp_event/esp_event.c:115
  272.  
  273.  
  274. ======================= ALL MEMORY REGIONS ========================
  275. Name   Address   Size   Attrs
  276. .rtc.text 0x400c0000 0x0 RW
  277. .rtc.dummy 0x3ff80000 0x0 RW
  278. .rtc.force_fast 0x3ff80000 0x0 RW
  279. .rtc_noinit 0x50000000 0x0 RW
  280. .rtc.force_slow 0x50000000 0x0 RW
  281. .iram0.vectors 0x40080000 0x400 R XA
  282. .iram0.text 0x40080400 0x19294 R XA
  283. .dram0.data 0x3ffb0000 0x5d78 RW A
  284. .noinit 0x3ffb5d78 0x0 RW
  285. .flash.appdesc 0x3f400020 0x100 R  A
  286. .flash.rodata 0x3f400120 0x38520 RW A
  287. .flash.text 0x400d0020 0xc83c8 R XA
  288. .iram0.text_end 0x40099694 0x0 RW
  289. .iram0.data 0x40099694 0x0 RW
  290. .iram0.bss 0x40099694 0x0 RW
  291. .dram0.heap_start 0x3ffbd360 0x0 RW
  292. .coredump.tasks.data 0x3ffd06b0 0x1bc RW
  293. .coredump.tasks.data 0x3ffd0884 0x168 RW
  294. .coredump.tasks.data 0x3ffccd14 0x168 RW
  295. .coredump.tasks.data 0x3ffcc400 0x8fc RW
  296. .coredump.tasks.data 0x3ffd1114 0x168 RW
  297. .coredump.tasks.data 0x3ffd0f50 0x1ac RW
  298. .coredump.tasks.data 0x3ffcb584 0x168 RW
  299. .coredump.tasks.data 0x3ffcb240 0x32c RW
  300. .coredump.tasks.data 0x3ffdfdb8 0x168 RW
  301. .coredump.tasks.data 0x3ffe0b30 0x148 RW
  302. .coredump.tasks.data 0x3ffdfc3c 0x168 RW
  303. .coredump.tasks.data 0x3ffe0720 0x144 RW
  304. .coredump.tasks.data 0x3ffbffc0 0x168 RW
  305. .coredump.tasks.data 0x3ffbfe20 0x188 RW
  306. .coredump.tasks.data 0x3ffc0750 0x168 RW
  307. .coredump.tasks.data 0x3ffc05b0 0x188 RW
  308. .coredump.tasks.data 0x3ffc72e4 0x168 RW
  309. .coredump.tasks.data 0x3ffc7070 0x25c RW
  310. .coredump.tasks.data 0x3ffd01f4 0x168 RW
  311. .coredump.tasks.data 0x3ffd0040 0x19c RW
  312. .coredump.tasks.data 0x3ffdba74 0x168 RW
  313. .coredump.tasks.data 0x3ffdb8b0 0x1ac RW
  314. .coredump.tasks.data 0x3ffbe9bc 0x168 RW
  315. .coredump.tasks.data 0x3ffe3650 0x374 RW
  316. .coredump.tasks.data 0x3ffdd13c 0x168 RW
  317. .coredump.tasks.data 0x3ffdcfa0 0x184 RW
  318. .coredump.tasks.data 0x3ffc160c 0x168 RW
  319. .coredump.tasks.data 0x3ffc14a0 0x154 RW
  320. .coredump.tasks.data 0x3ffdfa50 0x168 RW
  321. .coredump.tasks.data 0x3ffdf860 0x1d8 RW
  322. .coredump.tasks.data 0x3ffd28e0 0x168 RW
  323. .coredump.tasks.data 0x3ffd2500 0x1c0 RW
  324. .coredump.tasks.data 0x3ffc9bf4 0x168 RW
  325. .coredump.tasks.data 0x3ffc9760 0x47c RW
  326. .coredump.tasks.data 0x3ffe18a4 0x168 RW
  327. .coredump.tasks.data 0x3ffe16f0 0x19c RW
  328. .coredump.tasks.data 0x3ffbd80c 0x168 RW
  329. .coredump.tasks.data 0x3ffbd670 0x184 RW
  330. .coredump.tasks.data 0x3ffbdeec 0x168 RW
  331. .coredump.tasks.data 0x3ffbdd50 0x184 RW
  332. .coredump.tasks.data 0x3ffd5268 0x168 RW
  333. .coredump.tasks.data 0x3ffd5050 0x1bc RW
  334. .coredump.tasks.data 0x3ffafb5c 0x168 RW
  335. .coredump.tasks.data 0x3ffaf9b0 0x194 RW
  336. .coredump.tasks.data 0x3ffc6558 0x168 RW
  337. .coredump.tasks.data 0x3ffc63b0 0x190 RW
  338. .coredump.tasks.data 0x3ffc2384 0x168 RW
  339. .coredump.tasks.data 0x3ffc2190 0x1dc RW
  340.  
  341. ===================== ESP32 CORE DUMP END =====================
  342. ===============================================================
  343. Done!
GeSHi © Codebox Plus Extension

-jw
Top
MicroController
Posts: 1726
Joined: Mon Oct 17, 2022 7:38 pm
Location: Europe, Germany

Re: Help with Interrupt WDT Timeout Resets

Postby MicroController » Sat Jun 10, 2023 11:27 am

Note that the name "interrupt WDT" may be a bit confusing. As per the docs, the IWDT is "fed" from the FreeRTOS tick ISR and as such only detects if this specific ISR did not run for a certain amount of time.
Hence, the point at which the IWDT triggers may be not directly linked to the problematic code section.
In your case for example, it is unlikely that the I2C ISR runs longer than the IWDT timeout period.

Assuming your own code doesn't spend excessive time in a critical section, I think it is possible that in your application there may be periods with a lot of interrupts happening in quick succession and that this causes execution to basically jump back and forth between critical sections (queue/semaphore operations) and ISRs.

Another thing is the "spin_task"s. Name and stack trace suggest that these tasks may starve the IDLE task of CPU time, which among other things can cause FreeRTOS timer callbacks to not be processed in time.

As to the gaps in the backtrace, this does not necessarily indicate any problem. Not 100% certain about the Xtensa architecture, but on some/many platforms the call stack may fail to be correctly traced back if/when the exact stack layout is not known at build time, which can happen if/when a function allocates a dynamic amount of space from the stack at runtime (e.g. variable length arrays).
Top
jwwest
Posts: 6
Joined: Sat Dec 28, 2019 8:31 pm

Re: Help with Interrupt WDT Timeout Resets

Postby jwwest » Sat Jun 10, 2023 1:50 pm

Hi MC,
Thank you for your response, it has given me a couple of things to ponder in hopes to get to the bottom of this issue.

It may be quite possible that the I2C handling and the UART handling are consuming quite a bit of time and thus preventing FreeRTOS from "kicking" the IWDT. I think I will try disabling the IWDT in a test and collect some metrics from the FreeRTOS tick handler on just how long it is getting starved. Not quite sure how I to do that off the top of my head though.

As for the spin_tasks, I have enabled the FreeRTOS stats collection, but only allow it to run when for 1.5 seconds via a console command. The 2 spin_tasks sit in the suspended state otherwise.

-jw
Top
MicroController
Posts: 1726
Joined: Mon Oct 17, 2022 7:38 pm
Location: Europe, Germany

Re: Help with Interrupt WDT Timeout Resets

Postby MicroController » Sat Jun 10, 2023 2:33 pm

collect some metrics from the FreeRTOS tick handler
esp_register_freertos_tick_hook(...)

If you determine that the condition the IWDT detects is only temporary and normal in your application, you can of course always set a longer timeout period.
Top
Post Reply

Return to “ESP-IDF”

Who is online

Users browsing this forum: No registered users and 155 guests