How do I use espcoredump.py on binary coredump?

HighVoltage
Posts: 52
Joined: Mon Oct 24, 2022 9:37 pm

How do I use espcoredump.py on binary coredump?

Postby HighVoltage » Tue Mar 19, 2024 6:21 pm

I'm trying to use espcoredump to look at my binary coredump.

This is what the environment reports at windows command prompt start:

Code: Select all

Setting PYTHONNOUSERSITE, was not set
Using Python in C:\exp\python_env\idf4.4_py3.11_env\Scripts\
Python 3.11.2
Using Git in C:\exp\tools\idf-git\2.39.2\cmd\
git version 2.39.2.windows.1
Setting IDF_PATH: C:\exp\frameworks\esp-idf-v4.4.6

Adding ESP-IDF tools to PATH...
    C:\exp\tools\xtensa-esp-elf-gdb\11.2_20220823\xtensa-esp-elf-gdb\bin
    C:\exp\tools\xtensa-esp32-elf\esp-2021r2-patch5-8.4.0\xtensa-esp32-elf\bin
    C:\exp\tools\esp32ulp-elf\2.35_20220830\esp32ulp-elf\bin
    C:\exp\tools\cmake\3.23.1\bin
    C:\exp\tools\openocd-esp32\v0.12.0-esp32-20230419\openocd-esp32\bin
    C:\exp\tools\ninja\1.10.2\
    C:\exp\tools\idf-exe\1.0.3\
    C:\exp\tools\ccache\4.3\ccache-4.3-windows-64
    C:\exp\frameworks\esp-idf-v4.4.6\tools

Checking if Python packages are up to date...
Python requirements from C:\exp\frameworks\esp-idf-v4.4.6\requirements.txt are satisfied.

Done! You can now compile ESP-IDF projects.
Go to the project directory and run:

  idf.py build
I queried the options:

Code: Select all

C:\exp\myproj>python ..\frameworks\esp-idf-v4.4.6\components\espcoredump\espcoredump.py --help
usage: espcoredump.py [-h] [--chip {auto,esp32,esp32s2,esp32s3,esp32c3}] [--port PORT] [--baud BAUD]
                      [--gdb-timeout-sec GDB_TIMEOUT_SEC]
                      {dbg_corefile,info_corefile} ...

espcoredump.py v0.4-dev - ESP32 Core Dump Utility

positional arguments:
  {dbg_corefile,info_corefile}
    dbg_corefile        Starts GDB debugging session with specified corefile
    info_corefile       Print core dump info from file

options:
  -h, --help            show this help message and exit
  --chip {auto,esp32,esp32s2,esp32s3,esp32c3}
                        Target chip type
  --port PORT, -p PORT  Serial port device
  --baud BAUD, -b BAUD  Serial port baud rate used when flashing/reading
  --gdb-timeout-sec GDB_TIMEOUT_SEC
                        Overwrite the default internal delay for gdb responses

C:\exp\myproj>python ..\frameworks\esp-idf-v4.4.6\components\espcoredump\espcoredump.py info_corefile --help
usage: espcoredump.py info_corefile [-h] [--debug DEBUG] [--gdb GDB] [--core CORE] [--core-format {b64,elf,raw}] [--off OFF]
                                    [--save-core SAVE_CORE] [--rom-elf ROM_ELF] [--print-mem]
                                    prog

positional arguments:
  prog                  Path to program's ELF binary

options:
  -h, --help            show this help message and exit
  --debug DEBUG, -d DEBUG
                        Log level (0..3)
  --gdb GDB, -g GDB     Path to gdb
  --core CORE, -c CORE  Path to core dump file (if skipped core dump will be read from flash)
  --core-format {b64,elf,raw}, -t {b64,elf,raw}
                        File specified with "-c" is an ELF ("elf"), raw (raw) or base64-encoded (b64) binary
  --off OFF, -o OFF     Offset of coredump partition in flash (type "make partition_table" to see).
  --save-core SAVE_CORE, -s SAVE_CORE
                        Save core to file. Otherwise temporary core file will be deleted. Does not work with "-c"
  --rom-elf ROM_ELF, -r ROM_ELF
                        Path to ROM ELF file. Will use "<target>_rom.elf" if not specified
  --print-mem, -m       Print memory dump
So I try it as below. Is there something not configured in my environment? path? Strange errors...

Code: Select all

C:\exp\myproj>python ..\frameworks\esp-idf-v4.4.6\components\espcoredump\espcoredump.py info_corefile -t raw -c coredump.raw build\myproj.elf

espcoredump.py v0.4-dev
Traceback (most recent call last):
  File "C:\exp\frameworks\esp-idf-v4.4.6\components\espcoredump\espcoredump.py", line 387, in <module>
    temp_core_files = info_corefile()
                      ^^^^^^^^^^^^^^^
  File "C:\exp\frameworks\esp-idf-v4.4.6\components\espcoredump\espcoredump.py", line 181, in info_corefile
    core_elf_path, target, temp_files = get_core_dump_elf(e_machine=exe_elf.e_machine)
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\frameworks\esp-idf-v4.4.6\components\espcoredump\espcoredump.py", line 69, in get_core_dump_elf
    loader = ESPCoreDumpFileLoader(args.core, args.core_format == 'b64')
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\frameworks\esp-idf-v4.4.6\components\espcoredump\corefile\loader.py", line 575, in __init__
    self.target = self._load_core_src()
                  ^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\frameworks\esp-idf-v4.4.6\components\espcoredump\corefile\loader.py", line 179, in _load_core_src
    raise ESPCoreDumpLoaderError('Core dump version "0x%x" is not supported!' % self.dump_ver)
corefile.ESPCoreDumpLoaderError: Core dump version "0x61f4" is not supported!

C:\exp\dmcs>python ..\frameworks\esp-idf-v4.4.6\components\espcoredump\espcoredump.py
Traceback (most recent call last):
  File "C:\exp\frameworks\esp-idf-v4.4.6\components\espcoredump\espcoredump.py", line 371, in <module>
    if args.debug == 0:
       ^^^^^^^^^^
AttributeError: 'Namespace' object has no attribute 'debug'

User avatar
ESP_Roland
Posts: 265
Joined: Tue Oct 09, 2018 10:28 am

Re: How do I use espcoredump.py on binary coredump?

Postby ESP_Roland » Wed Mar 20, 2024 8:49 am

How did you obtain coredump.raw? From the the output I think it might not be in RAW format.

HighVoltage
Posts: 52
Joined: Mon Oct 24, 2022 9:37 pm

Re: How do I use espcoredump.py on binary coredump?

Postby HighVoltage » Wed Mar 20, 2024 4:59 pm

I've switched it to ELF format now so I can get a backtrace online. I'm now trying the same procedure with the ELF coredump and get errors as well.

I have a coredump partition, it is encrypted.

Exception is thrown, see this logged:

I (10549) esp_core_dump_flash: Save core dump to flash...
I (10555) esp_core_dump_flash: Erase flash 24576 bytes @ 0x14000
I (10850) esp_core_dump_flash: Write end offset 0x58a4, check sum length 4
I (10850) esp_core_dump_flash: Core dump has been saved to flash.

Use "esptool.py read_flash" to read the data from that partition.
Use "espsecure.py decrypt_flash_data" to decrypt it with my key.

Try to view it:

Code: Select all

>python ..\frameworks\esp-idf-v4.4.6\components\espcoredump\espcoredump.py info_corefile -t elf -c coredump-dec.elf build\myproj.elf
espcoredump.py v0.4-dev
Traceback (most recent call last):
  File "C:\exp\frameworks\esp-idf-v4.4.6\components\espcoredump\espcoredump.py", line 387, in <module>
    temp_core_files = info_corefile()
                      ^^^^^^^^^^^^^^^
  File "C:\exp\frameworks\esp-idf-v4.4.6\components\espcoredump\espcoredump.py", line 182, in info_corefile
    core_elf = ESPCoreDumpElfFile(core_elf_path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\frameworks\esp-idf-v4.4.6\components\espcoredump\corefile\elf.py", line 328, in __init__
    super(ESPCoreDumpElfFile, self).__init__(elf_path, _e_type, _e_machine)
  File "C:\exp\frameworks\esp-idf-v4.4.6\components\espcoredump\corefile\elf.py", line 132, in __init__
    self.read_elf(elf_path)
  File "C:\exp\frameworks\esp-idf-v4.4.6\components\espcoredump\corefile\elf.py", line 143, in read_elf
    header_tables = ElfHeaderTables.parse(elf_bytes)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\python_env\idf4.4_py3.11_env\Lib\site-packages\construct\core.py", line 288, in parse
    return self.parse_stream(io.BytesIO(data), **contextkw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\python_env\idf4.4_py3.11_env\Lib\site-packages\construct\core.py", line 300, in parse_stream
    return self._parsereport(stream, context, "(parsing)")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\python_env\idf4.4_py3.11_env\Lib\site-packages\construct\core.py", line 312, in _parsereport
    obj = self._parse(stream, context, path)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\python_env\idf4.4_py3.11_env\Lib\site-packages\construct\core.py", line 1982, in _parse
    subobj = sc._parsereport(stream, context, path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\python_env\idf4.4_py3.11_env\Lib\site-packages\construct\core.py", line 312, in _parsereport
    obj = self._parse(stream, context, path)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\python_env\idf4.4_py3.11_env\Lib\site-packages\construct\core.py", line 2440, in _parse
    return self.subcon._parsereport(stream, context, path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\python_env\idf4.4_py3.11_env\Lib\site-packages\construct\core.py", line 312, in _parsereport
    obj = self._parse(stream, context, path)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\python_env\idf4.4_py3.11_env\Lib\site-packages\construct\core.py", line 1982, in _parse
    subobj = sc._parsereport(stream, context, path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\python_env\idf4.4_py3.11_env\Lib\site-packages\construct\core.py", line 312, in _parsereport
    obj = self._parse(stream, context, path)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\python_env\idf4.4_py3.11_env\Lib\site-packages\construct\core.py", line 2440, in _parse
    return self.subcon._parsereport(stream, context, path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\python_env\idf4.4_py3.11_env\Lib\site-packages\construct\core.py", line 312, in _parsereport
    obj = self._parse(stream, context, path)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\python_env\idf4.4_py3.11_env\Lib\site-packages\construct\core.py", line 1982, in _parse
    subobj = sc._parsereport(stream, context, path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\python_env\idf4.4_py3.11_env\Lib\site-packages\construct\core.py", line 312, in _parsereport
    obj = self._parse(stream, context, path)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\python_env\idf4.4_py3.11_env\Lib\site-packages\construct\core.py", line 2440, in _parse
    return self.subcon._parsereport(stream, context, path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\python_env\idf4.4_py3.11_env\Lib\site-packages\construct\core.py", line 312, in _parsereport
    obj = self._parse(stream, context, path)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\python_env\idf4.4_py3.11_env\Lib\site-packages\construct\core.py", line 2512, in _parse
    raise ConstError("parsing expected %r but parsed %r" % (self.value, obj), path=path)
construct.core.ConstError: Error in path (parsing) -> elf_header -> e_ident -> EI_MAG
parsing expected b'\x7fELF' but parsed b'\xa4X\x00\x00'
My app also has a download link to grab it via browser. I tried that way as well, and got the same error. That version appears to be already decrypted, which makes sense. I got the same "but parsed" bytes as above from this downloaded file.

User avatar
ESP_Roland
Posts: 265
Joined: Tue Oct 09, 2018 10:28 am

Re: How do I use espcoredump.py on binary coredump?

Postby ESP_Roland » Thu Mar 21, 2024 8:23 am

Again, it looks that the file is not in ELF format. I'll ask my colleagues to take a look at this....

ESP-erhankur
Posts: 8
Joined: Fri Jan 08, 2021 10:37 pm

Re: How do I use espcoredump.py on binary coredump?

Postby ESP-erhankur » Thu Mar 21, 2024 10:05 am

At the beginning of the coredump elf file, there is a header part which is 20 Bytes in IDF-4.4. You have to trim that part before passing it to the espcoredump.py

On linux, something like;

Code: Select all

dd bs=20 skip=1 if=core.elf of=trimmed.elf
Then;

Code: Select all

espcoredump.py --chip esp32 info_corefile -c trimmed.elf -t elf your_app_elf

ESP-erhankur
Posts: 8
Joined: Fri Jan 08, 2021 10:37 pm

Re: How do I use espcoredump.py on binary coredump?

Postby ESP-erhankur » Thu Mar 21, 2024 10:43 am

And what happens when you try to run the below command without trimming the header part?

Code: Select all

espcoredump.py --chip esp32 info_corefile -c core.elf -t raw your_app_elf

HighVoltage
Posts: 52
Joined: Mon Oct 24, 2022 9:37 pm

Re: How do I use espcoredump.py on binary coredump?

Postby HighVoltage » Fri Mar 22, 2024 5:11 am

The chip option does not make a difference, but I did get a response when I trimmed the header.

It starts processing it, and then just hangs:

Code: Select all

espcoredump.py v0.4-dev
===============================================================
==================== ESP32 CORE DUMP START ====================

Crashed task handle: 0x3ffdcf3c, name: '', GDB name: 'process 1073598268'

================== CURRENT THREAD REGISTERS ===================
exccause       0x1d (StoreProhibitedCause)
excvaddr       0x0
epc1           0x401224ff
epc2           0x0
epc3           0x0
epc4           0x0
epc5           0x0
epc6           0x0
eps2           0x0
eps3           0x0
eps4           0x0
eps5           0x0
eps6           0x0


==================== CURRENT THREAD STACK =====================
[New process 1073598268]
[New process 1073527168]
[New process 1073529056]
[New process 1073535472]
[New process 1073525280]
[New process 1073594844]
[New process 1073604136]
[New process 1073508900]
[New process 1073411924]
[New process 1073608364]
[New process 1073608712]
[New process 1073513808]
[New process 1073625240]
[Current thread is 1 (process 1073598268)]

======================== THREADS INFO =========================

Traceback (most recent call last):
  File "C:\exp\frameworks\esp-idf-v4.4.6\components\espcoredump\espcoredump.py", line 387, in <module>
    temp_core_files = info_corefile()
                      ^^^^^^^^^^^^^^^
  File "C:\exp\frameworks\esp-idf-v4.4.6\components\espcoredump\espcoredump.py", line 237, in info_corefile
    threads, _ = gdb.get_thread_info()
                 ^^^^^^^^^^^^^^^^^^^^^
  File "C:\exp\frameworks\esp-idf-v4.4.6\components\espcoredump\corefile\gdb.py", line 114, in get_thread_info
    current_thread_id = result['current-thread-id']
                        ~~~~~~^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not subscriptable

ESP-erhankur
Posts: 8
Joined: Fri Jan 08, 2021 10:37 pm

Re: How do I use espcoredump.py on binary coredump?

Postby ESP-erhankur » Fri Mar 22, 2024 9:32 am

Is it possible to send your decoded coredump file? You can create an issue here and attach the file. https://github.com/espressif/esp-coredump

Or, if you can create an app to reproduce the issue, it would be perfect.

Looks like something is broken in the crashed task TCB/stack and can not be parsed from espcoredump.py.

We made some improvements in the latest IDFs (currently in 5.2 and master) for the broken task data. If you have a chance you can give it a try.

But for the previous IDFs, It seems we need more improvement on the parser side. I will have a look at it.

ESP-erhankur
Posts: 8
Joined: Fri Jan 08, 2021 10:37 pm

Re: How do I use espcoredump.py on binary coredump?

Postby ESP-erhankur » Fri Mar 22, 2024 1:46 pm

In the meantime, I have tested together IDF v4.4 + esp-coredump 0.4.0 with the binary and elf formatted coredump data. Both are ok for me. I didn't need to trim the header using the 'raw' type

Code: Select all

espcoredump.py info_corefile -t raw -c core.elf build/blink.elf

Code: Select all

espcoredump.py info_corefile -t raw -c core.bin build/blink.elf
I will be happy to find the root cause if you can share some more data as I mentioned in my previous reply.


Who is online

Users browsing this forum: Bing [Bot] and 175 guests