Debugging ESP32 programs on Linux using an "ESP32 Emulator Library"

[kolban]

As I build out ESP32 applications by editing source in Eclipse, compiling it with the ESP32 ESP-IDF framework and then deploying it and watching the console ... I found myself a little frustrated at the lag between edit, compile, test -> repeat. Obviously if I was a better programmer I would get it right first time

It then dawned on me that the majority of what I am working on is literal C code with a large chunk of POSIX API supplied by the "newlib" functions and a relatively small set of functions provided by ESP-IDF. At this point my light-bulb started glowing (at about 25 watts) and I said "What if I didn't develop ON the ESP32 but instead developed and test on Linux?". Sound pretty obvious ... but the down side is that there are "enough" functions that I am using in ESP32 ... GPIO, NVS, VFS, WiFi etc etc that my code would become cumbersome to maintain two sets or even one set with lots of #ifdefs. And this is where the notion of a Linux based "ESP32" emulator library comes into the picture. What if we developed a set of libraries that exposed the same public APIs as those exposed by ESP-IDF but instead ran on Linux? For example, a GPIO read/write would be visible and controlled on a browser based dash board to emulate a signal change, SPI/I2C would be logged and visualized as a signal analytics chart, WiFi would be mapped to the local network with the dashboard able to show "what was requested" and to generate interesting events. The majority of functions we will get for free because of POSIX compatibility.

Now obviously, I can't be even close to the first person to consider this notion so before I run away with myself and start planning and designing and coding ... I wanted to see if anyone has started such a project already (in which case I'll try and join) or maybe Espressif has this today and would be willing to share. If the answer to these is not yet ... then if anyone sees merit in it, post here and we can start a collaborative project to start breaking it up and "get er done".

[ESP_igrr]

We haven't started working on this yet, but I do have this on the plan for the next year. We have a new team member joining us soon (early January?), this will be one of the things he will be working on. I'm still unsure if this port should be integrated into the ESP-IDF, or be done as a standalone project. There are pros and cons with each approach.
Another thing, we do have an RTOS abstraction layer coming soon to the ESP-IDF, to support some other RTOSes within ESP-IDF without littering the code with #ifdef's. So we will have a way to add, say, Linux port of FreeRTOS without much pain.
Supporting Linux as a target for ESP-IDF is important for one more reason: it will help us find components which are not adequately structured, and don't have separation between generic functionality and target-specific implementation.

[kolban]

That is great news!!! Might this be a consideration for a community based sub-project? It would "appear" that relatively little ESP32 internals skills would be needed as the core (opinion) would be Linux based programming and a mapping of the ESP-IDF exposed functions. On loose thinking, we might also want to include a built-in WebServer that then exposed a dashboard for both output examination, logging controls and event/signal generation ... so the other skills would be web programming.

If there is community interest (I for one would sign-up) then maybe that would be an exciting project for the community to "build out" with guidance and instructions from Espressif. That might then un-burden y'all some grunt programming work (like this) to work on the internals and architecture stuff that can only be worked upon by Espressif staff?

Merely a thought ... looking to offer assistance where I can/

[brainstorm]

Kolban, I found this one recently, thought you might find it interesting:

https://github.com/Ebiroll/qemu_esp32

[Philius]

As useful as an "ESP32 Emulator Library" might be, you run into problems when you want to connect things to it, which is most/all of the time.
I've started looking into SimulIDE which allows you to drop an Arduino onto a schematic, wire components to its pins, upload firmware(!) and simulate the circuit!!!

My primary motivation is to be able to step through my ESP32 source code with kdbg, a gdb front end while running the circuit in the simulator.

I don't expect it to be able to simulate an ESP32 in real time so I've had a look into porting the esp-idf to linux using a freeRTOS pc port that works now, but is version 6.0.4.

I can't see a ChangeLog between 6.0.4 and 8.2.0, the version in the esp-idf right now, but I've made a start at commenting out + marking all assembler instructions with /* TODO SimulIDE */.
Also, where possible I've replaced parts of the freeRTOS with the posix simulator parts.

I'm calling it esp32-emu for now, suggestions for a better name are welcome!
Once I get a full compile, much less getting it running in SimulIDE I'll create a SourceForge/GitHub project so I'll need a name for it then.

[jsantosrico]

Hey,

Sorry for resurrecting an old post, but I'm getting into unit testing and was wondering if there ever were any advances in this topic. I did a brief search on the internet and found nothing, which leads me to think that apart from @Philius effort, nothing was released by Espressif.

Cheers!

[ESP_igrr]

Hi jsantosrico,
This got pushed back a bit, we plan to resume working on this once CMake support lands into IDF master branch.

[meowsqueak]

The Ebiroll QEMU project is really interesting in my opinion, because it's relatively easy to write device drivers "on the other side" of the emulation, then develop ESP32 code against those, allowing the exact same code to run on the target hardware or inside QEMU. What would be super-useful is if Espressif took this project onboard and developed a set of QEMU drivers for the ESP32 peripherals. I think it's worth a serious look, but it needs serious investment.

[leenowell]

This would be a fantastic addition. Do we have a rough idea when this would be possible?

[Koulwa]

Any progress update on this? Been about a year, Cmake seems pretty much done. Is this still on the roadmap? Maybe bumped up a bit in priority? For professional use, a simulator would be amazing. Would make a lot of sense for testing all rainy day scenarios much faster.