Impact of concurrency constraints in SPI Flash interface

[szguyer]

Hello all,

I am trying to track down some issues in the FastLED support for ESP32, particularly when combined with SPIFFS (for loading web interface stuff and storing animation patterns). We have had a number of reports that our driver (which uses the RMT peripheral to push bits out to the LED strips) produces weird glitchy effects when the program is accessing Flash memory using the SPI API. I recently ran across this discussion on the GitHub page:

https://github.com/espressif/esp-idf/tr ... sh-on-spi1

One issue is that apparently there is a bug in the xtensa support in gcc: the IRAM_ATTR is not properly retained during template expansion. As a result, some of the key functions in our driver are not in IRAM; in particular, some of the functions called by the interrupt handler. (We have a bug report filed)

BUT, it still seems puzzling why there would be interference between SPIFFS and FastLED because for the most part people call them synchronously. That is, they open a file, read it, and close it, then call FastLED.show(). It really shouldn't matter if the FastLED code is not in IRAM.

My question (finally) is whether there might be some other reason for this problem. Is there SPI bus activity going on outside the explicit flash read and write calls? Maybe some async task that is pulling data in or syncing the meta-data?

Any help would be greatly appreciated!

[ESP_igrr]

Outside of SPI flash operations, cache should not get disabled. It is possible that some other task performs flash operations, though, such as Wi-Fi when connection happens — it will store some data in NVS.


Regarding the section attribute on template functions, a quick test shows that the behavior is same with GCC configured for arm-none-eabi, so it doesn't look like this is Xtensa specific?

[szguyer]

Thanks Ivan! On the first point, how can I check to see if some other code is disabling the caches? What API is used to disable/enable them?

Yes, you're right about the compiler issue -- it is not Xtensa specific. I'm not the one who discovered the problem, so I didn't investigate it in detail. We filed a bug report, but who knows how long it takes to fix these things. I would have never noticed, though, that template methods are not going into IRAM if it weren't for this weird interaction with the flash memory.

[szguyer]

Would it be reasonable to call spi_flash_op_lock() explicitly before I start a time-sensitive operation (and then spi_flash_op_unlock() at the end)? I just need some way to avoid flash operations shutting off the caches in the middle of the process of sending bits to the LED strips, since the timing is so critical.

I'm in a bind here because of three things (none of which I can easily change):

1. The FastLED library relies on templates (mostly for performance reasons on other platforms besides ESP)
2. The IRAM_ATTR does not work on templated methods (a gcc bug)
3. The SPI flash interface on the ESP32 disables all code that is not in IRAM

So, my goal is to put in synchronization that just ensures that the two things -- flash access and LED output -- don't happen at the same time.

[ESP_igrr]

Are you working with Arduino IDE or ESP-IDF? If you are working with the latter, it is possible to place specific object files into IRAM using a linker fragment file.

For Arduino, maybe a way to work around the issue is to modify the linker script to place every object file named *-iram.o into IRAM. (Would need a change in FastLED and in Arduino-esp32).

[szguyer]

Most of our users are developing code in the Arduino IDE. And their level of programming expertise varies widely (many are beginners), so I would be reluctant to tell them to make changes to their dev environment.

Really, the right answer is to figure out how to get around the GCC compiler bug, so the interrupt handler can reside in IRAM. What's frustrating is that the bug was first report back in 2016 (the fact that section attributes are lost during template instantiation), so I'm not that hopeful about a fix.