EMAC and I2S interfering with each other?

MikeNaylor
Posts: 2
Joined: Mon Nov 11, 2024 10:30 pm

EMAC and I2S interfering with each other?

Postby MikeNaylor » Mon Nov 11, 2024 11:17 pm

I'm building a firmware for the ESP32 (specifically ESP32-WROOM-32D) to drive WS2812 LEDs (and similar).

I have a board on which the ESP32 is hooked up to a LAN8720 via RMII (external 50Mhz clock on GPIO0), and generally, all is well.

ESP-IDF version is v4.4.7 at the moment.

The board receives pixel data via ethernet and pushes it out of the I2S peripheral in LCD mode (much like the FastLED project). The I2S peripheral is supplied with data via DMA buffers that are cycled/refreshed on EOF interrupts. There are currently 3 DMA buffers in play (1 being drained, 1 full, 1 being replenished), so there is a certain amount of jitter buffering available.

The problem occurs when the board is dealing with a lot of pixels. The limit for the WS2812 protocol is roughly 800 pixels at 40fps, but we started seeing artefacts in the pixel data stream once we got to about 650-700px. It turns out that the data source starts transmitting the next frames-worth of pixel data at that point.

For a while, I assumed that the CPU was getting overloaded with receiving and processing the pixel data, but a deeper dive shows this isn't the case. I instrumented some functions (to toggle unused GPIO) and proved that the data stream corruption was occurring shortly before we were processing the incoming network packets. Instrumenting some functions within the emac driver reveals that the data corruption occurs just before the emac interrupt (emac_isr_default_handler) is fired.

As far as I'm aware, the 2 interrupts are executing on separate CPU cores (emac on CPU0, I2S on CPU1). Happy to check and confirm this if it helps, though.

This seems to imply that it's the PHY transferring the data to the EMAC that's causing the glitch. I know that this transfer is performed via DMA, so is there any chance that the transfer is interfering with the transfer of data to the I2S peripheral?

I noticed that receiving lots of small packets seemed to cause less problems than a few larger packets. I guessed that maybe the transfer was less disruptive if broken into smaller pieces, so I adjusted CONFIG_ETH_DMA_BUFFER_SIZE to 256. This does seem to resolve the issue with larger packets, but now I'm having issues with packet loss when receiving lots of small packets, so this isn't a great solution for me. In the final version of this, I need to be able to handle both scenarios for different incoming protocols. I have no control of the source of the ethernet data.

I have attached an image showing what I'm seeing - It's a screengrab from PulseView, as follows:
  • The first line shows the emac interrupt activity - The GPIO calls are made at the start and end of the emac_isr_default_handler function in esp_eth_mac_esp.c
  • The second line shows the emac_rx task activity - The GPIO calls are made in the function emac_esp32_rx_task. The line goes high just after the call to "ulTaskNotifyTake(pdTRUE, portMAX_DELAY)", and low after the while loop ( outside "} while (emac->frames_remain);")
  • The third line is also in emac_esp32_rx_task, but the GPIO calls are wrapped around the call to emac_hal_receive_frame.
  • The fourth line is the activity of the i2s interrupt. There is a little jitter on the interrupt behaviour, but nothing that should cause data stream issues.
  • The fifth line is the WS2812 data stream, and below it is the PulseView decode. You can see I've added 2 arrows where the bitstream is weirdly elongated just before the emac interrupt fires - this data stream should be repeating perfectly at even intervals. The data stream content is intentionally all zeroes (no LED output), to make it easier to see the erroneous flashing. While PulseView still successfully decodes the data, these weirdly long bits corrupt the data stream for physical LED chips.
So - Is there any reason this should be the case, and is there anything I can do to the hardware config to prevent it?

(And apologies if none of this makes any sense - More than happy to answer questions, etc :) )
Attachments
pulseview.png
pulseview.png (76.77 KiB) Viewed 737 times

ESP_ondrej
Posts: 211
Joined: Fri May 07, 2021 10:35 am

Re: EMAC and I2S interfering with each other?

Postby ESP_ondrej » Tue Nov 12, 2024 8:17 am

We've already observed that high traffic on Ethernet can also affect the SPI DMA transactions. The same seems to apply for I2S. Try to reduce `dma_burst_len` when initializing Ethernet with `eth_esp32_emac_config_t` config structure.

MikeNaylor
Posts: 2
Joined: Mon Nov 11, 2024 10:30 pm

Re: EMAC and I2S interfering with each other?

Postby MikeNaylor » Tue Nov 12, 2024 9:49 am

And that's fixed it - you're a star :)

That param/struct isn't present in 4.4.7, so I went poking through the driver code and hacked it in manually for the moment. However, adjusting it to 16 bits (with large DMA buffers in place) seems to have resolved the problem I was having.

Got to go learn how to patch out stuff in the IDF distribution now, but that feels easier than the rollercoaster I've been on for the last few weeks :)

Thanks again :)

ESP_ondrej
Posts: 211
Joined: Fri May 07, 2021 10:35 am

Re: EMAC and I2S interfering with each other?

Postby ESP_ondrej » Wed Nov 13, 2024 7:18 am

Great to see the issue has been fixed!

You were really close to the solution based on your detailed issue description. Hence it implies to me there must be something wrong with documentation. Would you have any suggestion how it could be improved? What would you help during your investigation?

Who is online

Users browsing this forum: JK Sleepling and 54 guests