Timing-sensitive peripheral access - trying to start two peripherals with deterministic delay

[andyn_ff]

Hi,

I am using a device which has a serial data input (uses a shift register internally), and a latch signal. I am using the ESP32 SPI controller to transmit the data and clock to the device. The latch signal is required to go high during the final 3 clock pulses of the SPI transmission (the width of the pulse is used by the device to communicate different commands, so it must be exactly 3 clock pulses). Therefore I cannot use the normal CS signal to control the latch.

My current design for controlling this is to use the LEDC controller to generate the latch pulse at the right time, and start the SPI transmission and LEDC timer one line after another in the code. It doesn't matter that the SPI and LEDC don't start at exactly the right time, because I can time how long the time takes in-between lines of code and include that as an offset in the timing setup of the LEDC controller. The transmission starts within an interrupt handler, with priority level 3.

However, there has to be a deterministic time between starting the SPI transmission and starting the LEDC timer. I hoped to achieve this by putting these lines within a critical section so that no higher-priority interrupts on this core would happen in-between the execution of these two lines - but it isn't working! For some transmissions, the LEDC starts on-time, but the SPI starts a little later than usual, so the latch pulse (caused by LEDC) is too early.

I have managed to reduce the variation in timing between starting the SPI and LEDC, by putting the interrupt handler and all data accessed into IRAM / DRAM. I guess the flash and its cache were causing some of the occasional delays - perhaps from an interrupt running on the other core.

There is still a small extra delay to starting the SPI sometimes. At the same time, I have another FreeRTOS task running which triggers an ADC capture from the other core every 10ms, and I think this is what is causing the small delay sometimes. If I stop this task, or if I pin it to the same core as the SPI/LEDC, I get rock-solid timing consistently.

My guess with what is happening here is that starting the ADC capture and starting the SPI both involve setting peripheral registers, and they cannot happen at exactly the same time, so one has to wait until the other one is set successfully.

My question is will this be the case with any peripheral register access, or just certain peripherals (is it anything special about shared a shared bus between ADC / SPI / LEDC, perhaps?). I guess there is one APB bus which all peripherals share, because I cannot see any distinction further than this in the reference manual?

If it is any peripherals, I don't think I can control the system enough to have all the peripheral access happen from one core (to protect the critical timing between SPI and LEDC), so I may have to stall core 1 while I start SPI and LEDC on core 0, to make sure the timing is deterministic.

I am also curious if this small delay would only happen if the peripheral registers are accessed from both cores at the same time, or if the same effect could happen from the DMA accessing registers at the same time as a core?

Thanks for your support!
Andy

[ESP_Sprite]

What causes the delay can mostly be inferred from what the 'small' delay is. The APB clock runs at 80 MHz, and assuming that an APB access takes at most a handful of cycles, any delay of up to, say, half an uS could be caused by hardware things. More than that and you're looking at a software issue.

Starting two peripherals at exactly the same time is tricky without going deep down into the registers and tweaking the bits that start the peripherals at only a few clock cycles apart... I don't really see a way to do that without basically writing your own driver.

What particular device are you trying to control? Perhaps there's another way to solve this.

[andyn_ff]

Thanks for your reply. It is an LED driver, several similar devices daisy-chained. I need to run SPI at 1MHz clock in order to transfer data across quickly enough through the whole daisy-chain of devices. Therefore, half a uS is enough to move the latch pulse into the wrong clock cycle. So I think we are seeing a hardware cause from APB access.

The signals required for the device are shown in the diagram:


There is also a signal to reset the device's internal shift register which we do not use at the moment.

If there is some other way to use ESP32 features to provide these signals, I am happy to try out another idea. I did think about using the ULP with the RTC GPIO pins: I think with the 8MHz clock to ULP it will not be fast enough to bit-bang the whole SPI, but I thought about using the ULP to count the SPI clock pulses and create the latch pulse at the right time. But could ULP's access to set the GPIO high or low be delayed by APB access in the same way as the ADC and LEDC delayed each other before?

[ESP_Sprite]

If the latch enable is only something that needs to be controlled during the last 3 clocks and the SPI transmission itself is not particularily timing sensitive (as in: the clocks can be any length), why not use SPI to send all the data minus the last three, then enable the latch enable, send out the last three bits either using SPI or by simply bit-banging them, and then disable latch enable again?

[andyn_ff]

Thanks for the suggestion. I think I can try that with bit-banging. I assume the ESP32 SPI must transmit whole bytes, not just an arbitrary number of bits, so I guess I would need to bit-bang the full last byte of the transaction, with the latch low for the first 5 bits and then high for the last 3. Is that right, or can I use the SPI peripheral to do n bytes + 5 bits, then set latch enable, then use SPI peripheral to do 3 bits?

I would like to understand the original problem with the APB bus delay a bit more too:

1 - Would DMA transferring data to/from another SPI or UART cause the delay accessing APB bus in the same way as if the two cores tried to set a peripheral register at the same time as each other? I am interested in this because it could mean that stalling the other core during this critical section will not work.

2 - Could I hit the same problem if I tried to use ULP to create the latch enable pulse, with SPI CLK as input on RTC GPIO read by ULP (counting clock pulses and setting latch enable at the right time)? If core 0 set a peripheral register at the same time as ULP tried to set a RTC GPIO state, could setting the RTC GPIO be delayed by APB bus?

Thanks,
Andy

[ESP_Sprite]

Thanks for the suggestion. I think I can try that with bit-banging. I assume the ESP32 SPI must transmit whole bytes, not just an arbitrary number of bits, so I guess I would need to bit-bang the full last byte of the transaction, with the latch low for the first 5 bits and then high for the last 3. Is that right, or can I use the SPI peripheral to do n bytes + 5 bits, then set latch enable, then use SPI peripheral to do 3 bits?

No, it actually has per-bit granularity; you could transmit everything but the last 3 bits.

1 - Would DMA transferring data to/from another SPI or UART cause the delay accessing APB bus in the same way as if the two cores tried to set a peripheral register at the same time as each other? I am interested in this because it could mean that stalling the other core during this critical section will not work.

No, I think the DMA transfers go over a separate bus. If I recall correctly, APB is only for reads/writes from the CPU to the peripheral registers.

2 - Could I hit the same problem if I tried to use ULP to create the latch enable pulse, with SPI CLK as input on RTC GPIO read by ULP (counting clock pulses and setting latch enable at the right time)? If core 0 set a peripheral register at the same time as ULP tried to set a RTC GPIO state, could setting the RTC GPIO be delayed by APB bus?

I don't think so, but keep in mind that the ULP runs at least 10 times slower than the APB bus; any jitter caused by the APB bus probably is about as large as the jitter caused by the slow-running ULP.

Note that you still assume the APB bus is to blame here, which I find somewhat unlikely, honestly...e.g. if your current code is using an (C-code) interrupt to set the latch signal, the latency and jitter in the code to take that interrupt is likely to be way larger than whatever delay the APB bus may give you.

[andyn_ff]

Thank you for your replies, that's really helpful. I will experiment with splitting my SPI transmission into two sections so that the first one stops just before I need to set the latch enable high.

The reason I thought this is a hardware limitation instead of code is because I have related it to ADC reads being triggered on the other core. I.e. if I increase the frequency of ADC reads, I get much more frequent jitter (of the same magnitude) in the SPI + LEDC start operation. If I stop the ADC reads entirely, or if I pin them to the same core as the SPI + LEDC, there is no jitter.

Do you think it still may be wrong to suspect the hardware?

Either way, the best solution is to avoid having to align these two peripherals in time if I possibly can - so I will try splitting up the SPI transmission now.

[ESP_Sprite]

Do you think it still may be wrong to suspect the hardware?

Possibly. I don't know the ADC code that well - if it's a blocking driver, there are interrupts involved and that could very well cause the jitter (as in: the CPU can't service two interrupts at the same time, so one needs to wait)