I2S sound recorder using only ULP - Possible?

[EDsteve]

Hello,

I hope i am in the correct forum for my question.
For our elephant-sound-recorders we try to build a recorder which can run 24/7 on a small solar panel. Basically a ESP32 with a digital microphone and a connected SD card.
We already built a few working recorders, which work great but the solar panels in the jungle can't keep up if the main processor is doing the work (Already reduced to 80MHz). I am no expert but as far as i understand the ULP should be sufficient to "route" the sound from the I2S microphone over memory to the SD card and write a wav file, right? So we would be able to record in sleep mode. And then the solar panel would be enough even in the shade.

The ULP runs with 8MHz which should be more than enough to deal with i2s signals even for sample rates higher than 44KHz. Has someone done that already?

I have a few questions for that approach:
- Will the sound quality be affected in any way by using ULP?
- If it is possible in theory. Can it be done "easily"?
- So if i find someone who can do that on freelancer.com for example. Would a few hundred dollars be enough?- I'm hoping to get a max.power consumption of 5mA. Realistic?
- Or would it be easier to just add an external clock for recording instead of using the ULP?

Why do we use the ESP32? We still need to wake up the main cpu a few times per day for more CPU hungry tasks. At the end this "recorder" should become a elephant detector.

Wish I could do it myself but my hobbyist knowledge is not enough unfortunately. That' why i am asking here first before i try to hire someone on freelancer.com
As a non profit organization we have limited resources if it comes to money. Doesn't make things easy. But to use the ULP for recording would bring us a big step closer to reach our aim which is a solution for human-elephant conflict. At the end your help will save elephants (and humans).


Thanks for taking your time to read this post. Any advice is appreciated. 

If interested. You can find more information about our work on the open source elephant detector at https://wildlifebug.com 

[boarchuz]

The ULP cannot access I2S or SDMMC peripherals, so you would need to do this all in software. This is not a trivial task, if even possible given the limited speed and memory of the ULP.

Light sleep might be an option, given you have 5mA to work with. I'm not familiar with I2S but if it can operate in light sleep and fire an interrupt each time the buffer is ready to be dumped to SD, then that would be a much easier solution.

Do you need to be recording 24/7? Could you incorporate a part like the VM3011 (microphone with noise detection interrupt @ 10uA) to wake the ESP32 only when necessary? You will miss up to 50ms of audio as the ESP32 wakes up from deep sleep, or <1ms from light sleep.

[EDsteve]

I always thought that even 3Mhz should be enough to deal with I2S and sample rates of >44Khz. So you think that the 8Mhz from the ULP is not fast enough the task?

This VM3011 microphone looks very interesting for our elephant detector. Thanks for that link. Awesome.

For now we need to get the 24/7 recorder working which should be able to record 24/7.
Thanks for the info. Light sleep seems to be the way to go. The question is how to keep I2S active during light sleep mode. So that would be the next thing to find out.

If i am not mistaken he kept DAC alive while using the ULP instead of the main CPU. Would that work similar with I2S?
https://www.youtube.com/watch?v=6PGrsZmYAJ0

Hmm. I am still a bit confused about it.

[boarchuz]

I always thought that even 3Mhz should be enough to deal with I2S and sample rates of >44Khz. So you think that the 8Mhz from the ULP is not fast enough the task?

Bear in mind that instructions may take up to 12 cycles. Quick back of the napkin math says you can forget about 44kHz. Again, I haven't had much experience with audio, but if I'm not wrong, 8 bit mono at 44kHz would require 352,000 bits per second. Consider a very minimal bit read:
eg.

Code: Select all

clock_pin_low // 12 cycles
clock_pin_high // 12 cycles
read_data_pin // 8 cycles

That's already down to a maximum 250,000 bits per second (32 cycles @ 8MHz) without any loop overhead, pushing to memory, writing to SD, filtering or event detection, etc. Maybe the newer RISC-V ULPs change the picture? Maybe some other peripheral (eg. I2C) could help out the ULP? Maybe PDM would be easier? Maybe it's possible to get a great result. Regardless, the main takeaway is to not underestimate how challenging or time-consuming it could be to answer these questions.

If i am not mistaken he kept DAC alive while using the ULP instead of the main CPU. Would that work similar with I2S?

The ULP can only access a limited range of registers. This happens to include the registers to control the DAC, but not for I2S or SD.

[EDsteve]

Thanks for your reply.
If i understand you correctly. I should give up on the idea recording sound on a ESP32 with a power consumption of less than 5mA?
It's possible but it seems a few hundred bucks will not be sufficient to achieve my goal.

Hmm. Sad to hear. Still. If anybody has a solution or a different approach. I am all yours.
And if you can make it work. The few hundred bucks will be all yours too.

[boarchuz]

I should give up on the idea recording sound on a ESP32 with a power consumption of less than 5mA?

Did you rule out light sleep?

[EDsteve]

Did you rule out light sleep?

I kind of did because i thought that peripherals are OFF during light sleep mode as well. But thanks for asking again. That made me do some reading and I2S might be working during light sleep.
Still not really sure about this because - as you already know - I am no expert in that field and there are just too many confusing clocks involved.

In the I2S programming guide i found this sentence:
https://docs.espressif.com/projects/esp ... s/i2s.html
"When the power management is enabled (i.e. CONFIG_PM_ENABLE is on), the system will adjust or stop the source clock of I2S before going into light sleep, thus potentially changing the I2S signals and leading to transmitting or receiving invalid data."

Specially the word "adjust or stop the source clock". This tells me that I2S can still operate during light sleep. Using another clock. APLL or PLL i guess. So if I2S is able to be fully functional during light sleep mode. This should be possible:

- In light sleep mode the I2S interface writes data from the I2S microphone to a buffer (I hope that DMA is accessible)
- A timer wakes up the main CPU when the buffer is full which writes the data to the SD card
- I2S and the main CPU can't have access to the same buffer at the same time. So once the timer triggered. I2S starts writing data to a second buffer
- When the first buffer is written to SD card. The CPU will go to sleep again waiting for the next time trigger.

Is that doable? Not sure if i understand it all correctly.


One negative side effect for deep or light sleep might be the RTC which has a strong drift (+/- 5%). Too big time drift for our purpose. But a external clock should solve that issue though.

[timstercb]

@EDsteve - very curious if you were ever able to get an answer to whether I2S -> DMA is possible during Light Sleep?

I have a bioacoustic application as well - and am keen to extend battery life as much as possible.

Thanks!
-Tim