Unable to do simple audio transformations

SlimyRedstone
Posts: 9
Joined: Mon Oct 09, 2023 8:31 pm

Unable to do simple audio transformations

Postby SlimyRedstone » Mon Oct 09, 2023 9:55 pm

I'm trying to do a I2S microphone + headphone example program with basic audio manipulation (eg: gain, delay, etc...)
I have successfully done the simple part that gets the data from the microphone (using the INMP441, 24bits @ 48kHz), and sends it to the headphones (using the UDA1334, 24bits @ 48kHz).
I'm using ESP-IDFv5.1.1 in VSCode, on the ESP32-S3 N32R8

The I2S parameters are :
(this is generated on startup)
Polling interval: 1000ms
Buffer size: 1920 bytes
DMA frame number: 240 bytes
DMA description number: 9 bytes

--- Input ---
Bits per sample: 24
Frame length: 32
Sample rate: 48kHz
Channels: Stereo
Bit shift: Yes
Left align: Yes

--- Output ---
Bits per sample: 24
Frame length: 32
Sample rate: 48kHz
Channels: Stereo
Bit shift: No
Left align: No

However, I don't know how to change the gain. I've done some research and found that, to change the gain of a signal, you have to multiply the sampled signal by the gain.
When I do it, I only get noise on the output.
When I don't try to modify the gain, the output is acting normal, I can hear myself talk into the mic.

Here is a "simplified" version of the whole code, I only removed methods & variables not used in this part (tell me if you need more but this is really the part responsible for the I2S data)
Note: I also tried using the example given by Espressif, although in the example the I2S is 16bits, but got the same result

  1. #define CONVERT8bits(n,o) (uint8_t)((n>>8*o)&0xFF)
  2.  
  3. double gain = 1.5;
  4.  
  5. void setAudioGain(uint8_t *buffer, size_t bLength, double gain) {
  6.     uint32_t *sample = calloc(2,sizeof(uint32_t)); // L : R
  7.     for (uint32_t sIndex = 0; sIndex < bLength; sIndex += 6 * 4) {
  8.         for (uint8_t j=0;j<6*4;j+=6){
  9.         // Here, Left = 0 and Right = 1
  10.             sample[Left]  = buffer[sIndex + j + 0 ]<<16 | buffer[sIndex + j + 1 ]<<8 | buffer[sIndex + j + 2 ];
  11.             sample[Right] = buffer[sIndex + j + 3 ]<<16 | buffer[sIndex + j + 4 ]<<8 | buffer[sIndex + j + 5 ];
  12.  
  13.             sample[Left]  = (uint32_t)((double)sample[Left]*gain);
  14.             sample[Right] = (uint32_t)((double)sample[Right]*gain);
  15.  
  16.             *((uint8_t *)(buffer + sIndex + j + 0 )) = CONVERT8bits(sample[Left],2);  //  Left Channel MSB
  17.             *((uint8_t *)(buffer + sIndex + j + 1 )) = CONVERT8bits(sample[Left],1);  //  Left Channel
  18.             *((uint8_t *)(buffer + sIndex + j + 2 )) = CONVERT8bits(sample[Left],0);  //  Left Channel LSB
  19.  
  20.             *((uint8_t *)(buffer + sIndex + j + 3 )) = CONVERT8bits(sample[Right],2);  //  Right Channel MSB
  21.             *((uint8_t *)(buffer + sIndex + j + 4 )) = CONVERT8bits(sample[Right],1);  //  Right Channel
  22.             *((uint8_t *)(buffer + sIndex + j + 5 )) = CONVERT8bits(sample[Right],0);  //  Right Channel LSB
  23.         }
  24.     }
  25.     free(sample);
  26. }
  27.  
  28.  
  29. static void audio_task(void *args) {
  30.     uint8_t *data = calloc(BUFFER_SIZE,sizeof(uint8_t));
  31.     audio_init();
  32.     while (true) {
  33.         i2s_channel_read(i2s_rx, data, BUFFER_SIZE, &n_bytes_read, 10e3); // 10 sec max
  34.         setAudioGain(data, BUFFER_SIZE, 1.1); // If I comment this, the sound is normal
  35.         i2s_channel_write(i2s_tx, data, BUFFER_SIZE, &n_bytes_write, 10e3); // 10 sec max
  36.     }
  37.     free(data);
  38.     vTaskDelete(NULL);
  39. }

ESP_Sprite
Posts: 9772
Joined: Thu Nov 26, 2015 4:08 am

Re: Unable to do simple audio transformations

Postby ESP_Sprite » Tue Oct 10, 2023 4:24 am

If anything, your code expect your samples to be big-endian. Is that true?

SlimyRedstone
Posts: 9
Joined: Mon Oct 09, 2023 8:31 pm

Re: Unable to do simple audio transformations

Postby SlimyRedstone » Tue Oct 10, 2023 7:10 am

Both I2S port are configured as little endian (or big endian to false) through this parameter
Do you think this is the source of the problem ?
https://docs.espressif.com/projects/esp ... ig_endianE

(extract from the code)
  1. i2s_std_slot_config_t tx_slot_config = {
  2.     .data_bit_width = I2S_SPK_BITS_PER_SAMPLE,
  3.     .slot_bit_width = I2S_SPK_BITS_PER_CHANNEL,
  4.     .slot_mode = I2S_SLOT_MODE_STEREO,
  5.     .slot_mask = I2S_STD_SLOT_BOTH,
  6.     .ws_width = I2S_SPK_BITS_PER_CHANNEL,
  7.     .ws_pol = true,
  8.     .bit_shift = false,
  9.     .left_align = false,
  10.     .big_endian = false,
  11.     .bit_order_lsb = false,
  12. };
  13. i2s_std_slot_config_t rx_slot_config = {
  14.     .data_bit_width = I2S_MIC_BITS_PER_SAMPLE,
  15.     .slot_bit_width = I2S_MIC_BITS_PER_CHANNEL,
  16.     .slot_mode = I2S_SLOT_MODE_STEREO,
  17.     .slot_mask = I2S_STD_SLOT_BOTH,
  18.     .ws_width = I2S_MIC_BITS_PER_CHANNEL,
  19.     .ws_pol = false,
  20.     .bit_shift = true,
  21.     .left_align = true,
  22.     .big_endian = false,
  23.     .bit_order_lsb = false,
  24. };

SlimyRedstone
Posts: 9
Joined: Mon Oct 09, 2023 8:31 pm

Re: Unable to do simple audio transformations

Postby SlimyRedstone » Tue Oct 10, 2023 5:40 pm

I tried to set manually the sample into little endian and in big endian, still noise.
  1. // @return Returns the first 24 bits
  2. #define as24bit(n) (uint32_t)(n&0xFFFFFF)
  3.  
  4. #define CONVERT32bits(b,o) (uint32_t)(b[o]<<16 | b[o+1]<<8 | b[o+2])
  5. /*
  6.   @param n The number to convert
  7.   @param o Byte offset (0:Right, 1:Middle, 2:Left)
  8.   @return Returns the nth byte from a variable (from right to left)
  9. */
  10. #define CONVERT8bits(n,o) (uint8_t)((n>>8*o)&0xFF)
  11.  
  12. /*
  13.   @return Transform 0x11223344 into 0x44332211
  14. */
  15. #define LEtoBE(n) (uint32_t)( CONVERT8bits(n,0)<<24 | CONVERT8bits(n,1)<<16 | CONVERT8bits(n,2)<<8 | CONVERT8bits(n,3) )
  16.  
  17.  
  18. sample[Left] = as24bit( LEtoBE(sample[Left]) );
  19. sample[Right] = as24bit( LEtoBE(sample[Right]));

MicroController
Posts: 1749
Joined: Mon Oct 17, 2022 7:38 pm
Location: Europe, Germany

Re: Unable to do simple audio transformations

Postby MicroController » Tue Oct 10, 2023 10:26 pm

Plus, the samples are signed, two's complement values; so you have to sign-extend them to int32_t before multiplying.

SlimyRedstone
Posts: 9
Joined: Mon Oct 09, 2023 8:31 pm

Re: Unable to do simple audio transformations

Postby SlimyRedstone » Wed Oct 11, 2023 12:34 am

MicroController wrote:
Tue Oct 10, 2023 10:26 pm
Plus, the samples are signed, two's complement values; so you have to sign-extend them to int32_t before multiplying.
I also tried converting the values into signed int32_t, either by bit shifting or by map() (to get a more accurate conversion, I guess).
Nothing works. I don't know why, because when I log (printf the variable) every line, before and after each manipulation, it outputs coherent data.
(Before treatment)
sample[Left] = 0X831700 (8591104)
sample[Right] = 0X831700 (8591104)

(Applying the gain: 1.1, should slightly amplify the signal)

(After treatment)
sample[Left] = 0X9032E6 (9450214)
sample[Right] = 0X9032E6 (9450214)


(Putting uint32_t into 3 uint8_t *)
[L] buffer[0] = 0X90 (144)
[L] buffer[1] = 0X32 (50)
[L] buffer[2] = 0XE6 (230)

(Putting uint32_t into 3 uint8_t *)
[R] buffer[3] = 0X90 (144)
[R] buffer[4] = 0X32 (50)
[R] buffer[5] = 0XE6 (230)

ESP_Sprite
Posts: 9772
Joined: Thu Nov 26, 2015 4:08 am

Re: Unable to do simple audio transformations

Postby ESP_Sprite » Wed Oct 11, 2023 3:45 am

Erm... you sure the I2S doesn't require 24 bits of sample data in 32-bits variables?

SlimyRedstone
Posts: 9
Joined: Mon Oct 09, 2023 8:31 pm

Re: Unable to do simple audio transformations

Postby SlimyRedstone » Wed Oct 11, 2023 11:11 am

ESP_Sprite wrote:
Wed Oct 11, 2023 3:45 am
Erm... you sure the I2S doesn't require 24 bits of sample data in 32-bits variables?
Do you mean using 1 u32bit variable instead of a 3 u8bit ? I already tried that as well
In the datasheet of the UDA1334 (page 10 https://www.nxp.com/docs/en/data-sheet/ ... df#page=10),
in 24 bit mode, the word select signal lasts (expected to be a minimum 24 bit clock cycle, maximum 32 bit clock cycle, see page 9 #8.6.1) 32 bit clock cycle, where the 24 last are the actual I2S data (MSB on the left).
I've just tried setting 24 bit frame instead of 32, still the same.

MicroController
Posts: 1749
Joined: Mon Oct 17, 2022 7:38 pm
Location: Europe, Germany

Re: Unable to do simple audio transformations

Postby MicroController » Wed Oct 11, 2023 6:48 pm

(Before treatment)
sample[Left] = 0X831700 (8591104)
That would be a large negative value (-8186112), very close to the maximum amplitude representable in 24bits (-8388608).
Double-check the byte order in the buffer.
And make sure you operate on the signed values. (int32_t signed32 = ((int32_t)(sample24 << 8)) >> 8)

SlimyRedstone
Posts: 9
Joined: Mon Oct 09, 2023 8:31 pm

Re: Unable to do simple audio transformations

Postby SlimyRedstone » Thu Oct 12, 2023 7:17 pm

MicroController wrote:
Wed Oct 11, 2023 6:48 pm
(Before treatment)
sample[Left] = 0X831700 (8591104)
That would be a large negative value (-8186112), very close to the maximum amplitude representable in 24bits (-8388608).
Double-check the byte order in the buffer.
And make sure you operate on the signed values. (int32_t signed32 = ((int32_t)(sample24 << 8)) >> 8)
Can you write me a basic code that would do what you saying ? I'm pretty sure I've already tried that but I can't get my head around it.
This is what I would write (with your addition):
(Note: I've used definitions to ease its comprehension, but if you don't recommend using #define, I will not use it)
  1. /*
  2.   @param n The number to convert
  3.   @param o Byte offset (w/ 32bit -> 0:Right most, 3:Left most)
  4.   @return Returns the nth byte from a variable (from right to left)
  5. */
  6. #define CONVERT8bits(n,o) (uint8_t)((n>>8*o)&0xFF)
  7.  
  8. #define CONVERT32bits(b,o) (int32_t)(b[o]<<16 | b[o+1]<<8 | b[o+2])
  9.  
  10. void setAudioGain(uint8_t *buf, size_t bLength, double gain) {
  11.   int32_t *sample = calloc(2,sizeof(int32_t)); // L : R
  12.   for (uint32_t sIndex = 0; sIndex < bLength; sIndex += 6 * 4) {
  13.     for (uint8_t j=0;j<6*4;j+=6) {
  14.         sample[Left]  = CONVERT32bits(buf, j + 0 );
  15.         sample[Right] = CONVERT32bits(buf, j + 3 );
  16.  
  17.         sample[Left] = ((int32_t)(sample[Left]<<8))>>8;
  18.         sample[Right] = ((int32_t)(sample[Right]<<8))>>8;
  19.  
  20.         sample[Left]  = ((double)sample[Left]*gain);
  21.         sample[Right] = ((double)sample[Right]*gain);
  22.  
  23.       *((uint8_t *)(buf + sIndex + j + 0 )) = CONVERT8bits(sample[Left],2);   //  Left Channel MSB
  24.       *((uint8_t *)(buf + sIndex + j + 1 )) = CONVERT8bits(sample[Left],1);   //  Left Channel
  25.       *((uint8_t *)(buf + sIndex + j + 2 )) = CONVERT8bits(sample[Left],0);   //  Left Channel LSB
  26.  
  27.       *((uint8_t *)(buf + sIndex + j + 3 )) = CONVERT8bits(sample[Right],2);  //  Right Channel MSB
  28.       *((uint8_t *)(buf + sIndex + j + 4 )) = CONVERT8bits(sample[Right],1);  //  Right Channel
  29.       *((uint8_t *)(buf + sIndex + j + 5 )) = CONVERT8bits(sample[Right],0);  //  Right Channel LSB
  30.     }
  31.   }
  32.   free(sample);
  33. }

Who is online

Users browsing this forum: No registered users and 64 guests