WORKAROUND: ADC in continuous mode MISSING SAMPLES - 255 Sample LIMIT

jcolebaker
Posts: 64
Joined: Thu Mar 18, 2021 12:23 am

WORKAROUND: ADC in continuous mode MISSING SAMPLES - 255 Sample LIMIT

Postby jcolebaker » Mon Nov 27, 2023 2:28 am

Hi,

I've set up the ADC in continuous (DMA) mode to store samples to a 1024 byte buffer with 256 byte frame size. The sample rate is 40 kHz and the resolution is 12 bits. I'm feeding a 2.2v, 1 kHz sine wave to the input (input looks good on the scope).

I've set up the code with a task which is woken up for each frame (on_conv_done callback) and copies frame data to a static buffer until all frames have been copied. I use two static buffers, and the "main" task prints out 128 samples (256 bytes) from the other buffer then swaps the buffers (with mutex use to prevent contentions). This is a bit crude, but I just want to make sure the sampling is working correctly without any ring buffer overflows.

I have been pasting the 128 samples into Excel and plotting the sine wave to check what it looks like.

It looks like the ADC sampling is dropping or missing every 10th and 11th sample, i.e. 9 samples look OK, then there is a jump which looks like 2 samples are missing. The plot looks OK if I insert a 2 row gap every 9 rows. See attachments.

I know the ring buffer is getting cleared in time because I included a callback for "on_pool_ovf" which generates an execption, and it doesn't get called.

I have tried different sample rates but that doesn't change anything.
Here is my code (main.c):

Code: Select all

#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/timers.h"
#include "freertos/semphr.h"

#include "esp_log.h"
#include "esp_adc/adc_continuous.h"

#include <assert.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

static const char *TAG = "MAIN";

#define ADC_BUFFER_SIZE 1024
#define ADC_FRAME_SIZE  256

#define SEM_WAIT_TIME 50  // Ticks

// Use two buffers so we don't have to stop the ADC while printing out a frame:
uint8_t sample_data_uint8_a[ADC_FRAME_SIZE] = {0};
uint8_t sample_data_uint8_b[ADC_FRAME_SIZE] = {0};

uint8_t *p_store_buffer = sample_data_uint8_a;
uint8_t *p_print_buffer = sample_data_uint8_b;

static SemaphoreHandle_t l_buffer_mutex;

static TaskHandle_t l_store_task_handle;
static TaskHandle_t l_main_task_handle;

static adc_continuous_handle_t l_adc_handle = NULL;

#define ENSURE_TRUE(ACTION)           \
    do                                \
    {                                 \
        BaseType_t __res = (ACTION);  \
        assert(__res == pdTRUE);      \
        (void)__res;                  \
    } while (0)

static void adc_store_task(void *arg);


// =============== Conversion Done Callback: ========================================================
static bool IRAM_ATTR adc_conversion_done_cb(adc_continuous_handle_t handle, const adc_continuous_evt_data_t *edata, void *user_data)
{
    BaseType_t must_yield = pdFALSE;

    // Notify "store" task that a frame has finished.
    vTaskNotifyGiveFromISR(l_store_task_handle, &must_yield);

    return (must_yield == pdTRUE);
}


// ================ Ring buffer overflow callback: ==================================================
static bool IRAM_ATTR adc_pool_overflow_cb(adc_continuous_handle_t handle, const adc_continuous_evt_data_t *edata, void *user_data)
{
    // OVERFLOW! We don't expect this to happen, so generate exception.
    assert(false);

    return false; // Ssouldn't get to here.
}


// ========== Configure the ADC in continuous mode: ============
void adc_init(void)
{
    adc_continuous_handle_cfg_t continuous_handle_config =
    {
        .max_store_buf_size = ADC_BUFFER_SIZE,
        .conv_frame_size = ADC_FRAME_SIZE,
    };

    ESP_ERROR_CHECK( adc_continuous_new_handle(&continuous_handle_config, &l_adc_handle) );

    adc_digi_pattern_config_t pattern_config =
    {
        .unit = ADC_UNIT_1,
        .channel = ADC_CHANNEL_0,
        .bit_width = 12,
        .atten = ADC_ATTEN_DB_11,
    };

    adc_continuous_config_t adc_config =
    {
        .pattern_num = 1,
        .adc_pattern = &pattern_config,

        .sample_freq_hz = 60 * 1000,

        .conv_mode = ADC_CONV_SINGLE_UNIT_1,
        .format = ADC_DIGI_OUTPUT_FORMAT_TYPE1,
    };

    ESP_ERROR_CHECK( adc_continuous_config(l_adc_handle, &adc_config) );

    adc_continuous_evt_cbs_t adc_callbacks =
    {
        .on_conv_done = adc_conversion_done_cb,
        .on_pool_ovf = adc_pool_overflow_cb,
    };

    ESP_ERROR_CHECK( adc_continuous_register_event_callbacks(l_adc_handle, &adc_callbacks, NULL) );

    ESP_ERROR_CHECK( adc_continuous_start(l_adc_handle) );
}


// ADC store task:
// Woken up by the callback when a frame is finished.
// Read the frame data from the ADC and store to static buffer.
// This must be done frequently to prevent ring buffer overflow.
static void adc_store_task(void *arg)
{
    while (true)
    {
        ulTaskNotifyTake(pdTRUE, portMAX_DELAY);

        esp_err_t read_result;

        do
        {
            // Read all available frames from the ring buffer (probably only 1):
            ENSURE_TRUE( xSemaphoreTake(l_buffer_mutex, SEM_WAIT_TIME) );

            uint32_t bytes_read = 0;
            read_result = adc_continuous_read(l_adc_handle, p_store_buffer, ADC_FRAME_SIZE, &bytes_read, 0);

            ENSURE_TRUE( xSemaphoreGive(l_buffer_mutex) );

        } while (read_result == ESP_OK); // ESP_OK means more frames may be available

        // Result should be a timeout (no more frames available):
        assert(read_result == ESP_ERR_TIMEOUT);

        // Wake the main task to print out the latest frame:
        xTaskNotifyGive(l_main_task_handle);
    }
}


void app_main(void)
{
    ESP_LOGI(TAG, "Test ADC Continuous Mode");

    l_main_task_handle = xTaskGetCurrentTaskHandle();

    l_buffer_mutex = xSemaphoreCreateMutex();
    assert(l_buffer_mutex != NULL);

    const BaseType_t APP_CORE = 1;
    xTaskCreatePinnedToCore(adc_store_task, "ADC_Store", 4096, NULL, 6, &l_store_task_handle, APP_CORE);

    adc_init();

    // Main task: Print out the current "print" buffer, then swap buffers:
    while (true)
    {
        ulTaskNotifyTake(pdTRUE, portMAX_DELAY);

        size_t n_samples = ADC_FRAME_SIZE / sizeof(adc_digi_output_data_t);
        ESP_LOGI(TAG, "------- %zu samples available (%lu bytes) ------", n_samples, (uint32_t)ADC_FRAME_SIZE);

        // Sample data is actually an array of adc_digi_output_data_t structs:
        adc_digi_output_data_t *p_sample_data_adc = (adc_digi_output_data_t *)p_print_buffer;

        for (size_t i = 0; i < n_samples - 1; i += 2)
        {
            // Note sample order 2, 1, 3, 4, ... , N, N-1:
            adc_digi_output_data_t *p_sample_a = p_sample_data_adc + i + 1;
            adc_digi_output_data_t *p_sample_b = p_sample_data_adc + i;

            printf("%d\n", p_sample_a->type1.data);
            printf("%d\n", p_sample_b->type1.data);
        }

        // Swap the "store" buffer with the "print" buffer:
        ENSURE_TRUE( xSemaphoreTake(l_buffer_mutex, SEM_WAIT_TIME) );

        if (p_store_buffer == sample_data_uint8_a)
        {
            p_store_buffer = sample_data_uint8_b;
            p_print_buffer = sample_data_uint8_a;
        }
        else
        {
            p_store_buffer = sample_data_uint8_a;
            p_print_buffer = sample_data_uint8_b;
        }

        ENSURE_TRUE( xSemaphoreGive(l_buffer_mutex) );

        vTaskDelay(1); // Don't hog the CPU from this task!
    }
}
ESP32 (ESP32-WROVER-B module)
ESP-IDF 5.1.2

Why am I losing every 10th and 11th sample?

Sampled-Sine-Points-Missing.png
Sampled sine wave with jumps
Sampled-Sine-Points-Missing.png (31.07 KiB) Viewed 14020 times
Sampled-Sine-Points-Missing-Gaps.png
Sampled sine wave with 2 empty points added every 9 points
Sampled-Sine-Points-Missing-Gaps.png (17.91 KiB) Viewed 14020 times
Last edited by jcolebaker on Sun Jan 07, 2024 7:14 pm, edited 1 time in total.

jcolebaker
Posts: 64
Joined: Thu Mar 18, 2021 12:23 am

Re: ADC in continuous mode MISSING SAMPLES - 255 Sample LIMIT

Postby jcolebaker » Thu Nov 30, 2023 12:09 am

UPDATE:

I found this in esp32\include\hal\adc_ll.h:

Code: Select all

#define ADC_LL_DEFAULT_CONV_LIMIT_EN      1
#define ADC_LL_DEFAULT_CONV_LIMIT_NUM     10
These settings are used to set bits in register APB_SARADC_CTRL2_REG:

APB_SARADC_MAX_MEAS_NUM (8 bit)
APB_SARADC_MEAS_NUM_LIMIT (1 bit).

According to comments in the code, APB_SARADC_MEAS_NUM_LIMIT must be 1 for continuous mode (i.e. ADC measurement number limit ON) and ADC_LL_DEFAULT_CONV_LIMIT_NUM can be 0-255.

ADC_LL_DEFAULT_CONV_LIMIT_NUM could be configured in the old API (IDF 4) but is hard-coded to 10 in IDF 5 as shown above. Changing this value changes the point where samples are dropped, which means we can get at most 255 samples in continuous mode without a glitch!

With further searching, I found this PR with a long discussion thread:

https://github.com/espressif/esp-idf/pull/1991

Buried part way down is this discussion:
APB_SARADC_MAX_MEAS_NUM determines the max number of continuous sampling of the digital ACD.

e.g:
You set APB_SARADC_MAX_MEAS_NUM = 200. When you start the ADC, the digital ADC will start working according to your ADC mode table. After 200 samples, the hardware will restart the ADC and continue the sampling,but you will lose some samples during the restart. This is the hardware limitation.
thanks !!
@szmodz
Yes, there is no way to get continuous adc samples. This hardware issue can not be fixed by software.

Is this correct? As mentioned in the thread, this seems like critical info which is not mentioned in the technical reference manual or API documentation!!

(For context, we want to continuous sample ~2600 samples at 33,500 Samples/Sec (~80 mS), then carry out some analysis on the data after sampling is finished).

ESP_Sprite
Posts: 9766
Joined: Thu Nov 26, 2015 4:08 am

Re: ADC in continuous mode MISSING SAMPLES - 255 Sample LIMIT

Postby ESP_Sprite » Thu Nov 30, 2023 2:44 am

As it says a bit later, you can disable that sampling limit by setting SYSCON.saradc_ctrl2.meas_num_limit = 0; as far as I can read the hardware is capable of doing continuous sampling.

jcolebaker
Posts: 64
Joined: Thu Mar 18, 2021 12:23 am

Re: ADC in continuous mode MISSING SAMPLES - 255 Sample LIMIT

Postby jcolebaker » Fri Dec 01, 2023 2:58 am

I have tried setting this bit to 0 (by modifying the code in adc_ll.h), but then the ADC didn't work at all (at least, when using the ADC API, I did not receive the "on_conv_done" callback).

It may be worth experimenting some more. However, the TRM does say that the 0 value for this bit is reserved and it must be set to 1 for continuous mode.

jcolebaker
Posts: 64
Joined: Thu Mar 18, 2021 12:23 am

Re: ADC in continuous mode MISSING SAMPLES - 255 Sample LIMIT

Postby jcolebaker » Wed Dec 06, 2023 11:34 pm

Any update to this? Has anyone had success by setting "SYSCON.saradc_ctrl2.meas_num_limit = 0" as @ESP_Sprite suggested?

This did not work for me - the ADC "conversion done" callback doesn't occur if the bit is set to 0.

davidribeiro
Posts: 3
Joined: Thu Dec 14, 2023 3:36 pm

Re: ADC in continuous mode MISSING SAMPLES - 255 Sample LIMIT

Postby davidribeiro » Thu Dec 21, 2023 1:59 pm

I'm having the same problem. Tried your code, used SYSCON.saradc_ctrl2.meas_num_limit = 0 before calling adc_continuous_start(), and, although the ADC works if you change the register before initializing it, the missing samples persist.

Code: Select all

#include "soc/syscon_periph.h"

Code: Select all

void adc_init(void)
{
    adc_continuous_handle_cfg_t continuous_handle_config =
    {
        .max_store_buf_size = ADC_BUFFER_SIZE,
        .conv_frame_size = ADC_FRAME_SIZE,
    };

    ESP_ERROR_CHECK( adc_continuous_new_handle(&continuous_handle_config, &l_adc_handle) );

    adc_digi_pattern_config_t pattern_config =
    {
        .atten = ADC_ATTEN_DB_11,
        .channel = ADC_CHANNEL_0,
        .unit = ADC_UNIT_1,
        .bit_width = 12
    };

    adc_continuous_config_t adc_config =
    {
        .pattern_num = 1,
        .adc_pattern = &pattern_config,

        .sample_freq_hz = 60 * 1000,

        .conv_mode = ADC_CONV_SINGLE_UNIT_1,
        .format = ADC_DIGI_OUTPUT_FORMAT_TYPE1,
    };

    ESP_ERROR_CHECK( adc_continuous_config(l_adc_handle, &adc_config) );

    adc_continuous_evt_cbs_t adc_callbacks =
    {
        .on_conv_done = adc_conversion_done_cb,
        .on_pool_ovf = adc_pool_overflow_cb,
    };

    ESP_ERROR_CHECK( adc_continuous_register_event_callbacks(l_adc_handle, &adc_callbacks, NULL) );

    SYSCON.saradc_ctrl2.meas_num_limit = 0;
    ESP_ERROR_CHECK( adc_continuous_start(l_adc_handle) );
}

missing_points.png
missing_points.png (42.97 KiB) Viewed 13140 times

jcolebaker
Posts: 64
Joined: Thu Mar 18, 2021 12:23 am

Re: ADC in continuous mode MISSING SAMPLES - 255 Sample LIMIT

Postby jcolebaker » Sun Jan 07, 2024 7:12 pm

@davidribeiro Interesting result... I suspect SYSCON.saradc_ctrl2.meas_num_limit isn't implemented, which is why it is reserved (i.e. the ADC will always reset after N samples due to hardware limitations).

FIX / WORKAROUND:

I have found a hacky work-around which is working well for us so far:

* Modify esp-idf\components\hal\esp32\include\hal\adc_ll.h, and change "#define ADC_LL_DEFAULT_CONV_LIMIT_NUM" to 2 (was 10). This sets the "APB_SARADC_MAX_MEAS_NUM" register bits to 2, which translates to one ADC measurement before ADC reset.

* Set the ADC sample rate to approximately 3 x the sample rate you want.

* Tweak the sample rate until it's correct!

This solution relies on the ADC reset taking approximately 2 sample periods regardless of sample rate. I think it takes slightly more than 2 sample periods, hence you need to tweak the sample rate. By using 3 x the sample rate and resetting after each sample, you end up with regular samples at your desired sample rate. However, it obviously limits your maximum sampling rate! Also, it relies on undocumented black-box hardware features, so I don't know whether it will work consistently between different chip revisions or even on all chips from the same revision!

Instead of hacking the SDK, it is also possible to call:

Code: Select all

#include "hal/adc_ll.h"
...
adc_ll_digi_set_convert_limit_num(2);
Note, this must be called AFTER starting the ADC* (adc_continuous_start(...);) since the value is reset to 10 by the ADC startup code.
Last edited by jcolebaker on Mon Sep 16, 2024 8:40 pm, edited 1 time in total.

davidribeiro
Posts: 3
Joined: Thu Dec 14, 2023 3:36 pm

Re: WORKAROUND: ADC in continuous mode MISSING SAMPLES - 255 Sample LIMIT

Postby davidribeiro » Thu May 02, 2024 7:52 am

@jcolebaker Sorry for the late reply, I've been working on others parts of the project and haven't been programming ESPs lately. Thank you for the workaround! Haven't tested it yet, but I might try. At the time I converged to a solution using the one shot measurement to measure at 4kHz, since I was only going to use the extra samples for oversampling.

I would just make a small suggestion: I think there was a register (APB_SARADC_MAX_MEAS_NUM, page 657 of the reference manual) that could be used to replace your first step. Using it you would avoid making changes to the IDF libraries, at least I always avoid to do so for portability.

There was also some suggestions on a github issue thread that this problem might not be present on the ESP32-S3 (I think the issue exists out of the box, but SYSCON.saradc_ctrl2.meas_num_limit does work there). So, if that is a viable option for you, it might save you some trouble. I couldn't use it because it has no DAC and I didn't want the hassle of using an external one.

Great hack! I was so focused on solving the root problem I didn't even focus on the almost deterministic nature of the problem, with the (almost?) constant 2 missing readings. Best of luck for your projects!

Who is online

Users browsing this forum: Bing [Bot] and 139 guests