Heap corrupting when using xQueue

ESP_Angus
Posts: 2344
Joined: Sun May 08, 2016 4:11 am

Re: Heap corrupting when using xQueue

Postby ESP_Angus » Wed Jun 20, 2018 11:28 pm

IDF-Monitor can also help you decode addresses in stack traces, if you use it to access the console:
https://docs.espressif.com/projects/esp ... nitor.html

Or you can run xtensa-esp32-elf-addr2line manually (the command line to use is written at the above page.)

x-8973
Posts: 20
Joined: Tue Apr 03, 2018 4:49 pm

Re: Heap corrupting when using xQueue

Postby x-8973 » Fri Jun 22, 2018 3:34 pm

Based on the results of the experiments, I determined that damage to the heap causes any malloc() call inside the RTOS task. After that, the application's operation becomes unstable, and any access to the heap causes crash or inadequate behavior.
I found a description of thread-safe implementations, pvPortMalloc(), but in my IDF they are described as follows:

Code: Select all

#define pvPortMalloc malloc
#define pvPortFree free

x-8973
Posts: 20
Joined: Tue Apr 03, 2018 4:49 pm

Re: Heap corrupting when using xQueue

Postby x-8973 » Sat Jun 23, 2018 12:01 pm

I think the problem is solved. I added a file named heap_3.c to the project, which was taken from the full release of FreeRTOS, and replaced the definitions in the portable.h.

It was:

Code: Select all

#define pvPortMalloc malloc
#define vPortFree free
Became:

Code: Select all

void *pvPortMalloc( size_t xWantedSize );	// In heap_3.c
void vPortFree( void *pv );				// In heap_3.c

WiFive
Posts: 3529
Joined: Tue Dec 01, 2015 7:35 am

Re: Heap corrupting when using xQueue

Postby WiFive » Sat Jun 23, 2018 8:36 pm

That doesn't seem reasonable.

x-8973
Posts: 20
Joined: Tue Apr 03, 2018 4:49 pm

Re: Heap corrupting when using xQueue

Postby x-8973 » Sun Jun 24, 2018 6:21 am

Why do you think so?
I did not find a clear explanation of why the dynamic allocation of memory inside the RTOS task can mess up the heap. Perhaps at this point the system switches to another task, and the heap allocation table is corrupted.
On freertos.org was about the same question, and there they advised using thread-safe memory functions. This solved my problem.

ESP_Sprite
Posts: 9772
Joined: Thu Nov 26, 2015 4:08 am

Re: Heap corrupting when using xQueue

Postby ESP_Sprite » Mon Jun 25, 2018 2:50 am

First of all: heap_3.c is not threadsafe in the ESP32 environment: any tasks on the 2nd core will continue running when vTaskSuspendAll is called. Secondly: this allocator is just a small wrapper around the existing malloc() code; the difference in calling process probably changes the timing on a race condition in your code in such a way that it happens to work in this case. Thirdly: the fact that the error occurs in a FreeRTOS function does not mean the FreeRTOS function caused the memory cporruption: it just crashes because it can't handle the *results* of that corruption.

My guess is that you have a race condition somewhere in your code that tends to corrupt your memory. I suggest you try to figure out what this is using the heap debugging functions. If you're okay with posting your code here, we can also take a look at it.

x-8973
Posts: 20
Joined: Tue Apr 03, 2018 4:49 pm

Re: Heap corrupting when using xQueue

Postby x-8973 » Mon Jun 25, 2018 3:38 pm

In fact, the project is already very large and quite confusing. In addition, comments in Russian)
However, I can give a link to my SVN so you can take a look. I will send it to you with a personal message.
Also I want to note that the error does not necessarily occur in the RTOS task. It can occur in any random place, but it manifests itself only if in the RTOS task in some way were caused by malloc/realloc/free.
In addition, all RTOS tasks, except one, were commented out. In the event that the malloc was called in, a crash occurred.

ESP_Sprite
Posts: 9772
Joined: Thu Nov 26, 2015 4:08 am

Re: Heap corrupting when using xQueue

Postby ESP_Sprite » Tue Jun 26, 2018 3:59 am

Yep, just browsed through it, unfortunately it is indeed to complicated to casually look for buffer overflows and I can't easily compile it in such a way that it doesn't break because it needs certain bits of hardware. (tried, but I can't get the error, instead I get a crash when entering 'help' on the console...)

x-8973
Posts: 20
Joined: Tue Apr 03, 2018 4:49 pm

Re: Heap corrupting when using xQueue

Postby x-8973 » Tue Jun 26, 2018 5:01 am

Yes, it uses NVS to store some settings, and only today I found that if they are not there can be a drop. However, this does not seem to be an error with a heap damage.
I will try to fix this problem and make a fresh commit.

OneTwo
Posts: 20
Joined: Mon Jan 15, 2018 9:24 am

Re: Heap corrupting when using xQueue

Postby OneTwo » Tue Jun 26, 2018 6:17 am

x-8973 wrote:I use xQueue to exchange messages between two RTOS tasks. I encountered a problem when using xQueue causes damage to dynamic memory, and sometimes kernel panic. The xQueueCreate(), xQueueSend() and xQueueReceive() functions are used in accordance with the documentation for the RTOS.

Code: Select all

void app_main()
{
	...
	systemEvtQueue = xQueueCreate(10, sizeof(uint8_t));
	...
}

static void task1(void *pvParameters)
{
	uint8_t event = 0;
	while (1)
	{
		event = 3;
		xQueueSend(systemEvtQueue, (void *)&event, (TickType_t)0);
		ESP_LOGI(__func__, "Sending event %i", event);
		
		vTaskDelay(1000 / portTICK_PERIOD_MS);
	}
}

static void task2(void *pvParameters)
{
	uint8_t event = 0;
	while (1)
	{
		if (xQueueReceive(systemEvtQueue, (void *)&event, (TickType_t)0))
		{
			ESP_LOGI(__func__, "Received event %i", event);
		}
		vTaskDelay(100 / portTICK_PERIOD_MS);
	}
}
I determine the memory corruption by the appearance of graphical artifacts on the OLED, for which the framebuffer was created using the function malloc(). Sometimes a kernel panic occurs, indicating the file is multi-heap.c. I can show the console output when the kernel panic reoccurs.
Maybe im wrong but your local variable event is used between xQueueSend and xQueueReceive.

Furthermore you create the queue by xQueueCreate(10, sizeof(uint8_t));
but you send and receive a "pointer".

Have you tried with xQueueCreate(10, sizeof(uint8_t*));

Who is online

Users browsing this forum: Majestic-12 [Bot] and 90 guests