ESP_ping issue in v3.3-beta3

[thethinker]

It seems like there is an issue with the esp_ping that causes wifi driver to stop acting properly.

I had a code that has been running for a while, but as soon as I updated to this version of IDF it stopped working.

The program is running a task called main which checks for connectivity by pinging the gateway address (because I'm using ethernet interface and there is no lost_ip event.
In this task I basically have this running in a loop:

Code: Select all

while (true) {
		ESP_ERROR_CHECK(tcpip_adapter_get_ip_info(TCPIP_ADAPTER_IF_ETH/*ESP_IF_ETH*/, &ip));
		if (connection_mode_wifi)
		{
			ping_deinit();
		}
		if (!connection_mode_wifi && (ip.gw.addr != 0) && !waiting_results && ((xEventGroupGetBits(blah_event_group) & INTERNET_ACTIVE_BIT) != 0)) {uint32_t ping_count = 25;  //how many pings per report
			uint32_t ping_timeout = 500; //mS till we consider it timed out
			uint32_t ping_delay = 1000; //mS between pings
			esp_ping_set_target(PING_TARGET_IP_ADDRESS_COUNT, &ping_count, sizeof(uint32_t));
			esp_ping_set_target(PING_TARGET_RCV_TIMEO, &ping_timeout, sizeof(uint32_t));
			esp_ping_set_target(PING_TARGET_DELAY_TIME, &ping_delay, sizeof(uint32_t));
			esp_ping_set_target(PING_TARGET_IP_ADDRESS, &ip.gw.addr, sizeof(uint32_t));
			esp_ping_set_target(PING_TARGET_RES_FN, (void*)&ethGwPingResults, sizeof(esp_err_t));
			ping_init();
			waiting_results = true;
			ESP_LOGI(main_tag, "Resetting Ping ...");

and here is the call back:

Code: Select all

esp_err_t ethGwPingResults(ping_target_id_t msgType, esp_ping_found * pf) {
	ESP_LOGI(gw_ping_tag, "AvgTime:%.1fmS Sent:%d Rec:%d Err:%d min(mS):%d max(mS):%d ", (float)pf->total_time / pf->recv_count, pf->send_count, pf->recv_count, pf->err_count, pf->min_time, pf->max_time);
	ESP_LOGI(gw_ping_tag, "Resp(mS):%d Timeouts:%d Total Time:%d\n", pf->resp_time, pf->timeout_count, pf->total_time);
	if(pf->send_count >= 25)
		waiting_results = false;
	if (pf->timeout_count >= 10 && !connection_mode_wifi) // We lost ETH connection 
	{
		xEventGroupClearBits(blah_event_group, INTERNET_ACTIVE_BIT); 
		xEventGroupSetBits(eth_event_group, DHCP_LOST_BIT);

	}
	return ESP_OK;
}

and there is another task which is just a sniffer.

The sniffer stops working unless I comment out the ping! I also tried connecting to wifi and that doesn't work anymore neither.

The sniffer task and wifi stack are pinned to core 0 and everything else and emac is pinned to core 1. The sniffer task has high priority (18) , emac has priority 10, and ping task has priority 2.


Can someone else confirm this please?

[thethinker]

I found the issue! I was calling the following function in my ping task every 100 ms, and apparently calling this too often will choke the wifi stack in the new version of IDF, can someone please clarify why?

ESP_ERROR_CHECK(tcpip_adapter_get_ip_info(TCPIP_ADAPTER_IF_ETH, &ip));

[Ritesh]

Hi,

In which IDF version, It was working fine?

Also, Did you try to check into master branch as well?

If it is really issue then better to raise issue over Github ESP32 IDF as well. So that it will be helpful to get quick fix and response fir same.

[thethinker]

I Upgraded from 3.1, master is doing the same thing. Actually my previous thought was wrong, the issue had nothing to do with the interface call. The issue is ping process starting before wifi stack (I use ethernet interface as well). So what I did is put a flag so it waits for wifi to initialize and wait an additional 1 second before it starts the ping. Now it works perfectly! I have no idea why !!!

[Ritesh]

I Upgraded from 3.1, master is doing the same thing. Actually my previous thought was wrong, the issue had nothing to do with the interface call. The issue is ping process starting before wifi stack (I use ethernet interface as well). So what I did is put a flag so it waits for wifi to initialize and wait an additional 1 second before it starts the ping. Now it works perfectly! I have no idea why !!!

Yes.

That is correct way to start Ping Service like first of wait till you get connectivity and then start Ping Service once you get connectivity event.

[thethinker]

I Upgraded from 3.1, master is doing the same thing. Actually my previous thought was wrong, the issue had nothing to do with the interface call. The issue is ping process starting before wifi stack (I use ethernet interface as well). So what I did is put a flag so it waits for wifi to initialize and wait an additional 1 second before it starts the ping. Now it works perfectly! I have no idea why !!!

Yes.

That is correct way to start Ping Service like first of wait till you get connectivity and then start Ping Service once you get connectivity event.

Hi There,
I'm using Ethernet interface, WIFI is just used for sniffing. I turn on the ping after Ethernet gets IP. So this can't be the reason.

It seems to be a race condition happening. Its almost as if the task priority on either WIFI or PING modules or one of the dependent tasks changed between these two versions.

[Ritesh]

I Upgraded from 3.1, master is doing the same thing. Actually my previous thought was wrong, the issue had nothing to do with the interface call. The issue is ping process starting before wifi stack (I use ethernet interface as well). So what I did is put a flag so it waits for wifi to initialize and wait an additional 1 second before it starts the ping. Now it works perfectly! I have no idea why !!!

Yes.

That is correct way to start Ping Service like first of wait till you get connectivity and then start Ping Service once you get connectivity event.

Hi There,
I'm using Ethernet interface, WIFI is just used for sniffing. I turn on the ping after Ethernet gets IP. So this can't be the reason.

It seems to be a race condition happening. Its almost as if the task priority on either WIFI or PING modules or one of the dependent tasks changed between these two versions.

Ok.

So, Did you find anything after that? or still looking for solution related to task priority or something like that?

[thethinker]

Still looking. I think I will have to dig deeper into the API source, but I have to narrow it down first. I'm thinking it is most likely in WiFi, LWIP modules.

[Ritesh]

Still looking. I think I will have to dig deeper into the API source, but I have to narrow it down first. I'm thinking it is most likely in WiFi, LWIP modules.

Sounds Good.

Let me know if need any further help regarding that.

[thethinker]

I found the issue !
Under menu config, if I enable any or both of the following two options, weird things such as the one I explained starts happening:
-- CONFIG_SPIRAM_CACHE_WORKAROUND

As soon as I turn it off, things start working again!! So between idf 3.1 and 3.3 someone has changed something in LWIP logic to be in this workaround and this is causing the trouble (I'm guessing my code is getting removed at per-compilelation by this option)

To give you some insight I'm using a PICO-D4 with modified psram code to bypass the "psram not supported" errors, but even when I upgrade to 4.0 master branch which supports psram for pico-d4 it still has this issue.
I also have my sdkconfig attached.

Unfortunately I don't have time to dig deeper right now, but app so I'm all good for now!