ESP32 PPP disconnection re-connection issue

[el_kadenaz]

Hi
in my company we developed an IOT device that use the ESP32 and a Quectel modem. The internet connection part is based on the PPP example of the idf version that we are using: the v4.3.
All works fine until a disconnection event, we are not able to perform a clean re-connection to the internet without a

Code: Select all

Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.
Core  0 register dump:

.

In order to reproduce the issue we set a wrong APN in order to test a user typing error.

The steps are these:

Tested either
1

Code: Select all

tcpip_adapter_init();

or

Code: Select all

esp_netif_init();

with the same results

2 LTE modem powering on and configuration

3 trying a PPP connection with wrong APN simulating user typing error

Code: Select all

esp_modem_setup_ppp(dte)

the connection fail for timeout or MODEM_EVENT_PPP_DISCONNECT

4

Code: Select all

esp_modem_free_netif_adapter()

5 enter the correct APN

6

Code: Select all

esp_modem_setup_ppp(dte)

Code: Select all

Monitor Log:

netif: GW address of interface pp set to 10.64.64.64
netif_set_ipaddr: netif address being changed
I (183960) esp-netif_lwip-ppp: Connected
I (183970) esp-netif_lwip-ppp: Name Server1: 151.5.216.25
I (183970) esp-netif_lwip-ppp: Name Server2: 151.5.216.225

Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.
Core  0 register dump:
PC      : 0x401f3058  PS      : 0x00060730  A0      : 0x801e147a  A1      : 0x3ffdf720
A2      : 0x0000001c  A3      : 0x0000007a  A4      : 0x0000007e  A5      : 0x00000082
A6      : 0x00000000  A7      : 0x3f96f428  A8      : 0x8009a448  A9      : 0x3ffdf700
A10     : 0x00000003  A11     : 0x00060723  A12     : 0x00060720  A13     : 0x3ffdf7c4
A14     : 0x3ffd5b30  A15     : 0x00000000  SAR     : 0x00000017  EXCCAUSE: 0x0000001c
EXCVADDR: 0x0000007a  LBEG    : 0x40091f64  LEND    : 0x40091f80  LCOUNT  : 0xffffffff

Backtrace:0x401f3055:0x3ffdf720 0x401e1477:0x3ffdf760 0x401e109a:0x3ffdf780 0x401f0e91:0x3ffdf7a0 0x401f0f3c:0x3ffdf7c0 0x4009a2bd:0x3ffdf7f0

0x401f3058: netif_set_addr at C:/git/esp32/p495_cronowifi/sub/esp-idf/components/lwip/lwip/src/core/netif.c:712 (discriminator 2)
0x40091f64: memcpy at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/machine/xtensa/memcpy.S:175
0x40091f80: memcpy at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/machine/xtensa/memcpy.S:197
0x401f3055: netif_set_addr at C:/git/esp32/p495_cronowifi/sub/esp-idf/components/lwip/lwip/src/core/netif.c:709
0x401e1477: esp_netif_up_api at C:/git/esp32/p495_cronowifi/sub/esp-idf/components/esp_netif/lwip/esp_netif_lwip.c:1215
0x401e109a: esp_netif_api_cb at C:/git/esp32/p495_cronowifi/sub/esp-idf/components/esp_netif/lwip/esp_netif_lwip.c:107
0x401f0e91: tcpip_thread_handle_msg at C:/git/esp32/p495_cronowifi/sub/esp-idf/components/lwip/lwip/src/api/tcpip.c:168 (discriminator 4)
0x401f0f3c: tcpip_thread at C:/git/esp32/p495_cronowifi/sub/esp-idf/components/lwip/lwip/src/api/tcpip.c:154
0x4009a2bd: vPortTaskWrapper at C:/git/esp32/p495_cronowifi/sub/esp-idf/components/freertos/port/xtensa/port.c:168

I'm missing something else to clean or free before perform the PPP setup again?
thanks, Roberto.

[chegewara]

Based on espressif example:
https://github.com/espressif/esp-idf/bl ... #L322-L325

here is my code, whic works with BG96:

Code: Select all

void disconnectLTEmodem()
{
    if(!dte) return;

    // /* Exit PPP mode */
    ESP_ERROR_CHECK_WITHOUT_ABORT(esp_modem_stop_ppp(dte));

    // /* Power down module */
    ESP_ERROR_CHECK_WITHOUT_ABORT(dce->power_down(dce));
    ESP_LOGI(TAG, "Power down");
    ESP_ERROR_CHECK_WITHOUT_ABORT(dce->deinit(dce));

    /* Unregister events, destroy the netif adapter and destroy its esp-netif instance */
    esp_modem_netif_clear_default_handlers(modem_netif_adapter);
    esp_modem_netif_teardown(modem_netif_adapter);
    esp_netif_destroy(esp_netif);

    ESP_ERROR_CHECK_WITHOUT_ABORT(dte->deinit(dte));
    dte = NULL;
    dce = NULL;
    ESP_LOGW(TAG, "LTE modem and connection destroyed");
}

[el_kadenaz]

Sorry, I missed that before the

Code: Select all

esp_modem_free_netif_adapter()

i call the

Code: Select all

esp_modem_stop_ppp( dte )

.

the code seems to be the same except that I do not deinit the dce and dte:

Code: Select all

  ESP_ERROR_CHECK_WITHOUT_ABORT(dce->deinit(dce));
    ESP_ERROR_CHECK_WITHOUT_ABORT(dte->deinit(dte));

Can you reconnect to the internet without reset after those operations?

Thanks, Roberto

[el_kadenaz]

Sorry, I missed to write that before

Code: Select all

esp_modem_free_netif_adapter()

that call the three netif function you use (

Code: Select all

esp_modem_netif_clear_default_handlers(modem_netif_adapter);
    esp_modem_netif_teardown(modem_netif_adapter);
    esp_netif_destroy(esp_netif);

)
I also call:

Code: Select all

esp_modem_stop_ppp(dte)

.

I don't need to deinit the modem structure and the uart I need only to retry the PPPoS connection.

Does your code reconnect to the internet without reset?

My backtrace say that the fail is always in the netif_set_addr() function

Thanks, Roberto

[chegewara]

Im not sure right now, but i think it didnt work without deinit dte/dce. Since it works very stable with deinit and it takes only few seconds i am using this way to do it.

[el_kadenaz]

I don't understand how the modem deinit can affect the LWIP part. However, for us it would be really bad to have to do it because at that moment the user is doing the provisioning with a smartphone APP which is already slow enough, making him wait another 20/30 seconds for the modem to switch on and hook up to the network is bad.

Roberto

[ESP_cermak]

Hi Roberto,

Could you please share you deinit logic? It should be possible to reconnect without restart without any issue.
I've posted a quick test code here https://gist.github.com/david-cermak/f7 ... f185069dc7 which works for me and just disconnects and reconnects periodically.

The error you're experiencing is very similar to https://github.com/espressif/esp-idf/issues/7469 reported on GitHub, but wasn't able to reproduce it, so would be good if you could share a minimal project or example that could be used to reproduce your crash.

Thanks,
David

[el_kadenaz]

Hi David,

yes, the error seems to be the same of the linked githup report.
I'm not able to share a minimal project in short time because the project I'm working on is quite big.
the connection disconnection phase is managed by a state machine and my deinit logic is

Code: Select all

   esp_modem_stop_ppp( dte );
   esp_event_handler_unregister(IP_EVENT, ESP_EVENT_ANY_ID, &on_ip_event);
   esp_modem_netif_clear_default_handlers(modem_netif_adapter);
   esp_modem_netif_teardown(modem_netif_adapter);
   esp_netif_destroy(esp_netif);

before the ppp stop I rise a disconnection event and destroy the MQTT client:

Code: Select all

		case MNGLTE_EVENT_DISCONNECT:
                      if(mqtt_mng.mqtt_client)
                      {
                              ESP_LOGI(TAG, "MQTT client STOP");
                              esp_mqtt_client_destroy(mqtt_mng.mqtt_client);
                              mqtt_mng.mqtt_client = nullptr;
                      }
			break;
			

I've a setup with a cyclic connect/disconnect phase and after 2 or 3 correct reconnection, when I receive the new IP the same error occurs.

Code: Select all

I (368752) ~~~ MNGLTE ~~~: FOUND APN: web.omnitel.it
I (368752) ~~~ MNGLTE ~~~: Setup PPP 
W (368752) event: handler already registered, overwriting
W (368752) event: handler already registered, overwriting
I (369062) ~~~ MNGLTE ~~~: FSM  ( MNGLTE_ON_IN_CMD_MODE -> MNGLTE_CONNECTION )
I (369162) ~~~ MNGLTE ~~~: Modem PPP Started
I (370452) esp-netif_lwip-ppp: Connected
I (370452) esp-netif_lwip-ppp: Name Server1: 10.133.47.86
I (370452) esp-netif_lwip-ppp: Name Server2: 10.132.100.212
Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.

[ESP_cermak]

Okay, I see that your deinit function is almost the same as in my example. As noted in that GitHub issue, there must have been some other network interface, and I believe this is your case, too. What other interfaces you use?

> I'm not able to share a minimal project in short time

No worries, please take your time, I'd really appreciate your help here.

> before the ppp stop I rise a disconnection event and destroy the MQTT client:

Does this even work? I mean this code probably disconnects your client, but wouldn't destroy it, and might be causing at least men-leak (and a hanging thread). I'd suggest calling `esp_mqtt_client_destroy()` from somewhere else that event handler.

[el_kadenaz]

What other interfaces you use?

During the provisioning I need to init also the wifi in order to use the espressif provided provisioning api. This made me doubt so I did the cyclic connection/disconnection code in an already provisioned device that has the same issue but no other internet interfaces.

at the second disconnection I' note tath doubled event

Code: Select all

I (319233) esp-netif_lwip-ppp: User interrupt
I (319233) esp-modem-netif: PPP state changed event 5
I (319233) esp-modem-netif: PPP state changed event 5
I (319233) esp_modem_free_netif_adapter: free

like if the esp_modem_free_netif_adapter() of the modem does not clean all well

Does this even work?

yes seems work, but I'll try also to move somewhere else.

In order to perform the disconnection from my FSM I've edited the esp_modem_compat.c of the example of idf 4.3 https://github.com/montirob/esp32_PPP/b ... m_compat.c maybe I did a mistake
thanks for the support