Recommendations for MQTT Library with ESPIDF

[PeterR]

Hi @ESP_tuanpm
I think it's not about re-send the failed packet.
The esp-mqtt library is *not* thread safe at all.
Just think about 2 tasks calling any of esp_mqtt_client_subscribe()/esp_mqtt_client_unsubscribe()/esp_mqtt_client_ping()/ esp_mqtt_client_publish() APIs at the same time on differnct CPU, All the client->mqtt_state.* fields can be racy.
This needs fix.

May I confirm your understanding of the issue?
(1) Are you saying that esp-mqtt is not thread safe when two CPU/tasks are write/reading from a single subscription?
(2) Are you saying that esp-mqtt is not thread safe when two CPU/tasks each owning their own subscription are in use?

Perhaps (1) is acceptable as it is a socket after all. (2) seems a more serious issue.

[permal]

PeterR,

The readme for esp-mqtt (https://github.com/espressif/esp-mqtt) states "Multiple instances (Multiple clients in one application)" which doesn't specifically state "thread safe", but the question is what are you asking for when you mean "thread safe"?

From the samples it looks like that if you do not share your esp_mqtt_client_config_t, esp_mqtt_client_handle_t or mqtt_event_handler across threads you should be fine. I've not used it myself (I wrote my own implementation in C++ before this made it into IDF) so I could be wrong, but the API does appear to support multiple instances in different threads as long as you use it as intended.

Side note: You can share the mqtt_event_handler between threads, if you make it thread-safe/reentrant, ie. it doesn't hold or accesses any state.

[PeterR]

??
The question is directed at ESP_tuanpm and axellin
They are clearly discussing and seem to have agreed that there is an RT issue.

question is what are you asking for when you mean "thread safe"?

As posted in my question. I asked for clarification if failing scenario is (1) and (2) (or both).

[axellin]

The esp_mqtt_client_start() implementation creates a task (esp_mqtt_task).
So even you just calling publish in your application task.
It's still racy because there is no locking for the data structure accessed by two tasks.
The mqtt library really needs fix.

[permal]

The esp_mqtt_client_start() implementation creates a task (esp_mqtt_task).
So even you just calling publish in your application task.
It's still racy because there is no locking for the data structure accessed by two tasks.
The mqtt library really needs fix.

Ah, I was deceived by the API Good to get it clarified.

[anandvilayil]

hi,

i am facing a problem, when i continuously doing mqtt publish[note: mqtt publish is done from separate thread not from the mqtt thread] mqtt connection got disconnected. the thing is esp still have ip, ie not disconnected from wifi its showing error of no internet connectivity. for the last mqtt publish it shown like this,
D (1287723) MQTT_CLIENT: mqtt_enqueue id: 38955, type=3 successful
D (1287723) OUTBOX: ENQUEUE msgid=38955, msg_type=3, len=246, size=246

after this arise i tried for esp_restart(), then its got stuck and showing like
W (150371100) wifi: TX Q not empty: 500
W (150376100) wifi: TX Q not empty: 1000
W (150381100) wifi: TX Q not empty: 1500
W (150386100) wifi: TX Q not empty: 2000

iam using latest esp idf: master branch, com id:fb7ba1baa068a2267ef54b7105cf3c8a004fe4eb

did anybody know why this happens.?