MQTT over SSL - Mosquitto

[balint603]

Hi,

I'm having trouble with MQTT connection.
I'd like to create SSL based communication between a Mosquitto broker and an ESP32 using ESP32 mqtt component.
Finally (it looks like) the transmission layer works.
But now the problem is the MQTT level connection doesn't work, because the client gets an error just after the broker sends connect acknowledge message.
What could go wrong?

I use this example project : https://github.com/espressif/esp-mqtt/t ... s/mqtt_ssl
I only changed the client settings to this:
const esp_mqtt_client_config_t mqtt_cfg = {
.uri = "mqtts://iot.eclipse.org:8883",
.host = "192.168.1.11",
.port = 8883,
.event_handle = mqtt_event_handler,
.cert_pem = (const char *)iot_eclipse_org_pem_start,
};

Client log:
I (5551) MQTT_CLIENT: Sending MQTT CONNECT message, type: 1, id: 0000
E (5571) MQTT_CLIENT: Invalid MSG_TYPE response: 0, read_len: 16
I (5571) MQTT_CLIENT: Error MQTT Connected
I (5581) MQTT_CLIENT: Reconnect after 10000 ms
I (5581) MQTTS_SAMPLE: MQTT_EVENT_DISCONNECTED

Please guys help me to solve this problem!

[Zeni241]

Try this

Code: Select all

 
         uri = "mqtts://iot.eclipse.org:8883"
        .event_handle = mqtt_event_handler,
        .cert_pem = (const char * ) server_cert_pem_start,
        .username = "your user name",
        .password = "your password",
       

[balint603]

Try this

Code: Select all

 
         uri = "mqtts://iot.eclipse.org:8883"
        .event_handle = mqtt_event_handler,
        .cert_pem = (const char * ) server_cert_pem_start,
        .username = "your user name",
        .password = "your password",
    

Thanks, but I need to connect to a local broker.
I changed my client settings to this:

Code: Select all

    const esp_mqtt_client_config_t mqtt_cfg = {
        .host = "192.168.1.11",
        .port = 8883,
        .event_handle = mqtt_event_handler,
        .cert_pem = (const char *)iot_eclipse_org_pem_start,
    };

But now, broker gets OpenSSL (wrong version) error, so It still has trouble with transmission...

[ESP_cermak]

Hi,
could you please try this?

Code: Select all

const esp_mqtt_client_config_t mqtt_cfg = {
.uri = "mqtts://192.168.1.11:8883",
.event_handle = mqtt_event_handler,
.cert_pem = (const char *)your_server_cert_pem_start,
};

[balint603]

Hi,
could you please try this?

Code: Select all

const esp_mqtt_client_config_t mqtt_cfg = {
.uri = "mqtts://192.168.1.11:8883",
.event_handle = mqtt_event_handler,
.cert_pem = (const char *)your_server_cert_pem_start,
};

I changed the URI but still the same error.
My Mosquitto broker detects socket error on client. Does that mean the client disconnects because of the MSG_TYPE error?
With debug logging:

D (5125) MQTT_CLIENT: MQTT client_id=ESP32_d5FE19
D (5135) MQTT_CLIENT: Core selection enabled on 0
I (5595) wifi: pm start, type:0

D (6035) MQTT_CLIENT: Transport connected to mqtts://192.168.1.11:8883
I (6035) MQTT_CLIENT: Sending MQTT CONNECT message, type: 1, id: 0000
E (6045) MQTT_CLIENT: Invalid MSG_TYPE response: 0, read_len: 16
I (6055) MQTT_CLIENT: Error MQTT Connected
I (6065) MQTT_CLIENT: Reconnect after 10000 ms
I (6065) MQTTS_SAMPLE: MQTT_EVENT_DISCONNECTED
D (21065) MQTT_CLIENT: Reconnecting...

Thanks.

[ESP_cermak]

Hi balint603

Espressif supports this library from IDF v3.2, but still this should work with 3.1 and 3.0 (Have just tested on idf 3.0 and connects correctly to the public broker)

Can you please test if
- you can connect to iot.eclipse.org (broker from the example)
- you connect to your local mosquitto with latest IDF (any version from 3.2 to master)?
- you connect to your local mosquitto with a python script?
- you connect to your local mosquitto skipping the certificate verification (just remove line `.cert_pem = (const char *)_cert_start`)

From the log it looks like the socket disconnects after sending connect message, there's no error type, just blank message, which does not make any sense to me...? Could some firewall be cutting the traffic off?

Thanks,
David

[balint603]

Hi balint603

Espressif supports this library from IDF v3.2, but still this should work with 3.1 and 3.0 (Have just tested on idf 3.0 and connects correctly to the public broker)

Can you please test if
- you can connect to iot.eclipse.org (broker from the example)->OK
- you connect to your local mosquitto with latest IDF (any version from 3.2 to master)?->FAILED
- you connect to your local mosquitto with a python script?->OK
- you connect to your local mosquitto skipping the certificate verification (just remove line `.cert_pem = (const char *)_cert_start`)->FAILED

From the log it looks like the socket disconnects after sending connect message, there's no error type, just blank message, which does not make any sense to me...? Could some firewall be cutting the traffic off?

Thanks,
David

I cloned the current IDF and made some tests saving the logs and the current client settings:
I also tried turning off the firewall of my ISR and Ubuntu, no success.
Running the python script from another machine works too.
If the generated .pem file was not appropriate, would a transmission error occur?

Code: Select all

TEST_1______________________________________________________________________________________OK
Connect to global broker (SSL example project)
Client log:
D (8135) MQTT_CLIENT: MQTT client_id=ESP32_d5FE19
D (8145) MQTT_CLIENT: Core selection disabled
D (9485) MQTT_CLIENT: Transport connected to mqtts://iot.eclipse.org:8883
I (9485) MQTT_CLIENT: Sending MQTT CONNECT message, type: 1, id: 0000
D (9635) MQTT_CLIENT: Connected
I (9635) MQTTS_EXAMPLE: MQTT_EVENT_CONNECTED
D (9635) MQTT_CLIENT: mqtt_enqueue id: 0, type=1 successful
D (9635) MQTT_CLIENT: Sent subscribe topic=/topic/qos0, id: 52190, type=8 successful
I (9645) MQTTS_EXAMPLE: sent subscribe successful, msg_id=52190
D (9655) MQTT_CLIENT: mqtt_enqueue id: 52190, type=8 successful
D (9655) OUTBOX: ENQUEUE msgid=52190, msg_type=8, len=18, size=18
D (9665) MQTT_CLIENT: Sent subscribe topic=/topic/qos1, id: 35928, type=8 successful
I (9675) MQTTS_EXAMPLE: sent subscribe successful, msg_id=35928
D (9675) MQTT_CLIENT: mqtt_enqueue id: 35928, type=8 successful
D (9685) OUTBOX: ENQUEUE msgid=35928, msg_type=8, len=18, size=36
D (9695) MQTT_CLIENT: unsubscribe, topic"/topic/qos1", id: 6477
D (9695) MQTT_CLIENT: Sent Unsubscribe topic=/topic/qos1, id: 6477, successful
I (9705) MQTTS_EXAMPLE: sent unsubscribe successful, msg_id=6477
D (9785) MQTT_CLIENT: msg_type=9, msg_id=52190
D (9785) MQTT_CLIENT: pending_id=6477, pending_msg_count = 3
D (9785) OUTBOX: DELETED msgid=52190, msg_type=8, remain size=18
D (9795) MQTT_CLIENT: Subscribe successful
I (9795) MQTTS_EXAMPLE: MQTT_EVENT_SUBSCRIBED, msg_id=52190
I (9805) MQTTS_EXAMPLE: sent publish successful, msg_id=0
D (9925) MQTT_CLIENT: msg_type=9, msg_id=35928
D (9935) MQTT_CLIENT: pending_id=6477, pending_msg_count = 2
D (9935) OUTBOX: DELETED msgid=35928, msg_type=8, remain size=0
D (9935) MQTT_CLIENT: Subscribe successful
I (9935) MQTTS_EXAMPLE: MQTT_EVENT_SUBSCRIBED, msg_id=35928
I (9945) MQTTS_EXAMPLE: sent publish successful, msg_id=0
D (10075) MQTT_CLIENT: msg_type=11, msg_id=6477
D (10075) MQTT_CLIENT: pending_id=6477, pending_msg_count = 1
D (10075) MQTT_CLIENT: UnSubscribe successful
I (10075) MQTTS_EXAMPLE: MQTT_EVENT_UNSUBSCRIBED, msg_id=6477
D (10215) MQTT_CLIENT: msg_type=3, msg_id=0
I (10215) MQTT_CLIENT: deliver_publish, message_length_read=19, message_length=19
D (10215) MQTT_CLIENT: Get data len= 4, topic len=11
I (10225) MQTTS_EXAMPLE: MQTT_EVENT_DATA
TOPIC=/topic/qos0
DATA=data
D (10425) MQTT_CLIENT: msg_type=3, msg_id=0
I (10425) MQTT_CLIENT: deliver_publish, message_length_read=19, message_length=19
D (10425) MQTT_CLIENT: Get data len= 4, topic len=11
I (10435) MQTTS_EXAMPLE: MQTT_EVENT_DATA
TOPIC=/topic/qos0
DATA=data

TEST_2____________________________________________________________________________________NOPE
Client config changes:
-        .uri = CONFIG_BROKER_URI,
+        //.uri = CONFIG_BROKER_URI,
+        .uri = "mqtts://192.168.1.11:8883",

Using the original .pem file.

Mosquitto:
1553162287: New connection from 192.168.1.115 on port 8883.
1553162287: OpenSSL Error: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
1553162287: Socket error on client <unknown>, disconnecting.

Client log:
I (8165) event: sta ip: 192.168.1.115, mask: 255.255.255.0, gw: 192.168.1.2
I (8165) MQTTS_EXAMPLE: [APP] Free memory: 236656 bytes
I (8165) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
D (8175) MQTT_CLIENT: MQTT client_id=ESP32_d5FE19
D (8185) MQTT_CLIENT: Core selection disabled
E (8315) TRANS_SSL: mbedtls_ssl_handshake returned -0x2700
E (8325) MQTT_CLIENT: Error transport connect
I (8325) MQTT_CLIENT: Reconnect after 10000 ms
I (8325) MQTTS_EXAMPLE: MQTT_EVENT_DISCONNECTED
D (18335) MQTT_CLIENT: Reconnecting...

TEST_2.1___________________________________________________________________________________NOPE
Client config changes:
-        .uri = CONFIG_BROKER_URI,
+        //.uri = CONFIG_BROKER_URI,
+        .uri = "mqtts://192.168.1.11:8883",

Using a generated .pem file (Steves's guide)

Mosquitto:
1553163090: New connection from 192.168.1.115 on port 8883.
1553163091: New client connected from 192.168.1.115 as ESP32_d5FE19 (c1, k120).
1553163091: Sending CONNACK to ESP32_d5FE19 (0, 0)
1553163091: Socket error on client ESP32_d5FE19, disconnecting.

Client log:
I (4656) event: sta ip: 192.168.1.115, mask: 255.255.255.0, gw: 192.168.1.2
I (4656) MQTTS_EXAMPLE: [APP] Free memory: 236660 bytes
I (4656) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
D (4666) MQTT_CLIENT: MQTT client_id=ESP32_d5FE19
D (4676) MQTT_CLIENT: Core selection disabled
D (5536) MQTT_CLIENT: Transport connected to mqtts://192.168.1.11:8883
I (5546) MQTT_CLIENT: Sending MQTT CONNECT message, type: 1, id: 0000
E (5556) MQTT_CLIENT: Invalid MSG_TYPE response: 0, read_len: 16
I (5556) MQTT_CLIENT: Error MQTT Connected
I (5576) MQTT_CLIENT: Reconnect after 10000 ms
I (5576) MQTTS_EXAMPLE: MQTT_EVENT_DISCONNECTED
I (5596) wifi: pm start, type:0
D (20576) MQTT_CLIENT: Reconnecting...

TEST_3______________________________________________________________________________________OK
Python script.

Mosquitto:
1553169896: New connection from 192.168.1.11 on port 8883.
1553169896: New client connected from 192.168.1.11 as controll (c1, k60).
1553169896: Sending CONNACK to controll (0, 0)
1553169900: Received PUBLISH from controll (d0, q0, r0, m0, 'movies/pulp_fiction', ... (72 bytes))
1553169904: Received DISCONNECT from controll
1553169904: Client controll disconnected.

Python client log:
('buffer ', 'Sending CONNECT (u0, p0, wr0, wq0, wf0, c1, k60) client_id=controll')
('waiting', False)
('buffer ', 'Received CONNACK (0, 0)')
('connected', True)
publishing
('buffer ', "Sending PUBLISH (d0, q0, r0, m1), 'movies/pulp_fiction', ... (72 bytes)")
('buffer ', 'Sending DISCONNECT')

TEST_4____________________________________________________________________________________NOPE
Skipping the certification.
Client config:
-        .uri = CONFIG_BROKER_URI,
+        //.uri = CONFIG_BROKER_URI,
+        .uri = "mqtts://192.168.1.11:8883",
         .event_handle = mqtt_event_handler,
-        .cert_pem = (const char *)iot_eclipse_org_pem_start,
+        //.cert_pem = (const char *)iot_eclipse_org_pem_start,

Mosquitto:
1553163659: New connection from 192.168.1.115 on port 8883.
1553163660: New client connected from 192.168.1.115 as ESP32_d5FE19 (c1, k120).
1553163660: Sending CONNACK to ESP32_d5FE19 (0, 0)
1553163660: Socket error on client ESP32_d5FE19, disconnecting.
 
Client log:
I (4633) event: sta ip: 192.168.1.115, mask: 255.255.255.0, gw: 192.168.1.2
I (4633) MQTTS_EXAMPLE: [APP] Free memory: 236480 bytes
I (4633) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
D (4643) MQTT_CLIENT: MQTT client_id=ESP32_d5FE19
D (4643) MQTT_CLIENT: Core selection disabled
D (5433) MQTT_CLIENT: Transport connected to mqtts://192.168.1.11:8883
I (5433) MQTT_CLIENT: Sending MQTT CONNECT message, type: 1, id: 0000
E (5443) MQTT_CLIENT: Invalid MSG_TYPE response: 0, read_len: 16
I (5443) MQTT_CLIENT: Error MQTT Connected
I (5463) MQTT_CLIENT: Reconnect after 10000 ms

Python script:

Code: [Select all] [Expand/Collapse]

1. import paho.mqtt.client as paho
2. import time
3. broker="192.168.1.11"
4. port=8883
5. conn_flag = False
6. def on_connect(client, userdata, flags, rc):
7.     global conn_flag
8.     conn_flag = True
9.     print("connected",conn_flag)
10.     conn_flag = True
11. def on_log(client, userdata, level, buf):
12.     print("buffer ",buf)
13. def on_disconnect(client, userdata, rc):
14.     print("client disconnected ok")
15. client1 = paho.Client("controll")
16. client1.on_log = on_log
17. client1.tls_set('ca.crt')
18. client1.on_connect = on_connect
19. client1.on_disconnect = on_disconnect
20. client1.connect(broker,port)
21. while not conn_flag:
22.     time.sleep(1)
23.     print("waiting",conn_flag)
24.     client1.loop()
25. time.sleep(3)
26. print("publishing")
27. client1.publish("movies/pulp_fiction","and you will know my name is the Lord when I lay my vengeance upon thee.")
28. time.sleep(2)
29. client1.loop()
30. time.sleep(2)
31. client1.disconnect()

GeSHi © Codebox Plus Extension

Thanks,
Balint Major

[ESP_cermak]

Thanks for this testing and sharing the results.
This is really strange, can you please share which version of mosquitto you use?
Ive had some trouble with older versions, but generally worked ok once a python script (using paho lib) could connect.

I seems you cannot connect to mosquitto on your end, even in the latest idf. This very basic scenario is tested in CI on every commit, so I would suspect your installation; but as you're saying python script (from another pc) connects and your firewall is off.

If you had a wrong certificate, you wouldn't connect, but with different error message (see your test2), with latest idf you could even see an error description of certificate verification.

[balint603]

Sorry for the late reply.
I am using 1.4.15-2ubuntu0.18.04.2 .
After reinstall mosquitto:
E (23524) TRANS_SSL: mbedtls_net_connect returned -44
I (23524) mbedtls: ssl_tls.c:7592 => write close notify

I (23524) mbedtls: ssl_tls.c:7608 <= write close notify

E (23524) MQTT_CLIENT: Error transport connect

I have not changed the mosquitto config file!

[ESP_cermak]

Looks like a totally different error, now I'd suspect if you switched off the firewall, the connection would work. This error says you cannot initiate a tcp connection to given host/port, before ssl handshake takes place.

In case you still cannot connect, please share also mosqutto configuration and logs from the server and use idf master for better reference and to rule out some older issues.