ESP32 OTA mqtt_client invalid header error

[saikiran]

hello i am working on esp32-ota using thingsboard i got an error that mqtt_client receives invalid header...

[saikiran]

I (16231) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
I (17341) MQTT_CLIENT: Sending MQTT CONNECT message, type: 1, id: 0000
I (18231) tb_ota: Connected to MQTT broker mqtt://cloud.thingsboard.io, on port 1883
I (18231) tb_ota: Waiting for shared attributes response
E (18631) MQTT_CLIENT: mqtt_message_receive: received a message with an invalid header=0x42
E (18631) MQTT_CLIENT: mqtt_process_receive: mqtt_message_receive() returned -1
W (19231) tb_ota: WAIT_OTA_CONFIG_FETCHED state, MQTT not connected, wait for the connect
W (20231) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect
W (21231) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect
W (22231) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect
W (23231) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect

i am getting this error in ota using thingsboard

[ESP_Angus]

(Moderator's note: Split to a new thread.)

[z_space]

hello,

Did you solve this issue? I am having the same exact problem.

Code: Select all

I (83925) MQTT_CLIENT: Sending MQTT CONNECT message, type: 1, id: 0000
I (84845) tb_ota: Connected to MQTT broker mqtt://****, on port 1883
I (84845) tb_ota: Waiting for shared attributes response
free message
free message
free message
E (84885) MQTT_CLIENT: mqtt_message_receive: received a message with an invalid header=0x42
E (84885) MQTT_CLIENT: mqtt_process_receive: mqtt_message_receive() returned -1
W (85845) tb_ota: WAIT_OTA_CONFIG_FETCHED state, MQTT not connected, wait for the connect
W (86845) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect
W (87845) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect
W (88845) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect
W (89845) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect
W (90845) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect
W (91845) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect
W (92845) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect

[Palonso]

Hi, I'm having the same issue but the header is other number. I'll leave the answer below.

Code: Select all

I (3580) tcpip_adapter: sta ip: 192.168.86.250, mask: 255.255.255.0, gw: 192.168.86.1
I (3580) MQTTS_SAMPLE: [APP] Free memory: 236108 bytes
I (3580) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
I (3830) MQTT_CLIENT: Sending MQTT CONNECT message, type: 1, id: 0000
I (4140) MQTTS_SAMPLE: MQTT_EVENT_CONNECTED
I (4140) MQTTS_SAMPLE: sent subscribe successful, msg_id=41920
E (4340) MQTT_CLIENT: mqtt_message_receive: received a message with an invalid header=0x92
E (4340) MQTT_CLIENT: mqtt_process_receive: mqtt_message_receive() returned -1
I (4350) MQTTS_SAMPLE: MQTT_EVENT_DISCONNECTED

And from there it start looping outputting the last 5 lines.

I'm using Thingsboard MQTT broker, and the best explanation (in my case) I found is that due to format on MQTT the header should be 0x90 but just the first quartet is important (the 9 in 0x92), the second quartet (the 2 in the 0x92) is another configuration that is being sent in the header. That's the difference in thingsboard MQTT broker and other brokers.

I think the problem is in mqtt_message_receive function (or the parameter that are being sent to it) but I'm not sure how to fix that. does any one have an idea?

[Palonso]

UPDATE: I came up with an idea to avoid the header problem.

I found that the headers are compared/analysed in a function called: int mqtt_has_valid_msg_hdr(uint8_t *buffer, uint16_t length)
which is located in $HOME/esp/esp-idf/components/mqtt/esp-mqtt/lib/mqtt_msg.c file.

Code: Select all

int mqtt_has_valid_msg_hdr(uint8_t *buffer, uint16_t length)
{
    int qos, dup;

    if (length < 1) {
        return 0;
    }
    switch (mqtt_get_type(buffer)) {
    case MQTT_MSG_TYPE_CONNECT:     //0x10
    case MQTT_MSG_TYPE_CONNACK:     //0x20
    case MQTT_MSG_TYPE_PUBACK:      //0x40
    case MQTT_MSG_TYPE_PUBREC:      //0x50
    case MQTT_MSG_TYPE_PUBCOMP:     //0x70
    case MQTT_MSG_TYPE_PINGREQ:     //0xC0
    case MQTT_MSG_TYPE_PINGRESP:    //0xD0
    case MQTT_MSG_TYPE_DISCONNECT:  //0xE0
        return (buffer[0] & 0x0f) == 0;  /* all flag bits are 0 */
//-----------------These are the headers that i had problems------------------
    case MQTT_MSG_TYPE_SUBACK:      //0x90
    case MQTT_MSG_TYPE_UNSUBACK:    //0xB0
        if ((buffer[0] & 0x0f) == 0)
        {
          return (buffer[0] & 0x0f) == 0;
        }
        else if ((buffer[0] & 0x0f) == 0x02)
        {
          return (buffer[0] & 0x0f) == 0x02;  /* only bit 1 is set */
        }
        else
          return 0;
//--------------------------------End of my "solution"---------------------------------
    case MQTT_MSG_TYPE_PUBREL:      //0x60
    case MQTT_MSG_TYPE_SUBSCRIBE:   //0x80
    case MQTT_MSG_TYPE_UNSUBSCRIBE: //0xA0
        return (buffer[0] & 0x0f) == 0x02;  /* only bit 1 is set */
    case MQTT_MSG_TYPE_PUBLISH:     //0x30
        qos = mqtt_get_qos(buffer);
        dup = mqtt_get_dup(buffer);
        /*
         * there is no qos=3  [MQTT-3.3.1-4]
         * dup flag must be set to 0 for all qos=0 messages [MQTT-3.3.1-2]
         */
        return (qos < 3) && ((qos > 0) || (dup == 0));
    default:
        return 0;
    }
}

Basically what i did is adding the headers that i receive (assuming they are correct) and filtered them as correct.
Now the problem is that the message is not published, it's supposed that it passes the headers problem but looks like is not working the publish function itself.

PS: In order to check the headers i changed the code from $HOME/esp/esp-idf/components/mqtt/esp-mqtt/mqtt_client.c in the function static int mqtt_message_receive(esp_mqtt_client_handle_t client, int read_poll_timeout_ms) and added this lines:

Code: Select all

/*
         * Verify the flags and act according to MQTT protocol: close connection
         * if the flags are set incorrectly.
         */

        ESP_LOGE(TAG, "buf=0x%x y buf_type = 0x%x", *buf, mqtt_get_type(buf));
        
        if (!mqtt_has_valid_msg_hdr(buf, read_len)) {
            ESP_LOGE(TAG, "%s: received a message with an invalid header=0x%x", __func__, *buf);
            goto err;
        }

I hope someone comes with an idea of how to fix this problem of publish in Thingsboard

[saikiran]

hello,

Did you solve this issue? I am having the same exact problem.

Code: Select all

I (83925) MQTT_CLIENT: Sending MQTT CONNECT message, type: 1, id: 0000
I (84845) tb_ota: Connected to MQTT broker mqtt://****, on port 1883
I (84845) tb_ota: Waiting for shared attributes response
free message
free message
free message
E (84885) MQTT_CLIENT: mqtt_message_receive: received a message with an invalid header=0x42
E (84885) MQTT_CLIENT: mqtt_process_receive: mqtt_message_receive() returned -1
W (85845) tb_ota: WAIT_OTA_CONFIG_FETCHED state, MQTT not connected, wait for the connect
W (86845) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect
W (87845) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect
W (88845) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect
W (89845) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect
W (90845) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect
W (91845) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect
W (92845) tb_ota: WAIT_MQTT state, MQTT not connected, wait for the connect

[saikiran]

You have install esp-idf v3.2 then invalid header will solved

[bcanustundag]

esp-idf v3.2 doesn't have these "message receive" and "check valid" methods.

I've also encountered with the same problem and came up with the same workaround.

Code: Select all

    case MQTT_MSG_TYPE_CONNACK:
        return (buffer[0] & 0x0f) == 0;
    case MQTT_MSG_TYPE_SUBACK:
        return (buffer[0] & 0x0f) == 0x02;

[saikiran]

Sorry, install esp-idf v3.3