HID Keyboard Example Crashes Windows Bluetooth Driver

[gnorkus]

Well. Problem NOT solved...It just takes longer for it to repeat. I'm assuming the stack size was the 3rd parameter in the xTaskCreate function call...

1. From a full reboot, make sure that the Esp32 HID keyboard (EnvisicPedal) is not in the Windows Settings... Bluetooth & other devices list. If it is, remove the device from the list and restart the OS.

2. Connect the ESP32 device to power and go to "Add Bluetooth or other device" on the Windows Settings...Bluetooth and other devices page.

3. Bring up an editor and press the GPIO 0 button a few times to see if it is working, sending 'z' characters.

4. Unplug the ESP32 device from power and wait for the Windows Settings...Bluetooth & other devices list to show
EnvisicPedal as Paired (instead of Connected). It takes about 1-2 seconds for this to happen.

5. Wait about 40 seconds.

6. Replug the ESP32 device to power. The EnvisicPedal may show connected again. Press the button a few times, and it will be working.

7. After a few minutes, all the bluetooth devices will stop workng on Windows. This will require a reboot to fix. It may also cause the device to start connecting and then disconnecting, or the button will not send keystrokes anymore...

Changing the stack size causes the problem to delay in occurring, but eventually it does. I have no clue as to what this problem is underneath. I'm just getting my way around in starting debugging with an OpenOCD jtag debugger and Eclipse. Should I be concerned about "correct" way of gracefully shutting down the esp32 so that the Windows Bluetooth driver doesn't lose it's mind? I can implement this when I do my shutdown sequence before going into deep sleep. Or is that somewhere buried in the Esp32 HID/BLE code as to why it still can hang the Bluetooth driver.

Should I invest in a different bluetooth stack?

[chegewara]

Its hard to say if you should have invest into new stack, but here is one in C:
https://github.com/bluekitchen/btstack/ ... er/example
I dont know it so i cant say how good it is.

and one in C++:
https://github.com/nkolban/esp32-snippe ... s/BLETests
here you will have to write procedure to handle key strokes or key inputs.

[gnorkus]

Oops! I meant to say "invest time", not "invest". Silly keyboard autocorrect!

MUCH SIMPLER WAY TO REPRODUCE PROBLEM
1. Attach a normal Bluetooth keyboard to a Windows computer.
2. Using the code provided with the correction in the xTaskCreate (see below), simply plug the ESP32 device in and connect it via Windows Bluetooth settings. No need to press the button and send keystrokes.
3. Just type for a while with the normal Bluetooth keyboard. Eventually the Windows Bluetooth driver will hang. The problem will happen sooner if you use a smaller stack size.

It seems that there might be some interaction between the bluedroid stack and Windows that's not being reported, or that the connected ESP32 might not be responding to messages that Windows might be sending. Does that sound logical? How do I get the log to show what's happening in bluedroid, not using OpenOCD? Just putting in more trace lines (i.e. LOG_ERROR) might not reveal this for me...

Code: Select all

// Copyright 2015-2016 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at

//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// Adaptions done:
// Copyright 2017 Benjamin Aigner <beni@asterics-foundation.org>

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/event_groups.h"
#include "esp_system.h"
#include "esp_wifi.h"
#include "esp_event_loop.h"
#include "esp_log.h"
#include "nvs_flash.h"
#include "esp_bt.h"

#include "config.h"

#include "esp_hidd_prf_api.h"
#include "esp_bt_defs.h"
#include "esp_gap_ble_api.h"
#include "esp_gatts_api.h"
#include "esp_gatt_defs.h"
#include "esp_bt_main.h"
#include "esp_bt_device.h"
#include "bt_trace.h"
#include "driver/gpio.h"
#include "driver/uart.h"
#include "hid_dev.h"
#include "keyboard.h"
#include "btm_ble_api.h"

#define GATTS_TAG "EnvisicPedal"

static uint16_t hid_conn_id = 0;
static bool sec_conn = false;
static config_data_t config;


#define CHAR_DECLARATION_SIZE   (sizeof(uint8_t))

static void hidd_event_callback(esp_hidd_cb_event_t event, esp_hidd_cb_param_t *param);

const char hid_device_name_fabi[] = "EnvisicPedal";
static uint8_t hidd_service_uuid128[] = {
	/* LSB <--------------------------------------------------------------------------------> MSB */
	//first uuid, 16bit, [12],[13] is the value
	0xfb, 0x34, 0x9b, 0x5f, 0x80, 0x00, 0x00, 0x80, 0x00, 0x10, 0x00, 0x00, 0x12, 0x18, 0x00, 0x00,
};

static esp_ble_adv_data_t hidd_adv_data = {
	.set_scan_rsp = false,
	.include_name = true,
	.include_txpower = true,
	.min_interval = 0x20,
	.max_interval = 0x30,
	.appearance = BTM_BLE_APPEARANCE_HID_KEYBOARD,       //HID Keyboard 0x03C1
	.manufacturer_len = 0,
	.p_manufacturer_data = NULL,
	.service_data_len = 0,
	.p_service_data = NULL,
	.service_uuid_len = sizeof(hidd_service_uuid128),
	.p_service_uuid = hidd_service_uuid128,
	.flag = 0x6,
};

static esp_ble_adv_params_t hidd_adv_params = {
	.adv_int_min = 0x20,
	.adv_int_max = 0x30,
	.adv_type = ADV_TYPE_IND,
	.own_addr_type = BLE_ADDR_TYPE_PUBLIC,
	//.peer_addr            =
	//.peer_addr_type       =
	.channel_map = ADV_CHNL_ALL,
	.adv_filter_policy = ADV_FILTER_ALLOW_SCAN_ANY_CON_ANY,
};


static void hidd_event_callback(esp_hidd_cb_event_t event, esp_hidd_cb_param_t *param)
{
	switch (event) {
	case ESP_HIDD_EVENT_REG_FINISH: {
		if (param->init_finish.state == ESP_HIDD_INIT_OK) {
			//esp_bd_addr_t rand_addr = {0x04,0x11,0x11,0x11,0x11,0x05};
			esp_ble_gap_set_device_name(hid_device_name_fabi);
			esp_ble_gap_config_adv_data(&hidd_adv_data);

		}
		break;
	}
	case ESP_BAT_EVENT_REG: {
		break;
	}
	case ESP_HIDD_EVENT_DEINIT_FINISH:
		break;
	case ESP_HIDD_EVENT_BLE_CONNECT: {
		hid_conn_id = param->connect.conn_id;
		sec_conn = true; //TODO: right here?!?
		LOG_ERROR("%s(), ESP_HIDD_EVENT_BLE_CONNECT", __func__);
		break;
	}
	case ESP_HIDD_EVENT_BLE_DISCONNECT: {
		sec_conn = false;
		hid_conn_id = 0;
		LOG_ERROR("%s(), ESP_HIDD_EVENT_BLE_DISCONNECT", __func__);
		esp_ble_gap_start_advertising(&hidd_adv_params);
		break;
	}
	default:
		break;
	}
	return;
}

static void gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_gap_cb_param_t *param)
{
	switch (event) {
	case ESP_GAP_BLE_ADV_DATA_SET_COMPLETE_EVT:
		esp_ble_gap_start_advertising(&hidd_adv_params);
		break;
		/*case ESP_GAP_BLE_SEC_REQ_EVT:
		for(int i = 0; i < ESP_BD_ADDR_LEN; i++) {
		LOG_DEBUG("%x:",param->ble_security.ble_req.bd_addr[i]);
		}
		esp_ble_gap_security_rsp(param->ble_security.ble_req.bd_addr, true);
		break;*/
	case ESP_GAP_BLE_AUTH_CMPL_EVT:
		sec_conn = true;
		if (param->ble_security.auth_cmpl.success)
			LOG_INFO("status = success, ESP_GAP_BLE_AUTH_CMPL_EVT");
		else
			LOG_INFO("status = fail, ESP_GAP_BLE_AUTH_CMPL_EVT");
		break;
		//unused events 
	case ESP_GAP_BLE_ADV_START_COMPLETE_EVT: break;
		//do we need this? occurs on win10 connect.
	case ESP_GAP_BLE_UPDATE_CONN_PARAMS_EVT: break;

	case ESP_GAP_BLE_PASSKEY_REQ_EVT:                           /* passkey request event */
																//esp_ble_passkey_reply(gl_profile_tab[PROFILE_A_APP_ID].remote_bda, true, 0x00);
		LOG_INFO("ESP_GAP_BLE_PASSKEY_REQ_EVT");
		break;
	case ESP_GAP_BLE_OOB_REQ_EVT:                                /* OOB request event */
		LOG_INFO("ESP_GAP_BLE_OOB_REQ_EVT");
		break;
	case ESP_GAP_BLE_LOCAL_IR_EVT:                               /* BLE local IR event */
		LOG_INFO("ESP_GAP_BLE_LOCAL_IR_EVT");
		break;
	case ESP_GAP_BLE_LOCAL_ER_EVT:                               /* BLE local ER event */
		LOG_INFO("ESP_GAP_BLE_LOCAL_ER_EVT");
		break;
	case ESP_GAP_BLE_NC_REQ_EVT:
		LOG_INFO("ESP_GAP_BLE_NC_REQ_EVT");
		break;
	case ESP_GAP_BLE_SEC_REQ_EVT:
		/* send the positive(true) security response to the peer device to accept the security request.
		If not accept the security request, should sent the security response with negative(false) accept value*/
		esp_ble_gap_security_rsp(param->ble_security.ble_req.bd_addr, true);
		LOG_INFO("ESP_GAP_BLE_SEC_REQ_EVT");
		break;

	case ESP_GAP_BLE_PASSKEY_NOTIF_EVT:  ///the app will receive this evt when the IO  has Output capability and the peer device IO has Input capability.
										 ///show the passkey number to the user to input it in the peer deivce.
		LOG_INFO("The passkey Notify number:%d", param->ble_security.key_notif.passkey);
		break;
	case ESP_GAP_BLE_KEY_EVT:
		//shows the ble key info share with peer device to the user.
		LOG_INFO("key type = %d", param->ble_security.ble_key.key_type);
		break;

	default:
		LOG_WARN("unhandled event: %d", event);
		break;
	}
}

void update_config()
{
	nvs_handle my_handle;
	esp_err_t err = nvs_open("envisic_c", NVS_READWRITE, &my_handle);
	if (err != ESP_OK) ESP_LOGE("MAIN", "error opening NVS");
	err = nvs_set_u8(my_handle, "btname_i", config.bt_device_name_index);
	if (err != ESP_OK) ESP_LOGE("MAIN", "error saving NVS - bt name");
	err = nvs_set_u8(my_handle, "locale", config.locale);
	if (err != ESP_OK) ESP_LOGE("MAIN", "error saving NVS - locale");
	printf("Committing updates in NVS ... ");
	err = nvs_commit(my_handle);
	printf((err != ESP_OK) ? "Failed!\n" : "Done\n");
	nvs_close(my_handle);
}

void button_reader(void *pvParameters)
{
	static uint8_t keycode = 0;
	long butlevel;
	static long lastlevel = 1;

	gpio_set_direction(GPIO_NUM_0, GPIO_MODE_INPUT);

	while (1)
	{
		butlevel = gpio_get_level(GPIO_NUM_0);
		if (butlevel == 0 && (butlevel != lastlevel))
		{
			//vTaskDelay(1000 / portTICK_PERIOD_MS);

			// Send the letter 'z' to the connected device
			keycode = 29;
			esp_hidd_send_keyboard_value(hid_conn_id, 0, &keycode, 1);
			keycode = 0;
			esp_hidd_send_keyboard_value(hid_conn_id, 0, &keycode, 1);
		}

		lastlevel = butlevel;
		vTaskDelay(50 / portTICK_PERIOD_MS);
	}
}


void app_main()
{
	esp_err_t ret;

	// Initialize NVS.
	ret = nvs_flash_init();
	if (ret == ESP_ERR_NVS_NO_FREE_PAGES) {
		ESP_ERROR_CHECK(nvs_flash_erase());
		ret = nvs_flash_init();
	}
	ESP_ERROR_CHECK(ret);

	// Read config
	nvs_handle my_handle;
	ret = nvs_open("envisic_c", NVS_READWRITE, &my_handle);
	if (ret != ESP_OK) ESP_LOGE("MAIN", "error opening NVS");
	ret = nvs_get_u8(my_handle, "btname_i", &config.bt_device_name_index);
	if (ret != ESP_OK)
	{
		ESP_LOGE("MAIN", "error reading NVS - bt name, setting to Envisic");
		config.bt_device_name_index = 0;
	}
	ret = nvs_get_u8(my_handle, "locale", &config.locale);
	if (ret != ESP_OK || config.locale >= LAYOUT_MAX)
	{
		ESP_LOGE("MAIN", "error reading NVS - locale, setting to US_INTERNATIONAL");
		config.locale = LAYOUT_US_INTERNATIONAL;
	}
	nvs_close(my_handle);


	esp_bt_controller_config_t bt_cfg = BT_CONTROLLER_INIT_CONFIG_DEFAULT();
	ret = esp_bt_controller_init(&bt_cfg);
	if (ret) {
		ESP_LOGE(GATTS_TAG, "%s initialize controller failed\n", __func__);
		return;
	}

	ret = esp_bt_controller_enable(ESP_BT_MODE_BLE);
	if (ret) {
		ESP_LOGE(GATTS_TAG, "%s enable controller failed\n", __func__);
		return;
	}

	ret = esp_bluedroid_init();
	if (ret) {
		LOG_ERROR("%s init bluedroid failed\n", __func__);
		return;
	}

	ret = esp_bluedroid_enable();
	if (ret) {
		LOG_ERROR("%s init bluedroid failed\n", __func__);
		return;
	}

	//load HID country code for locale before initialising HID
	hidd_set_countrycode(get_hid_country_code(config.locale));

	if ((ret = esp_hidd_profile_init()) != ESP_OK) {
		LOG_ERROR("%s init bluedroid failed\n", __func__);
	}

	///register the callback function to the gap module
	esp_ble_gap_register_callback(gap_event_handler);
	esp_hidd_register_callbacks(hidd_event_callback);

	/* set the security iocap & auth_req & key size & init key response key parameters to the stack*/
	esp_ble_auth_req_t auth_req = ESP_LE_AUTH_BOND;     //bonding with peer device after authentication
														/** Do not use "NONE", HID over GATT requires something more than NONE */
	esp_ble_io_cap_t iocap = ESP_IO_CAP_NONE;           //set the IO capability to No output No input
														/** CAP_OUT & CAP_IO work with Winsh***t, but you need to enter a pin which is shown in "make monitor" */
														//esp_ble_io_cap_t iocap = ESP_IO_CAP_OUT;           //set the IO capability to No output No input
														//esp_ble_io_cap_t iocap = ESP_IO_CAP_IO;           //set the IO capability to No output No input
														/** CAP_IN: host shows you a pin, you have to enter it (unimplemented now) */
														//esp_ble_io_cap_t iocap = ESP_IO_CAP_IN;           //set the IO capability to No output No input

	uint8_t key_size = 16;      //the key size should be 7~16 bytes
	uint8_t init_key = ESP_BLE_ENC_KEY_MASK | ESP_BLE_ID_KEY_MASK;
	uint8_t rsp_key = ESP_BLE_ENC_KEY_MASK | ESP_BLE_ID_KEY_MASK;
	esp_ble_gap_set_security_param(ESP_BLE_SM_AUTHEN_REQ_MODE, &auth_req, sizeof(uint8_t));
	esp_ble_gap_set_security_param(ESP_BLE_SM_IOCAP_MODE, &iocap, sizeof(uint8_t));
	esp_ble_gap_set_security_param(ESP_BLE_SM_MAX_KEY_SIZE, &key_size, sizeof(uint8_t));
	/* If your BLE device act as a Slave, the init_key means you hope which types of key of the master should distribut to you,
	and the response key means which key you can distribut to the Master;
	If your BLE device act as a master, the response key means you hope which types of key of the slave should distribut to you,
	and the init key means which key you can distribut to the slave. */
	esp_ble_gap_set_security_param(ESP_BLE_SM_SET_INIT_KEY, &init_key, sizeof(uint8_t));
	esp_ble_gap_set_security_param(ESP_BLE_SM_SET_RSP_KEY, &rsp_key, sizeof(uint8_t));

	xTaskCreate(&button_reader, "buttonread", 8192, NULL, tskIDLE_PRIORITY+1, NULL);
}

[Vader_Mester]

To me it looks like that the ESP is not receiving any data. This can be caused by the sending tasks locking up the ESP making it unable to handle received data.

[gnorkus]

Thx... The ESP32 isn't locking up, the Windows 10 bluetooth driver is.

[chegewara]

Can you test your esp32 hid app with android device and nRF connect? Can you use wireshark on windows and see what is going on?

[gnorkus]

Hi Chegewara,

Does that require I purchase a sniffer dongle (Ubertooth)? Or is nRF capable of using my Android cellphone as the sniffer?

Thanks,
GJN

[chegewara]

Hi,
i was not able to use wireshark on my laptop with integrated bluetooth card so i guess you need to buy dongle, i have regular dongle for $7-10, to check whats going on with nRF you dont need anything but android smartphone. In this case you can grab only information about services, characteristics, descriptors and its values, and eventually basic info about connection.
With sniffer you can get more info about packets exchange and if you know what you are looking for you may find issue.

[Vader_Mester]

While you keep testing the connection, i would suggest to change something else, which would be to separate the button reading and BT sending tasks.

A small ringbuffer (maximum 32bytes) should be allocated where the keyboard buttons are read into, and you just send this buffer through BT.
FreeRTOS has a Ringbuffer API which you could use.
Depending on the latency requirements you should read this buffer 20-30times a second and send the content. This is pretty fast but it is timed, and between the send events the other BT tasks should be managed.
Timing can be done with the FreeRTOS ticks and software timers.
By separsting the 2 you have the option to see which is not working correctly. For testing the BT connection you can continuously fill the ring buffer with keycode data which is sent to the windows driver (no key reading yet) so you will see what is going on there.

[gnorkus]

Hi,

Has anyone tried the Microsoft Message Analyzer for HID Keyboard debugging?

GJN