New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ESP32 BT-HID Device #230

Open
gitmh opened this Issue Nov 26, 2017 · 101 comments

Comments

Projects
None yet
@gitmh

gitmh commented Nov 26, 2017

Is it possible to use the ESP32_BLE_Arduino library to build a HID-Keyboard to send keystrokes to another device? My goal: I would like to connect a ESP32 board to a BT device like an iPad and send a space character. I found https://github.com/asterics/esp32_mouse_keyboard but can't figure out how to this running in Arduino IDE. Any help would be appreciated - I found a lot of users asking for a similar feature for the ESP32.

@nkolban

This comment has been minimized.

Owner

nkolban commented Nov 26, 2017

I'm afraid there is nothing yet in the BLE C++ classes that would make your request possible. Maybe a question posted to one of the forums here might give some positive results:

https://esp32.com/index.php

It is my loose understanding that HID support is based on Bluetooth classic (BR/EDR) as opposed to BLE and, so far, these classes only cover BLE functions.

@gitmh

This comment has been minimized.

gitmh commented Nov 26, 2017

Thank you very much @nkolban. I already found a project for esp-idf for HID and BT but I was hoping for an example for the arduino IDE. I have no experience with IDF and the tool chain and how to integrate those stuff with arduino. Maybe this will come in the future.

@nkolban

This comment has been minimized.

Owner

nkolban commented Nov 26, 2017

Do you have a link to the HID and BT project that uses ESP-IDF? I'd like to file that and have a look to see what it would take to eventually encapsulate that in the C++ libraries. That would be the first step towards an Arduino encapsulation.

@gitmh

This comment has been minimized.

gitmh commented Nov 27, 2017

The most complete and supported solution seems to be the BTStack port for the esp32 for IDF here
https://github.com/bluekitchen/btstack/tree/master/port/esp32 within this library there are various examples for HID devices and other BT applications.

@nkolban

This comment has been minimized.

Owner

nkolban commented Nov 27, 2017

OOOh thats cool. But scary as well. If I get a sense of that story, it is a replacement for the Bluetooth stack supplied by Espressif (which I believe is based on Bluedroid stack implementation).

@gitmh

This comment has been minimized.

gitmh commented Nov 28, 2017

Yes so this might be a problem to have both stacks available at the same time in an Arduino library. The example under https://github.com/asterics/esp32_mouse_keyboard uses the bt.h from the regular Espressif library so (see ble_hidd_demo_main.c).

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Dec 14, 2017

New day, new lessons:
hid

My esp32 just become HID keyboard. ANd now its HID mouse:
mouse

Of course its not real keyboard and mouse, but my windows 10 is stupid and thinks it is ;)

@gitmh

This comment has been minimized.

gitmh commented Dec 14, 2017

@chegewara Have you build an Arduino project for that already? I would be interested :-)
Sending a keyboard key to any BT device like an iPAD would be a great use case for the ESP32.

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Dec 14, 2017

Oh no, just fooled my laptop by setting appearance(). The rest of code has nothing to do with hid keyboard. But i may do some research and see if this is hard to do.

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Dec 15, 2017

I have some good news and some bad news. Good news that work is in progress. Bad news is driver crash under windows with error code 0x0a(device cant start). I can say for sure what is the reason, but its a chance that hid keybord requires secure connection and we dont have implemented it yet.

My next step will be to rewrite code in esp-idf and see what happen and maybe add some security there.

EDIT more good news, my samsung s4 connects to esp32 hid keyboard, so now i have to find way to send some text

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Dec 15, 2017

I have good news. Esp32 hid keyboard is connecting with my android phone and i can send text. This mean hid keyboard works, at least software part. Still is some work to do but we know it can be done.

@gitmh

This comment has been minimized.

gitmh commented Dec 15, 2017

@chegewara great work, would you share your current project to play around with it? I just need a running starting point for the Arduino IDE :-)

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Dec 15, 2017

Im not sure yet, i will provide code or share links ive been using during research. Im thinking about writing blog article about this. For now maybe this will help you a bit:
https://github.com/I0x0I/DIY-A-BLE-Keyboard

https://docs.mbed.com/docs/ble-hid/en/latest/api/md_doc_HIDService.html

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Dec 15, 2017

This is very important documentation if you want to play with HID devices. Appendix E gives some good look at what descriptors are required to setup hid device:
http://www.usb.org/developers/hidpage/HID1_11.pdf

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Dec 15, 2017

Services and characteristics:

This XML file does not appear to have any style information associated with it. The document tree is shown below.
<configuration>
<service uuid="1800">
<description>Generic Access Profile</description>
<characteristic uuid="2a00">
<properties read="true" const="true"/>
<value>BGT Keyboard Demo</value>
</characteristic>
<characteristic uuid="2a01">
<properties read="true" const="true"/>
<value type="hex">c103</value>
</characteristic>
</service>
<service type="primary" uuid="180A" id="manufacturer">
<characteristic uuid="2A29">
<properties read="true" const="true"/>
<value>Bluegiga</value>
</characteristic>
<characteristic uuid="2A50">
<!--  PnP ID required by HID Profile  -->
<properties read="true" const="true"/>
<value type="hex">014700ffffffff</value>
</characteristic>
</service>
<service uuid="180f">
<description>Battery</description>
<characteristic uuid="2a19" id="xgatt_battery">
<properties read="true"/>
<value type="hex">32</value>
</characteristic>
</service>
<service uuid="1812" advertise="true">
<characteristic uuid="2a4d" id="hid_keyboard_in">
<!--  Keyboard input report  -->
<properties notify="true" read="true"/>
<value length="20" variable_length="true"/>
<descriptor uuid="2908">
<!--  Report reference id=0x00 type=0x01 (input)  -->
<properties read="true" const="true"/>
<value type="hex">0001</value>
</descriptor>
</characteristic>
<characteristic uuid="2a4d" id="hid_keyboard_out">
<!--  Keyboard output report  -->
<properties write="true" write_no_response="true" read="true"/>
<value length="20" variable_length="true"/>
<descriptor uuid="2908">
<!--  Report reference id=0x00 type=0x02 (output)  -->
<properties read="true" const="true"/>
<value type="hex">0002</value>
</descriptor>
</characteristic>
<characteristic uuid="2a4d" id="hid_keyboard_feature">
<!--  Keyboard feature report  -->
<properties write="true" read="true"/>
<value length="20" variable_length="true"/>
<descriptor uuid="2908">
<!--  Report reference id=0x00 type=0x03 (feature)  -->
<properties read="true" const="true"/>
<value type="hex">0003</value>
</descriptor>
</characteristic>
<characteristic uuid="2a4b">
<!--  Report map example from USB HID Specification  -->
<properties read="true" authenticated_read="true" const="true"/>
<value type="hex">
05010906A101050719E029E71500250175019508810295017508810195057501050819012905910295017503910195067508150025650507190029658100C0
</value>
</characteristic>
<characteristic uuid="2a4a">
<!--
 HID Information version=0x0100 countrycode=0x00 flags=0x02 (normally connectable) 
-->
<properties read="true" const="true"/>
<value type="hex">00010002</value>
</characteristic>
<characteristic uuid="2a4c" id="hid_control">
<!--  HID Control point, used to control suspending  -->
<properties write_no_response="true"/>
<value length="1"/>
</characteristic>
<characteristic uuid="2a22" id="hid_boot_keyboard_in">
<!--  Boot Keyboard input report  -->
<properties notify="true" read="true"/>
<value length="20" variable_length="true"/>
</characteristic>
<characteristic uuid="2a32" id="hid_boot_keyboard_out">
<!--  Boot Keyboard output report  -->
<properties write_no_response="true" read="true" write="true"/>
<value length="20" variable_length="true"/>
</characteristic>
<characteristic uuid="2a4e" id="hid_protocol_mode">
<!--  Protocol mode select  -->
<properties write_no_response="true" read="true"/>
<value length="1"/>
</characteristic>
</service>
</configuration>
@gitmh

This comment has been minimized.

gitmh commented Dec 15, 2017

@chegewara I managed to run the BlueKitchen ESP32 Port for ESP-IDF (https://github.com/bluekitchen/btstack/blob/master/example/hid_keyboard_demo.c) it runs on my ESP32 board and sends continuously text messages to may mac book and to my android phone as soon as you pair them. The whole setup for the ESP-IDF and libraries is quite complicated so I cannot use this process for teaching. This is why I was looking for a nice wrapper for the Arduino IDE so that you can fire up a example project, hit flash and project runs :-)

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Dec 16, 2017

Can you wait few days, im still working on arduino code. At the moment im just sending random letter but i still missing one thing. My code requires that notifications to be turned on on client side (my android phone). At the moment only way to achieve this is to open nRF connect, connect to esp32 hid and turn on notifications.

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Dec 16, 2017

This is what i did so far. There is still a lot to do but it should send random character in loop:

#include <BLEDevice.h>
#include <BLEUtils.h>
#include <BLEServer.h>
#include <BLE2902.h>
#include <string>

// See the following for generating UUIDs:
// https://www.uuidgenerator.net/

bool connected = false;
  BLEService *pService;
  BLEService *pService1;
  BLECharacteristic *reportChar1;
  BLECharacteristic *reportChar2;
  BLECharacteristic *reportChar3;
  class MyCallbacks : public BLEServerCallbacks {
    void onConnect(BLEServer* pServer){
      connected=true;
    }
    
    void onDisconnect(BLEServer* pServer){
      connected=false;
    }
  };
  
void setup() {
  Serial.begin(115200);
  Serial.println("Starting BLE work!");

  BLEDevice::init("ESP32");
  BLEServer *pServer = BLEDevice::createServer();
  pServer->setCallbacks(new MyCallbacks());
  pService = pServer->createService((uint16_t)0x180a);
  pService1 = pServer->createService((uint16_t)0x1812, 30);
  setupCharacteristics();

  pService->start();
  pService1->start();
  BLEAdvertising *pAdvertising = pServer->getAdvertising();
  pAdvertising->setAppearance(961);
  pAdvertising->addServiceUUID((uint16_t)0x1812);
  pAdvertising->start();

  Serial.println("Characteristic defined! Now you can read it in your phone!");
}
uint8_t v[] = {0x0, 0x0, 0x0, 0x0,0x0,0x0,0x0,0x0};

void loop() {
  // put your main code here, to run repeatedly:
  delay(5000);
  if(connected){
    uint8_t a[] = {0x0, 0x0, random(0x04,0x26), 0x0,0x0,0x0,0x0,0x0};
    reportChar1->setValue(a,sizeof(a));
    reportChar1->notify();    

    reportChar1->setValue(v, sizeof(v));
    reportChar1->notify();
  }
}

void setupCharacteristics() {

  BLECharacteristic *manufacturer = pService->createCharacteristic(
                                       (uint16_t)0x2a29,
                                       BLECharacteristic::PROPERTY_READ 
                                     );
  std::string name = "espressif";
  manufacturer->setValue(name);
                                     
  BLECharacteristic *pnpIDChar = pService->createCharacteristic(
                                       (uint16_t)0x2a50,
                                       BLECharacteristic::PROPERTY_READ 
                                     );
  const uint8_t pnp[] = {0x01,0xe5,0x02,0xcd,0xab,0x01,0x00};
  pnpIDChar->setValue((uint8_t*)pnp, sizeof(pnp));
  
  BLECharacteristic *hidInfoChar = pService1->createCharacteristic(
                                       (uint16_t)0x2a4a,
                                       BLECharacteristic::PROPERTY_READ 
                                     );
  const uint8_t val1[] = {0x00,0x01,0x00,0x02};
  hidInfoChar->setValue((uint8_t*)val1, 4);

  BLECharacteristic *reportMapChar = pService1->createCharacteristic(
                                       (uint16_t)0x2a4b,
                                       BLECharacteristic::PROPERTY_READ 
                                     );
  const uint8_t val2[] = {
    0x05,0x01,0x09,0x06,0xA1,0x01,0x05,0x07,
    0x19,0xE0,0x29,0xE7,0x15,0x00,0x25,0x01,
    0x75,0x01,0x95,0x08,0x81,0x02,0x95,0x01,
    0x75,0x08,0x81,0x01,0x95,0x05,0x75,0x01,
    0x05,0x08,0x19,0x01,0x29,0x05,0x91,0x02,
    0x95,0x01,0x75,0x03,0x91,0x01,0x95,0x06,
    0x75,0x08,0x15,0x00,0x25,0x65,0x05,0x07,
    0x19,0x00,0x29,0x65,0x81,0x00,0xC0}; //TODO
  reportMapChar->setValue((uint8_t*)val2, sizeof(val2));

  BLECharacteristic *hidControlChar = pService1->createCharacteristic(
                                       (uint16_t)0x2a4c,
                                       BLECharacteristic::PROPERTY_WRITE_NR 
                                     );

  reportChar1 = pService1->createCharacteristic(
                                       (uint16_t)0x2a4d,
                                       BLECharacteristic::PROPERTY_READ   |
                                       BLECharacteristic::PROPERTY_NOTIFY                                       
                                     );
  BLEDescriptor *desc1 = new BLEDescriptor(BLEUUID((uint16_t)0x2908));
  const uint8_t desc1_val[] = {0x01, 0};
  desc1->setValue((uint8_t*)desc1_val, 1);
  reportChar1->addDescriptor(desc1);
  reportChar1->addDescriptor(new BLE2902());

  reportChar2 = pService1->createCharacteristic(
                                       (uint16_t)0x2a4d,
                                       BLECharacteristic::PROPERTY_READ   |
                                       BLECharacteristic::PROPERTY_WRITE                                       
                                     );
  BLEDescriptor *desc2 = new BLEDescriptor(BLEUUID((uint16_t)0x2908));
  const uint8_t desc2_val[] = {0x02, 0};
  desc2->setValue((uint8_t*)desc2_val, 1);
  reportChar2->addDescriptor(desc2);

  reportChar3 = pService1->createCharacteristic(
                                       (uint16_t)0x2a4d,
                                       BLECharacteristic::PROPERTY_READ     |
                                       BLECharacteristic::PROPERTY_WRITE    |
                                       BLECharacteristic::PROPERTY_WRITE_NR 
                                     );
  BLEDescriptor *desc3 = new BLEDescriptor(BLEUUID((uint16_t)0x2908));
  const uint8_t desc3_val[] = {0x03, 0};
  desc3->setValue((uint8_t*)desc3_val, 1);
  reportChar3->addDescriptor(desc3);

  BLECharacteristic *protocolModeChar = pService1->createCharacteristic(
                                       (uint16_t)0x2a4e,
                                       BLECharacteristic::PROPERTY_WRITE_NR 
                                     );

  BLECharacteristic *bootInputChar = pService1->createCharacteristic(
                                       (uint16_t)0x2a22,
                                       BLECharacteristic::PROPERTY_NOTIFY
                                     );
  bootInputChar->addDescriptor(new BLE2902());


  BLECharacteristic *bootOutputChar = pService1->createCharacteristic(
                                       (uint16_t)0x2a32,
                                       BLECharacteristic::PROPERTY_READ     |
                                       BLECharacteristic::PROPERTY_WRITE    |
                                       BLECharacteristic::PROPERTY_WRITE_NR 
                                     );



}
@gitmh

This comment has been minimized.

gitmh commented Dec 17, 2017

@chegewara Great work!!! It works like a charm on my Android phone. I can also connect the ESP32 with iOS and Mac OS, but no characters appear there - guess this is the security issue problem you were talking about. For testing I used two ESP32 Hellcat LoRa boards. One board is sending a string over LoRa, to the other board which is connected as HID device on Android transferring all input received as keystrokes.

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Dec 18, 2017

Sorry but my windows has Polish localization :(

hid

@ripper121

This comment has been minimized.

ripper121 commented Feb 13, 2018

Any news about this?
Would be super cool to use it as a AT/PS2 to BLE HID Keyboad adapter.

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Feb 13, 2018

@ripper121 What news do you need? Its working for few weeks now. There is one example added. If you have questions, issues or suggestions im open and ready to help.

@ripper121

This comment has been minimized.

ripper121 commented Feb 14, 2018

Is there a Arduino Lib for this or is it only possible with the SDK?

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Feb 14, 2018

It should work with arduino too. Last time i checked it did work with arduino.

@ripper121

This comment has been minimized.

ripper121 commented Feb 16, 2018

Yesss thx its working :)

@hmatulis

This comment has been minimized.

hmatulis commented Mar 22, 2018

Hi great work, I tried it on my windows 10, but couldn't manage to make it work.
First I got a "This device cannot start (Code 10)" and fixed it with https://www.kapilarya.com/fix-this-device-cannot-start-code-10-in-windows-10

After that I've got: "Driver error" - It connects and pairs ok, but doesn't work.

Monitor log:
Starting BLE work!
Characteristic defined! Now you can read it in your phone!
�[0;31mE (27800) BT: lmp_version_below LMP version 6 < 8�[0m
�[0;31mE (27860) BT: Call back not found for application conn_id=3�[0m
�[0;31mE (33978) BT: bta_gattc_conn_cback() - cif=3 connected=0 conn_id=3 reason=0x0013�[0m

any hint?
thanks

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Mar 22, 2018

Hi,
most likely, and im only guessing right now, its problem with report map. Its this part of code (example code):

		const uint8_t reportMap[] = {
			USAGE_PAGE(1),      0x01,       // Generic Desktop Ctrls
			USAGE(1),           0x06,       // Keyboard
			COLLECTION(1),      0x01,       // Application
			REPORT_ID(1),		0x01,		// REPORTID
			USAGE_PAGE(1),      0x07,       //   Kbrd/Keypad
			USAGE_MINIMUM(1),   0xE0,
			USAGE_MAXIMUM(1),   0xE7,
			LOGICAL_MINIMUM(1), 0x00,
			LOGICAL_MAXIMUM(1), 0x01,
			REPORT_SIZE(1),     0x01,       //   1 byte (Modifier)
			REPORT_COUNT(1),    0x08,
			INPUT(1),           0x02,       //   Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position
			REPORT_COUNT(1),    0x01,       //   1 byte (Reserved)
			REPORT_SIZE(1),     0x08,
			INPUT(1),           0x01,       //   Const,Array,Abs,No Wrap,Linear,Preferred State,No Null Position
			REPORT_COUNT(1),    0x05,       //   5 bits (Num lock, Caps lock, Scroll lock, Compose, Kana)
			REPORT_SIZE(1),     0x01,
			USAGE_PAGE(1),      0x08,       //   LEDs
			USAGE_MINIMUM(1),   0x01,       //   Num Lock
			USAGE_MAXIMUM(1),   0x05,       //   Kana
			OUTPUT(1),          0x02,       //   Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position,Non-volatile
			REPORT_COUNT(1),    0x01,       //   3 bits (Padding)
			REPORT_SIZE(1),     0x03,
			OUTPUT(1),          0x01,       //   Const,Array,Abs,No Wrap,Linear,Preferred State,No Null Position,Non-volatile
			REPORT_COUNT(1),    0x06,       //   6 bytes (Keys)
			REPORT_SIZE(1),     0x08,
			LOGICAL_MINIMUM(1), 0x00,
			LOGICAL_MAXIMUM(1), 0x65,       //   101 keys
			USAGE_PAGE(1),      0x07,       //   Kbrd/Keypad
			USAGE_MINIMUM(1),   0x00,
			USAGE_MAXIMUM(1),   0x65,
			INPUT(1),           0x00,       //   Data,Array,Abs,No Wrap,Linear,Preferred State,No Null Position
			END_COLLECTION(0)
		};

btw, ive been testing hid examples with windows 10 and there is no need to edit registers to make it works

If you can post report map i will try to check it, also screenshot with error would be great

@Azalgo

This comment has been minimized.

Azalgo commented Oct 4, 2018

No changes, still works on MacOS but not on iOS :(

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Oct 4, 2018

You have to wait till i borrow iPhone, but even then no promise i can find issue.

BTW if anyone got spare iPhone that can send i dont mind

@Staars

This comment has been minimized.

Staars commented Oct 5, 2018

No changes, still works on MacOS but not on iOS :(

Hi,

I tried your short example on macOs Mojave and it connects, but there are no keypresses recognised from the OS (e.g. in the Notes app). Should this happen (or maybe is my ESP32 faulty)?

@Azalgo

This comment has been minimized.

Azalgo commented Oct 8, 2018

I'm on macOS High Sierra and it's working great. Do you have the up to date library ?

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Oct 8, 2018

I am fighting with iPhone now, no key presses recognized too. Im getting crazy.

@Staars

This comment has been minimized.

Staars commented Oct 8, 2018

I'm on macOS High Sierra and it's working great. Do you have the up to date library ?

I hope so ;)
(downloaded from https://github.com/chegewara/esp32-snippets/tree/multi_connect/cpp_utils with the latest commit chegewara@bac1b24)

I will try to check again, if I made a mistake.

@toxuin

This comment has been minimized.

Contributor

toxuin commented Oct 9, 2018

@chegewara have you sorted out your iPhone situation? I have an older iphone (5? 5s?..) that I could send your way – for science. Let me know.

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Oct 9, 2018

I have no iPhone to test, but there is one small chance i did. Try this:
BLEHIDDevice* hid; <--- delete static operand in main app code

If you are willing to send my iPhone i dont mind. 🥇

@toxuin

This comment has been minimized.

Contributor

toxuin commented Oct 9, 2018

@chegewara where could we discuss the shipping arrangement in a more private way? I do have an older iphone that I’m sure you’ll find a use for.

@Azalgo

This comment has been minimized.

Azalgo commented Oct 9, 2018

I have no iPhone to test, but there is one small chance i did. Try this:
BLEHIDDevice* hid; <--- delete static operand in main app code

The BLEHIDDevice is not declared as static in my code.

@toxuin

This comment has been minimized.

Contributor

toxuin commented Oct 10, 2018

Still have no keystrokes on MacOS. Tried @Azalgo's code from above with multi_connect branch from Che – no avail. Symptoms with that code are as follows: I am able to connect, but then device is never actually connected to: it is left for a couple of minutes in the list with a spinny wheel against it and then disconnected.

What I've tried is to use this: https://github.com/asterics/esp32_mouse_keyboard and it's sorta kinda gets me a step further: I am able to stay "connected" and there is no more spinny wheel in settings, but still no keystrokes.

What am I missing?..

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Oct 10, 2018

You are missing nothing, its something i am missing. For example i know there is floating battery characteristic without assigned value. Most likely its too simple but it can be this.
I am running wireshark and as for now its only malformed packet that is exchanged between esp32 and windows 10. Only windows 10 laptop can accept this and stay connected and keystrokes are received.

With asterics library i am able to connect and send keystroke from putty by pressing 'q', it will send y key.

@toxuin

This comment has been minimized.

Contributor

toxuin commented Oct 10, 2018

I do have a hid->setBatteryLevel(7); in my code. Guess what it does to the overall situation? Nothing.

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Oct 10, 2018

I think it can be something wrong with sampleexample itself, not with library, because last my test with iPhone 2 days ago was as follow:

  • get asterics repo,
  • instead his ble library use latest ble library from this repo (or currently mine),
  • try it, and at the end it still works.

Sorry, but it is very frustrating and i have to rest from hid. I will try to add some more functionalities to library now. Have a good luck.

@toxuin

This comment has been minimized.

Contributor

toxuin commented Oct 12, 2018

In my experiments, I could not send keystrokes to a MacOS 10.14 with this library's example code, asterics's library example code (testmode = 1) or my own contraptions. Maximum of what I've got is for it to show as connected in "devices" preferences panel. I must note that my code works alright with android (maybe with windows too? Don't have a machine to test atm).

Asterics's lib even comes with sdkconfig and partition table (and I've used that), so that is not the cause of the problem.

If anyone has a project that they know can send keystrokes to iOS/MacOS (or both!) please share. I want to know what is different and fix it in this lib so it works everywhere equally great.

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Oct 12, 2018

@toxuin this is most frustrating thing with this library. You are saying that asterics code does not work for you, and i just few days ago got iPhone for few hours and it did work for me. All i had to do is git clone, and make flash, but you have to use newBLE branch, not master.

@MagicCube

This comment has been minimized.

MagicCube commented Oct 14, 2018

@chegewara Thank you for your excellent example I could finally made my first BLE HID project.

I think I should also share the source code, it is an ESP32 based BLE Keynote/PPT remote controller.

I also made a BLE Midi Controller and a USB HID Device.

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Oct 14, 2018

@MagicCube Im glad you found this library useful and very nice of you that you are sharing with us what you did.

BTW i am thinkg about changing BLEHID class to make it even easier to use, but now i am busy with other upgrades

I think you can share with wider community posting your code on forum in Showcase

@renu285

This comment has been minimized.

renu285 commented Oct 27, 2018

Hello All,

I was experimenting on esp32 board to create an HID mouse with Arduino library using code snippets provided in this thread, and modifying the keyboard example. So far I am able to pair and see a BLE mouse appear on my smartphone scan. How do I send the actual coordinate values in the message?
Any help is appreciated as I am new to this.

Here is my arduino code

===============================================================================
#include <BLEDevice.h>
#include <BLEUtils.h>
#include <BLEServer.h>
#include "BLE2902.h"
#include "BLEHIDDevice.h"
#include "HIDTypes.h"
#include "HIDKeyboardTypes.h"
#include <driver/adc.h>

BLEHIDDevice* hid;
BLECharacteristic* input;
BLECharacteristic* output;

uint8_t buttons = 0;
uint8_t button1 = 0;
uint8_t button2 = 0;
uint8_t button3 = 0;
bool connected = false;

class MyCallbacks : public BLEServerCallbacks {
void onConnect(BLEServer* pServer){
connected = true;
BLE2902* desc = (BLE2902*)input->getDescriptorByUUID(BLEUUID((uint16_t)0x2902));
desc->setNotifications(true);
}

void onDisconnect(BLEServer* pServer){
connected = false;
BLE2902* desc = (BLE2902*)input->getDescriptorByUUID(BLEUUID((uint16_t)0x2902));
desc->setNotifications(false);
}
};

/*

  • This callback is connect with output report. In keyboard output report report special keys changes, like CAPSLOCK, NUMLOCK
  • We can add digital pins with LED to show status
  • bit 0 - NUM LOCK
  • bit 1 - CAPS LOCK
  • bit 2 - SCROLL LOCK
    /
    class MyOutputCallbacks : public BLECharacteristicCallbacks {
    void onWrite(BLECharacteristic
    me){
    uint8_t* value = (uint8_t*)(me->getValue().c_str());
    ESP_LOGI(LOG_TAG, "special keys: %d", *value);
    }
    };

void taskServer(void*){

BLEDevice::init("ESP32-mouse");
BLEServer *pServer = BLEDevice::createServer();
pServer->setCallbacks(new MyCallbacks());

hid = new BLEHIDDevice(pServer);
input = hid->inputReport(1); // <-- input REPORTID from report map
output = hid->outputReport(1); // <-- output REPORTID from report map

output->setCallbacks(new MyOutputCallbacks());

std::string name = "chegawara";
hid->manufacturer()->setValue(name);

hid->pnp(0x02, 0xe502, 0xa111, 0x0210);
hid->hidInfo(0x00,0x02);

BLESecurity *pSecurity = new BLESecurity();
// pSecurity->setKeySize();
pSecurity->setAuthenticationMode(ESP_LE_AUTH_BOND);

/*
0x05, 0x01, // USAGE_PAGE (Generic Desktop)
0x09, 0x02, // USAGE (Mouse)
0xa1, 0x01, // COLLECTION (Application)
0x09, 0x01, // USAGE (Pointer)
0xa1, 0x00, // COLLECTION (Physical)
0x05, 0x09, // USAGE_PAGE (Button)
0x19, 0x01, // USAGE_MINIMUM (Button 1)
0x29, 0x03, // USAGE_MAXIMUM (Button 3)
0x15, 0x00, // LOGICAL_MINIMUM (0)
0x25, 0x01, // LOGICAL_MAXIMUM (1)
0x95, 0x03, // REPORT_COUNT (3)
0x75, 0x01, // REPORT_SIZE (1)
0x81, 0x02, // INPUT (Data,Var,Abs)
0x95, 0x01, // REPORT_COUNT (1)
0x75, 0x05, // REPORT_SIZE (5)
0x81, 0x03, // INPUT (Cnst,Var,Abs)
0x05, 0x01, // USAGE_PAGE (Generic Desktop)
0x09, 0x30, // USAGE (X)
0x09, 0x31, // USAGE (Y)
0x15, 0x81, // LOGICAL_MINIMUM (-127)
0x25, 0x7f, // LOGICAL_MAXIMUM (127)
0x75, 0x08, // REPORT_SIZE (8)
0x95, 0x02, // REPORT_COUNT (2)
0x81, 0x06, // INPUT (Data,Var,Rel)
0xc0, // END_COLLECTION
0xc0 // END_COLLECTION
*/

const uint8_t report[] = {
USAGE_PAGE(1), 0x01, // Generic Desktop Ctrls
USAGE(1), 0x02,
COLLECTION(1), 0x01, // Application
USAGE(1), 0x01,
COLLECTION(1), 0x00,
USAGE_PAGE(1), 0x09,
USAGE_MINIMUM(1), 0x01,
USAGE_MAXIMUM(1), 0x03,
LOGICAL_MINIMUM(1), 0x00,
LOGICAL_MAXIMUM(1), 0x01,
REPORT_COUNT(1), 0x03,
REPORT_SIZE(1), 0x01,
HIDINPUT(1) , 0x02,
REPORT_COUNT(1), 0x01,
REPORT_SIZE(1), 0x05,
HIDINPUT(1) , 0x03,
USAGE_PAGE(1), 0x01,
USAGE(1), 0x30,
USAGE(1), 0x31,
USAGE_MINIMUM(1), 0x81,
USAGE_MAXIMUM(1), 0x7f,
REPORT_SIZE(1), 0x08,
REPORT_COUNT(1), 0x02,
HIDINPUT(1), 0x06,
END_COLLECTION(0)
};

hid->reportMap((uint8_t*)report, sizeof(report));
hid->startServices();

BLEAdvertising *pAdvertising = pServer->getAdvertising();
pAdvertising->setAppearance(HID_MOUSE);
pAdvertising->addServiceUUID(hid->hidService()->getUUID());
pAdvertising->start();
hid->setBatteryLevel(7);

ESP_LOGD(LOG_TAG, "Advertising started!");
delay(portMAX_DELAY);

};

void setup() {
Serial.begin(115200);
Serial.println("Starting BLE work!");

xTaskCreate(taskServer, "server", 20000, NULL, 5, NULL);
}

void loop() {

if(connected){

}
}

@chegewara

This comment has been minimized.

@renu285

This comment has been minimized.

renu285 commented Oct 28, 2018

@chegewara I have tried these examples and also #621
when I connect the mouse to an android phone. I sometimes see a message saying pairing unsuccessful as passcode didn't match or if the pairing and connection is successful I don't see any pointer appear on the screen and don't see any movement.
Any ideas?

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Oct 28, 2018

@renu285 It was just reference link where you can find hid mouse implemented in code based on this library. You can find info here:
http://eleccelerator.com/tutorial-about-usb-hid-report-descriptors/

To build hid device you need to understand basics of report map or find library where you have already build in such report maps. Hid mouse is not complicated though. What you need is to send 1 byte that will have info about buttons state (usually 3 buttons, but you can have more) and 2 bytes with info about X and Y mouse movement. If you dont see cursor then probably you have something wrong with report map. You can find report map here:
https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/tests/BLETests/SampleHIDDevice.cpp#L112-L142

I dont have time now, but i have plans to refactor HID class to make it even easier to use with 3 basic usage report maps (keyboard, mouse, gamepad/joystick).

@renu285

This comment has been minimized.

renu285 commented Nov 1, 2018

Hello,

After several days of digging around and trying to understand how input reports work I finally was able to get it to work (sort of). I can now move around the mouse pointer (both x and y) and also the scroll wheel works. (Mandatory needed by Windows10)

However I am unable to get the mouse to click anything i.e perform left click, right click etc.

Here is the input report I am using that I scavenged off of internet . It has 5 buttons defined.
My current reporting format that works for me are as follows a[] = {x,y,vertical-wheel,hori-wheel,0}
It doesn't generate any clicks even after passing any values to a[4]. Don't know what to pass to get it to click with the following format. I have tried other reports but this is the only one that works so far

const uint8_t report[] = {

 0x05, 0x01,        // USAGE_PAGE (Generic Desktop)
 0x09, 0x02,        // USAGE (Mouse)
 0xa1, 0x01,        // COLLECTION (Application)
 0x09, 0x02,        //   USAGE (Mouse)
 0xa1, 0x02,        //   COLLECTION (Logical)
 0x09, 0x01,        //     USAGE (Pointer)
 0xa1, 0x00,        //     COLLECTION (Physical)
                    // ------------------------------  Buttons
 0x05, 0x09,        //       USAGE_PAGE (Button)      
 0x19, 0x01,        //       USAGE_MINIMUM (Button 1)
 0x29, 0x05,        //       USAGE_MAXIMUM (Button 5)
 0x15, 0x00,        //       LOGICAL_MINIMUM (0)
 0x25, 0x01,        //       LOGICAL_MAXIMUM (1)
 0x75, 0x01,        //       REPORT_SIZE (1)
 0x95, 0x05,        //       REPORT_COUNT (5)
 0x81, 0x02,        //       INPUT (Data,Var,Abs)
                    // ------------------------------  Padding
 0x75, 0x03,        //       REPORT_SIZE (3)
 0x95, 0x01,        //       REPORT_COUNT (1)
 0x81, 0x03,        //       INPUT (Cnst,Var,Abs)
                    // ------------------------------  X,Y position
 0x05, 0x01,        //       USAGE_PAGE (Generic Desktop)
 0x09, 0x30,        //       USAGE (X)
 0x09, 0x31,        //       USAGE (Y)
 0x15, 0x81,        //       LOGICAL_MINIMUM (-127)
 0x25, 0x7f,        //       LOGICAL_MAXIMUM (127)
 0x75, 0x08,        //       REPORT_SIZE (8)
 0x95, 0x02,        //       REPORT_COUNT (2)
 0x81, 0x06,        //       INPUT (Data,Var,Rel)
 0xa1, 0x02,        //       COLLECTION (Logical)
                    // ------------------------------  Vertical wheel res multiplier
 0x09, 0x48,        //         USAGE (Resolution Multiplier)
 0x15, 0x00,        //         LOGICAL_MINIMUM (0)
 0x25, 0x01,        //         LOGICAL_MAXIMUM (1)
 0x35, 0x01,        //         PHYSICAL_MINIMUM (1)
 0x45, 0x04,        //         PHYSICAL_MAXIMUM (4)
 0x75, 0x02,        //         REPORT_SIZE (2)
 0x95, 0x01,        //         REPORT_COUNT (1)
 0xa4,              //         PUSH
 0xb1, 0x02,        //         FEATURE (Data,Var,Abs)
                    // ------------------------------  Vertical wheel
 0x09, 0x38,        //         USAGE (Wheel)
 0x15, 0x81,        //         LOGICAL_MINIMUM (-127)
 0x25, 0x7f,        //         LOGICAL_MAXIMUM (127)
 0x35, 0x00,        //         PHYSICAL_MINIMUM (0)        - reset physical
 0x45, 0x00,        //         PHYSICAL_MAXIMUM (0)
 0x75, 0x08,        //         REPORT_SIZE (8)
 0x81, 0x06,        //         INPUT (Data,Var,Rel)
 0xc0,              //       END_COLLECTION
 0xa1, 0x02,        //       COLLECTION (Logical)
                    // ------------------------------  Horizontal wheel res multiplier
 0x09, 0x48,        //         USAGE (Resolution Multiplier)
 0xb4,              //         POP
 0xb1, 0x02,        //         FEATURE (Data,Var,Abs)
                    // ------------------------------  Padding for Feature report
 0x35, 0x00,        //         PHYSICAL_MINIMUM (0)        - reset physical
 0x45, 0x00,        //         PHYSICAL_MAXIMUM (0)
 0x75, 0x04,        //         REPORT_SIZE (4)
 0xb1, 0x03,        //         FEATURE (Cnst,Var,Abs)
                    // ------------------------------  Horizontal wheel
 0x05, 0x0c,        //         USAGE_PAGE (Consumer Devices)
 0x0a, 0x38, 0x02,  //         USAGE (AC Pan)
 0x15, 0x81,        //         LOGICAL_MINIMUM (-127)
 0x25, 0x7f,        //         LOGICAL_MAXIMUM (127)
 0x75, 0x08,        //         REPORT_SIZE (8)
 0x81, 0x06,        //         INPUT (Data,Var,Rel)
 0xc0,              //       END_COLLECTION
 0xc0,              //     END_COLLECTION
 0xc0,              //   END_COLLECTION
 0xc0               // END_COLLECTION

};

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Nov 1, 2018

@renu285 If i may suggest something, this report map is way too complicated for mouse. Im not saying it may or may not work, just saying there is a lot functionalities that you for sure not implement in your device.

This report map is sufficient:
https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/tests/BLETests/SampleHIDDevice.cpp#L116-L141
only thing that can be missing here is reportID. When ive been working on this class and everything that is related to HID i found that windows requires or works better when report map has reportID:
https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/tests/BLETests/SampleHIDKeyboard.cpp#L124
If this report map wont work with windows then probably is missing wheel (like you mention its most likely mandatory).

Now back to your report map. Buttons should be in byte 0 since it is first element in report map (like you said 5 buttons, 1 bit for each button and bits padding). All bytes are counted from beginning of report map. Next you should have cursor X and Y movement, one byte for each. Then wheels, but here i am not sure because i did not study push/pop and it was about 10 months ago when ive been working with HID.

If i can help you somehow else just ask.

@renu285

This comment has been minimized.

renu285 commented Nov 2, 2018

Thank you very much for your reply @chegewara. I have been struggling with these report maps from quite some time now. I am aware that for my purpose a simple x,y,button combination is fine and I have already tried the report maps in the links that you mentioned, But none of them seem to be recognized by android and windows even with the wheel. The report map which I have shared is the only one which works except the button clicks.

In the reportmap I have shared, from what I understand buttons should be in byte zero as it is at the beginning of the report map but in practice this is not the case I get x,y movements when I send something in byte 0 and byte 1, I get scroll movements when I send something in byte 2 and first 4 bits of byte 3 and no matter what I send nowhere I get right/left/click etc. I even modified the report map to have only 3 buttons and 5 padded bits but there is no improvement.

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Nov 2, 2018

Ive been testing this report map and it for sure works with android (windows i dont remember).

I will try to run simple hid example to confirm when i find time, probably even this week.

@renu285

This comment has been minimized.

renu285 commented Nov 2, 2018

Sure that will be very helpful...If you can share the working code

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Nov 18, 2018

https://eleccelerator.com/tutorial-about-usb-hid-report-descriptors/
@renu285

struct mouse_report_t
{
uint8_t buttons;
int8_t x;
int8_t y;
}

@coreboot2

This comment has been minimized.

coreboot2 commented Dec 4, 2018

I want to make a DIY footswitch. I have some questions... Firstly, I'm searching for a bluetooth module. I have come across BC417, CC2540, CC2640, ESP32...
I found the CC2540 is cheaper, just $1.45 https://www.aliexpress.com/item/AT-09-Android-IOS-BLE-4-0-Bluetooth-module-for-arduino-CC2540-CC2541-Serial-Wireless-Module/32818178064.html
The ESP32 is not that cheap, is $2.95 https://www.aliexpress.com/item/ESP-32S-ESP-WROOM-32-ESP32-ESP-32-Bluetooth-and-WIFI-Dual-Core-CPU-with-Low/32802438946.html

But it seems there are more people working on the ESP32 compared to CC2540...? Are there anyother modules that can be used for a keyboard? And how to choose, which one is the best?

I also want the keyboard to have a function like the logitech's easy switch https://www.logitech.com/en-us/product/illuminated-keyboard-for-mac-ipad-iphone (using the same keyboard to pair with different computers and switch among them with one click). Can these modules do this?

Also, is it possible to change the footswitch's key easily? Using OTA update?

@chegewara

This comment has been minimized.

Collaborator

chegewara commented Dec 4, 2018

Hi @coreboot2
did you try app from OP? Code is based on this library and as far as i know its working with iPhone and macOS. This library may be, im not saying it is, less stable now in HID apps at the moment. Its caused by changes i made in it to implement more features and still require a lot tests and some changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment