New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] Pluggable USB core for AVR #3304

Merged
merged 39 commits into from Jul 16, 2015

Conversation

8 participants
@facchinm
Member

facchinm commented Jun 8, 2015

This PR introduces the concept of Pluggable USB modules to help getting the best from our tiny ATmega32u4 without overhauling the core library with functions that could be useful for no more than 10% of the users.

Using this method we can plug a library using the USB core by simply including its header in the sketch, and if not used it will not occupy additional resources.

It is of course a work in progress (thus tagged as RFC), still not ported to DUE core, and VERY probably bug-prone. Also, the memory footprint is not minimized.

Useful infos

  • the CDC module is now non-removable (commit 43701fd)
  • the Pluggable USB module inherits most of the code from old HID and replaces its functions with hooks
  • max additional endpoints is arbitrarily set to 3
  • HID, Mouse, Keyboard and MouseAndKeyboard libraries are "generated" from master code (so all code from #1803 can be merged in a second time)
  • a MIDIUSB library based on PluggableUSB has been added (it is a very raw porting of #2943) only to demonstrate how to plug multiple modules
  • setupUSB() weak function has been added to main.c , so it will execute before setup() if USBCON is defined.

Known issues

  • I'll add them here as soon as they get discovered (no doubt this list will grow soon 馃槃 )

How to try it
I decided not to change any provided example . If you wish to test the core, download the @ArduinoBot generated packages and modify the examples in this way:

On the top of the file:

  • Mouse only examples: add
#include "Mouse.h"
#include "HID.h"
  • Keyboard only examples: add
#include "Keyboard.h"
#include "HID.h"
  • Mouse and Keyboard examples: add
#include "MouseAndKeyboard.h"
#include "HID.h"
  • HID and MIDIUSB example:
#include "MIDIUSB.h"
#include "HID.h"
#include "Mouse.h"
@NicoHood

This comment has been minimized.

Show comment
Hide comment
@NicoHood

NicoHood Jun 12, 2015

Contributor

This really sounds good to me. I havent looked at the code yet, but I will test this for sure.
I also made the CDC etc pluggable. Not via include, just because I moved the code in a seperate .cpp file which is not linked if not used. This saves a lot of space.

Also I would provide an include "USB.h" to enable USB stuff. Then users can remove the whole USB core and save even more space if not needed and also avoid problems with the PC.

You may want to have a look at my core and at the dev branches to get new Ideas possibly:
https://github.com/NicoHood/HID

Contributor

NicoHood commented Jun 12, 2015

This really sounds good to me. I havent looked at the code yet, but I will test this for sure.
I also made the CDC etc pluggable. Not via include, just because I moved the code in a seperate .cpp file which is not linked if not used. This saves a lot of space.

Also I would provide an include "USB.h" to enable USB stuff. Then users can remove the whole USB core and save even more space if not needed and also avoid problems with the PC.

You may want to have a look at my core and at the dev branches to get new Ideas possibly:
https://github.com/NicoHood/HID

@facchinm

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Jun 12, 2015

Member

Hi Nico,
thanks for taking the time to watch this PR!
I obviously took a look at your core before starting this effort and I think that most of your patches (and other peoples' patches as well) could be integrated if we are able to modularize the USB 馃槒
The biggest problem in using your approach is that you need to:

  • provide a lot of board variants (too noisy) or
  • let the user modify the #defines in the core itself (too advanced)

Also, your core handles a lot of HID-based devices but it's not ready for other kind of devices (like MIDI #2943 or something else that I can't imagine at the moment 馃槃 )

However, if you could test the patchset it would be great, I'm only waiting for the feedback!

Member

facchinm commented Jun 12, 2015

Hi Nico,
thanks for taking the time to watch this PR!
I obviously took a look at your core before starting this effort and I think that most of your patches (and other peoples' patches as well) could be integrated if we are able to modularize the USB 馃槒
The biggest problem in using your approach is that you need to:

  • provide a lot of board variants (too noisy) or
  • let the user modify the #defines in the core itself (too advanced)

Also, your core handles a lot of HID-based devices but it's not ready for other kind of devices (like MIDI #2943 or something else that I can't imagine at the moment 馃槃 )

However, if you could test the patchset it would be great, I'm only waiting for the feedback!

@NicoHood

This comment has been minimized.

Show comment
Hide comment
@NicoHood

NicoHood Jun 12, 2015

Contributor

Well I have a midi option that I could integrate. Its just under a dev state so I did not create a proper menu yet. Also it has a 2nd Audio device with errors under windows. I just used the arcore code for this.

I know that the boards file is not perfect, but it was the only way for me to simply add a menu to the IDE.

I really like your IDE to make the USB stuff more flexible. You know that this will change a lot of code. Is the Arduino Team willing to change it? Because so many people provided new features but mostly none of them were integrated yet. You are collaborator, does this mean that we can count on a PR merge if its good?

You should also consider to fix this bug when you update the USB core:
#2474 (comment)

And you should also consider to update the bootloader fuses and watchdog behaviour in order to fully deactivate the USB-Core. Right now the bootloader starts the sketch directly. You could better to a watchdog reset and start the code independent. The difference is that all settings like the USB clock is reset. Without this fix you have to add the USB clock ISR into the core as workaround. I could explain it better if someone is willing to update the bootloader in general. This would make life much easier.

I am not likely able to test your code this/next week, I am too busy, sorry. But I will have a look for sure.

Edit:
How do you want to solve different HID devices? Do you want to add a specific Endpoint for each device? This could be tricky if users want to add more devices. The 32u4 has only 6 endpoints by the way. Also you have to keep in mind that changing the endpoints makes windows go crazy. you better add more PIDs for each device. You have 2^16 as Vendor, so it shouldnt be that big Problem.

Contributor

NicoHood commented Jun 12, 2015

Well I have a midi option that I could integrate. Its just under a dev state so I did not create a proper menu yet. Also it has a 2nd Audio device with errors under windows. I just used the arcore code for this.

I know that the boards file is not perfect, but it was the only way for me to simply add a menu to the IDE.

I really like your IDE to make the USB stuff more flexible. You know that this will change a lot of code. Is the Arduino Team willing to change it? Because so many people provided new features but mostly none of them were integrated yet. You are collaborator, does this mean that we can count on a PR merge if its good?

You should also consider to fix this bug when you update the USB core:
#2474 (comment)

And you should also consider to update the bootloader fuses and watchdog behaviour in order to fully deactivate the USB-Core. Right now the bootloader starts the sketch directly. You could better to a watchdog reset and start the code independent. The difference is that all settings like the USB clock is reset. Without this fix you have to add the USB clock ISR into the core as workaround. I could explain it better if someone is willing to update the bootloader in general. This would make life much easier.

I am not likely able to test your code this/next week, I am too busy, sorry. But I will have a look for sure.

Edit:
How do you want to solve different HID devices? Do you want to add a specific Endpoint for each device? This could be tricky if users want to add more devices. The 32u4 has only 6 endpoints by the way. Also you have to keep in mind that changing the endpoints makes windows go crazy. you better add more PIDs for each device. You have 2^16 as Vendor, so it shouldnt be that big Problem.

@NicoHood NicoHood referenced this pull request Jun 12, 2015

Closed

Add MIDI on 32u4? #2

@matthijskooijman

This comment has been minimized.

Show comment
Hide comment
@matthijskooijman

matthijskooijman Jun 15, 2015

Collaborator

Overall, this looks like a nice implementation, using a linked list of modules and separate libraries is an elegant way to solve this.

setupUSB() weak function has been added to main.c , so it will execute before setup() if USBCON is defined.

Isn't a weak setup function completely counter to the idea of pluggable modules? Only one module can implement this setup function, otherwise you'll get linker errors (or if the modules make their implementation weak too, they'll get silently ignored). Wouldn't an init function (since the setup name seems to be used already for processing control packets) in the callbacks make sense instead?

Would it make sense to write some documentation about how to use this now? Having documentation would help to understand and review the PR, and writing documentation after merging usually means it'll take months before that happens...

Collaborator

matthijskooijman commented Jun 15, 2015

Overall, this looks like a nice implementation, using a linked list of modules and separate libraries is an elegant way to solve this.

setupUSB() weak function has been added to main.c , so it will execute before setup() if USBCON is defined.

Isn't a weak setup function completely counter to the idea of pluggable modules? Only one module can implement this setup function, otherwise you'll get linker errors (or if the modules make their implementation weak too, they'll get silently ignored). Wouldn't an init function (since the setup name seems to be used already for processing control packets) in the callbacks make sense instead?

Would it make sense to write some documentation about how to use this now? Having documentation would help to understand and review the PR, and writing documentation after merging usually means it'll take months before that happens...

@NicoHood

This comment has been minimized.

Show comment
Hide comment
@NicoHood

NicoHood Jun 15, 2015

Contributor

What do you mean by cb variable?

Can we also see u2 Series support with this update? HL2 is getting more and more popular this could be a great time to merge everything together.

Contributor

NicoHood commented Jun 15, 2015

What do you mean by cb variable?

Can we also see u2 Series support with this update? HL2 is getting more and more popular this could be a great time to merge everything together.

@NicoHood

This comment has been minimized.

Show comment
Hide comment
@NicoHood

NicoHood Jun 16, 2015

Contributor

I will add comments/thoughts to the PR here (this post will be edited):

The different Mouse/Keyboard/MouseAndKeyboard HID reports wont work properly under Windows I want to note. You have to use different PIDs to make the driver recognize it properly or reinstall the drivers and provide a correct tutorial somewhere.

May we get some diagrams on how this works together? So others can understand better. I have to fiddle through this right now.

And I have to mention that I just looked at the code in small parts. I do not understand it as a whole thing since I never intended to understand it.

Contributor

NicoHood commented Jun 16, 2015

I will add comments/thoughts to the PR here (this post will be edited):

The different Mouse/Keyboard/MouseAndKeyboard HID reports wont work properly under Windows I want to note. You have to use different PIDs to make the driver recognize it properly or reinstall the drivers and provide a correct tutorial somewhere.

May we get some diagrams on how this works together? So others can understand better. I have to fiddle through this right now.

And I have to mention that I just looked at the code in small parts. I do not understand it as a whole thing since I never intended to understand it.

#ifdef HID_ENABLED
total += HID_GetInterface(&interfaces);
#ifdef PLUGGABLE_USB_ENABLED
PUSB_GetInterface(&interfaces);
#endif
return interfaces;

This comment has been minimized.

@NicoHood

NicoHood Jun 16, 2015

Contributor

This returns always 0. Whats the sense of that function?

@NicoHood

NicoHood Jun 16, 2015

Contributor

This returns always 0. Whats the sense of that function?

This comment has been minimized.

@facchinm

facchinm Jun 22, 2015

Member

it does't return 0 because interfaces is incremented by the callee (instead, the original total variable was completely useless)

@facchinm

facchinm Jun 22, 2015

Member

it does't return 0 because interfaces is incremented by the callee (instead, the original total variable was completely useless)

//allocate 3 endpoints and remove const so they can be changed by the user
0,
0,
0,
#endif
};

This comment has been minimized.

@NicoHood

NicoHood Jun 16, 2015

Contributor

I dont know of there is a definition of the number of endpoints, but please consider that other MCUs dont have 6 endpoint and that not every endpoint can take double 64 bytes. If you want to make it flexible then let the use decide about the size and double bank or not.

@NicoHood

NicoHood Jun 16, 2015

Contributor

I dont know of there is a definition of the number of endpoints, but please consider that other MCUs dont have 6 endpoint and that not every endpoint can take double 64 bytes. If you want to make it flexible then let the use decide about the size and double bank or not.

This comment has been minimized.

@matthijskooijman

matthijskooijman Jun 18, 2015

Collaborator

Isn't there something defined in io.h that indicates how many endpoints are supported? Not sure what you mean with the double-thing (I don't really know the USB details).

@matthijskooijman

matthijskooijman Jun 18, 2015

Collaborator

Isn't there something defined in io.h that indicates how many endpoints are supported? Not sure what you mean with the double-thing (I don't really know the USB details).

This comment has been minimized.

@NicoHood

NicoHood Jun 18, 2015

Contributor

USB Banks can have 8,16,32,64 bytes of data. Double or single sized. Double means it has 2x64 in this case. But the u2 Series doesnt support that if you do this on any endpoint. It has less ram for the USB stuff (has a special name in the datasheet). Also to avoid problems with custom endpoint descriptors I'd let the programmer choose if its double banked or not and if it 8,16,32,64 bytes.

@NicoHood

NicoHood Jun 18, 2015

Contributor

USB Banks can have 8,16,32,64 bytes of data. Double or single sized. Double means it has 2x64 in this case. But the u2 Series doesnt support that if you do this on any endpoint. It has less ram for the USB stuff (has a special name in the datasheet). Also to avoid problems with custom endpoint descriptors I'd let the programmer choose if its double banked or not and if it 8,16,32,64 bytes.

This comment has been minimized.

@facchinm

facchinm Jun 22, 2015

Member

Yes, all the stuff is very 32u4-centric, so your help in the u2 side and your deep knowledge of the AVR USB stack are very precious!

@facchinm

facchinm Jun 22, 2015

Member

Yes, all the stuff is very 32u4-centric, so your help in the u2 side and your deep knowledge of the AVR USB stack are very precious!

@@ -72,18 +65,14 @@ const u8 STRING_PRODUCT[] PROGMEM = USB_PRODUCT;
const u8 STRING_MANUFACTURER[] PROGMEM = USB_MANUFACTURER;
#ifdef CDC_ENABLED

This comment has been minimized.

@NicoHood

NicoHood Jun 16, 2015

Contributor

I'd also not force the CDC to be automatically added. This is not pluggable if you ask me. There should be a way to remove it and use other functions for usb. And you should add weak events for the CDC Serial as in my HID Project.

@NicoHood

NicoHood Jun 16, 2015

Contributor

I'd also not force the CDC to be automatically added. This is not pluggable if you ask me. There should be a way to remove it and use other functions for usb. And you should add weak events for the CDC Serial as in my HID Project.

This comment has been minimized.

@facchinm

facchinm Jun 22, 2015

Member

CDC removal was a choice (indeed the commit is atomic and can be safely reverted)

@facchinm

facchinm Jun 22, 2015

Member

CDC removal was a choice (indeed the commit is atomic and can be safely reverted)

#define EP_TYPE_INTERRUPT_OUT 0xC0
#define EP_TYPE_ISOCHRONOUS_IN 0x41
#define EP_TYPE_ISOCHRONOUS_OUT 0x40
/** Pulse generation counters to keep track of the number of milliseconds remaining for each pulse type */
#define TX_RX_LED_PULSE_MS 100
volatile u8 TxLEDPulse; /**< Milliseconds remaining for data Tx LED pulse */

This comment has been minimized.

@NicoHood

NicoHood Jun 16, 2015

Contributor

I'd remove the u8 since it just makes things more complicated with the compatibility. Just use uint8_t if you ask me.

@NicoHood

NicoHood Jun 16, 2015

Contributor

I'd remove the u8 since it just makes things more complicated with the compatibility. Just use uint8_t if you ask me.

This comment has been minimized.

@matthijskooijman

matthijskooijman Jun 18, 2015

Collaborator

+1, but converting types is completely unrelated to this change and should probably be done separately to not complicate this PR.

@matthijskooijman

matthijskooijman Jun 18, 2015

Collaborator

+1, but converting types is completely unrelated to this change and should probably be done separately to not complicate this PR.

const DeviceDescriptor USB_DeviceDescriptorA =
D_DEVICE(DEVICE_CLASS,0x00,0x00,64,USB_VID,USB_PID,0x100,IMANUFACTURER,IPRODUCT,0,1);
const DeviceDescriptor USB_DeviceDescriptorB =
D_DEVICE(0xEF,0x02,0x01,64,USB_VID,USB_PID,0x100,IMANUFACTURER,IPRODUCT,0,1);

This comment has been minimized.

@NicoHood

NicoHood Jun 16, 2015

Contributor

I might get this wrong, but you have to add 0xEF, 0x02 and 0x01 to the upper Descriptor. Thats how I did it in my source at least.

Also I'd go for definitions of those values so they are better understandable.

@NicoHood

NicoHood Jun 16, 2015

Contributor

I might get this wrong, but you have to add 0xEF, 0x02 and 0x01 to the upper Descriptor. Thats how I did it in my source at least.

Also I'd go for definitions of those values so they are better understandable.

Show outdated Hide outdated hardware/arduino/avr/cores/arduino/PluggableUSB.cpp
PUSBListNode* node = rootNode;
for (u8 i=0; i<modules_count; i++) {
ret = node->cb.getInterface(interfaceNum);
node = node->next;

This comment has been minimized.

@NicoHood

NicoHood Jun 16, 2015

Contributor

Posted the comment in a commit, but still applies here. I am not sure if all those functions make so much sense with the retvalue. Also lastNode is not really used.

@NicoHood

NicoHood Jun 16, 2015

Contributor

Posted the comment in a commit, but still applies here. I am not sure if all those functions make so much sense with the retvalue. Also lastNode is not really used.

@matthijskooijman

This comment has been minimized.

Show comment
Hide comment
@matthijskooijman

matthijskooijman Jun 18, 2015

Collaborator

What do you mean by cb variable?

I presume this was in reply to this comment

I mean that instead of:

cb.setup = &HID_Setup;
cb.getInterface = &HID_GetInterface;
cb.getDescriptor = &HID_GetDescriptor;
cb.numEndpoints = 1;
cb.endpointType[0] = EP_TYPE_INTERRUPT_IN;

you could do:

static const PUSBCallbacks cb = {
    .setup = &HID_Setup,
    .getInterface = &HID_GetInterface,
    etc.
}
Collaborator

matthijskooijman commented Jun 18, 2015

What do you mean by cb variable?

I presume this was in reply to this comment

I mean that instead of:

cb.setup = &HID_Setup;
cb.getInterface = &HID_GetInterface;
cb.getDescriptor = &HID_GetDescriptor;
cb.numEndpoints = 1;
cb.endpointType[0] = EP_TYPE_INTERRUPT_IN;

you could do:

static const PUSBCallbacks cb = {
    .setup = &HID_Setup,
    .getInterface = &HID_GetInterface,
    etc.
}
@facchinm

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Jun 22, 2015

Member

@NicoHood @matthijskooijman Thank you so much for taking time to analyse the (huge) PR. I'll try to update it following your advices as soon as I can get some spare time.

About OS-specific issues, I think that Nico is our man, so any help is really appreciated!

Member

facchinm commented Jun 22, 2015

@NicoHood @matthijskooijman Thank you so much for taking time to analyse the (huge) PR. I'll try to update it following your advices as soon as I can get some spare time.

About OS-specific issues, I think that Nico is our man, so any help is really appreciated!

@facchinm

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Jun 22, 2015

Member

@matthijskooijman about the weak function, I could not imagine anything better.

From my tests the (also weak) setupUSB functions contained in the library do overwrite the one in main.c (I think it's only a linking order side effect but it works).

However, anyone can freely overwrite it with its own non-weak implementation (like in the MIDI+HID example). Any alternative however is very welcome 馃槃

Member

facchinm commented Jun 22, 2015

@matthijskooijman about the weak function, I could not imagine anything better.

From my tests the (also weak) setupUSB functions contained in the library do overwrite the one in main.c (I think it's only a linking order side effect but it works).

However, anyone can freely overwrite it with its own non-weak implementation (like in the MIDI+HID example). Any alternative however is very welcome 馃槃

@matthijskooijman

This comment has been minimized.

Show comment
Hide comment
@matthijskooijman

matthijskooijman Jun 23, 2015

Collaborator

However, anyone can freely overwrite it with its own non-weak implementation (like in the MIDI+HID example). Any alternative however is very welcome
The problem is that when you override the setupUSB function, all previous overrides will be discarded. This means that a sketch that includes multiple USB libraries, must provide its own setupUSB function that calls all the appropriate begin methods.

@matthijskooijman about the weak function, I could not imagine anything better.

Looking more closely, I see you are using setupUSB to call e.g. HID.begin(), which eventually calls PUSB_AddFunction() to register the usb module and its callbacks. My previous suggestion, to use a callback instead of a weak setupUSB function of course doesn't work, since it is needed to register the callbacks in the first place.

There is a good alternative, though: Use a global object, whose constructor calls PUSB_AddFunction(). Constructors of global objects are called on startup automatically.

As an example of how this works, see ModuleHandler and Module in https://github.com/Pinoccio/library-pinoccio/tree/master/src/modules Those register a Module* in a linked list inside ModuleHandler, which is the equivalent of calling PUSB_AddFunction

One caveat is that this all runs before main() is even started and the initialization order is undefined, so you should check that PUSB_AddFunction does not need any constructors or other initalization code to run (global variables initialized to 0 or some other constant are ok, zeroing of the .bss section and loading of the .data section happens before calling global constructor). A quick look around the code shows that this is already ok.

I even think that this approach doesn't require separate libraries for every module, since the compiler can completely remove a module, including its global constructor, if it's not referenced anywhere at all. I'm not 100% sure about this, though. And in most cases, using separate libraries is sane approach regardless of this.

I had a closer look at the HID libraries, and I'm not so happy with how those look. AFAICS, the Mouse and Keyboard libraries both define a few specially-named symbols, which are then used by the HID library. This of course doesn't allow more than one HID "submodule", which is why there is a KeyboardAndMouse library, which is of course fairly ugly.

Ideally, an approach with callbacks similar to PUSB_AddFunction is also used for HID submodules, allowing them to be completely separate from each other. However, I'm not sure if the HID descriptors are actually easy to compose (e.g. can you just concatenate or otherwise combine the Mouse and Keyboard descriptors into something valid)?

Collaborator

matthijskooijman commented Jun 23, 2015

However, anyone can freely overwrite it with its own non-weak implementation (like in the MIDI+HID example). Any alternative however is very welcome
The problem is that when you override the setupUSB function, all previous overrides will be discarded. This means that a sketch that includes multiple USB libraries, must provide its own setupUSB function that calls all the appropriate begin methods.

@matthijskooijman about the weak function, I could not imagine anything better.

Looking more closely, I see you are using setupUSB to call e.g. HID.begin(), which eventually calls PUSB_AddFunction() to register the usb module and its callbacks. My previous suggestion, to use a callback instead of a weak setupUSB function of course doesn't work, since it is needed to register the callbacks in the first place.

There is a good alternative, though: Use a global object, whose constructor calls PUSB_AddFunction(). Constructors of global objects are called on startup automatically.

As an example of how this works, see ModuleHandler and Module in https://github.com/Pinoccio/library-pinoccio/tree/master/src/modules Those register a Module* in a linked list inside ModuleHandler, which is the equivalent of calling PUSB_AddFunction

One caveat is that this all runs before main() is even started and the initialization order is undefined, so you should check that PUSB_AddFunction does not need any constructors or other initalization code to run (global variables initialized to 0 or some other constant are ok, zeroing of the .bss section and loading of the .data section happens before calling global constructor). A quick look around the code shows that this is already ok.

I even think that this approach doesn't require separate libraries for every module, since the compiler can completely remove a module, including its global constructor, if it's not referenced anywhere at all. I'm not 100% sure about this, though. And in most cases, using separate libraries is sane approach regardless of this.

I had a closer look at the HID libraries, and I'm not so happy with how those look. AFAICS, the Mouse and Keyboard libraries both define a few specially-named symbols, which are then used by the HID library. This of course doesn't allow more than one HID "submodule", which is why there is a KeyboardAndMouse library, which is of course fairly ugly.

Ideally, an approach with callbacks similar to PUSB_AddFunction is also used for HID submodules, allowing them to be completely separate from each other. However, I'm not sure if the HID descriptors are actually easy to compose (e.g. can you just concatenate or otherwise combine the Mouse and Keyboard descriptors into something valid)?

@facchinm

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Jun 23, 2015

Member
  • You are right about the ugliness of HID submodules (their need of special symbols especially) but the descriptor must be unique so only one #include is allowed "by design" (also enforced by 18a2f2a)
    Of course a fully modular solution is the way to go but from a library creator's point of view it would probably be more complicated than defining a specially named function (I'm thinking about descriptors overriding)
  • Thanks a lot for the global constructor tip, I missed that completely (I'm a C guy 馃槃 )

Now the produced binary should be comparable (in size) with the non-modular one so we can start thinking about integrating the various PR as libraries and test the results

Member

facchinm commented Jun 23, 2015

  • You are right about the ugliness of HID submodules (their need of special symbols especially) but the descriptor must be unique so only one #include is allowed "by design" (also enforced by 18a2f2a)
    Of course a fully modular solution is the way to go but from a library creator's point of view it would probably be more complicated than defining a specially named function (I'm thinking about descriptors overriding)
  • Thanks a lot for the global constructor tip, I missed that completely (I'm a C guy 馃槃 )

Now the produced binary should be comparable (in size) with the non-modular one so we can start thinking about integrating the various PR as libraries and test the results

@facchinm

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Jun 23, 2015

Member

@ArduinoBot build this please

Member

facchinm commented Jun 23, 2015

@ArduinoBot build this please

@facchinm

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Jun 23, 2015

Member

Size comparison with the current code:

Sketch Modular Old
KeyboardLogout 5574-218 5030-153
KeyboardSerial 4868-216 4324-151
KeyboardAndMouseControl 5704-217 5102-151
ButtonMouseControl 4962-195 5110-155
empty sketch 3476-133 4254-151
ASCIITable 4364-211 5144-231
Member

facchinm commented Jun 23, 2015

Size comparison with the current code:

Sketch Modular Old
KeyboardLogout 5574-218 5030-153
KeyboardSerial 4868-216 4324-151
KeyboardAndMouseControl 5704-217 5102-151
ButtonMouseControl 4962-195 5110-155
empty sketch 3476-133 4254-151
ASCIITable 4364-211 5144-231
@facchinm

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Jun 26, 2015

Member

https://github.com/facchinm/Arduino/tree/testPlugUSB_PR+1803 contains a complete porting of PR #1803 adapted to the Pluggable framework

Member

facchinm commented Jun 26, 2015

https://github.com/facchinm/Arduino/tree/testPlugUSB_PR+1803 contains a complete porting of PR #1803 adapted to the Pluggable framework

@facchinm

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Jul 2, 2015

Member

Almost end of week update:

  • HID is now pluggable itself! You can add
#include "Mouse.h"
#include "Keyboard.h"
#include "HID.h"

and the descriptor will be the combination of the submodules' descriptors. The method is not very memory efficient; however you can still write a library which plugs on PluggableUSB directly, embedding in it an HID implementation and only the functions you need.

@NicoHood @matthijskooijman @obra @follower @cmaglie @madrang @damellis @PaulStoffregen @weizenspreu and all people interested in USB on AVR, please test the latest @ArduinoBot image to make sure that nothing breaks spectacularly before merging 馃榿

Thank you very much!

Member

facchinm commented Jul 2, 2015

Almost end of week update:

  • HID is now pluggable itself! You can add
#include "Mouse.h"
#include "Keyboard.h"
#include "HID.h"

and the descriptor will be the combination of the submodules' descriptors. The method is not very memory efficient; however you can still write a library which plugs on PluggableUSB directly, embedding in it an HID implementation and only the functions you need.

@NicoHood @matthijskooijman @obra @follower @cmaglie @madrang @damellis @PaulStoffregen @weizenspreu and all people interested in USB on AVR, please test the latest @ArduinoBot image to make sure that nothing breaks spectacularly before merging 馃榿

Thank you very much!

@obra

This comment has been minimized.

Show comment
Hide comment
@obra

obra Jul 2, 2015

Contributor

Oh wow. That's amazing. Thank you so, so, so much. I'm on the road this week but will test when I can.

On Jul 2, 2015, at 10:26 AM, Martino Facchin notifications@github.com wrote:

Almost end of week update:

HID is now pluggable itself! You can add
#include "Mouse.h"
#include "Keyboard.h"
#include "HID.h"
and the descriptor will be the combination of the submodules' descriptors. The method is not very memory efficient; however you can still write a library which plugs on PluggableUSB directly, embedding in it an HID implementation and only the functions you need.

the PR now contains @matthijskooijman C++11 patches to get rid of a nasty warning

next step: enable lto (reduces flash occupation by 400 bytes) https://gist.github.com/facchinm/432fbb8ca4d9705644bd

@NicoHood @matthijskooijman @obra @follower @cmaglie @madrang @damellis @PaulStoffregen @weizenspreu and all people interested in USB on AVR, please test the latest @ArduinoBot image to make sure that nothing breaks spectacularly before merging

Thank you very much!


Reply to this email directly or view it on GitHub.

Contributor

obra commented Jul 2, 2015

Oh wow. That's amazing. Thank you so, so, so much. I'm on the road this week but will test when I can.

On Jul 2, 2015, at 10:26 AM, Martino Facchin notifications@github.com wrote:

Almost end of week update:

HID is now pluggable itself! You can add
#include "Mouse.h"
#include "Keyboard.h"
#include "HID.h"
and the descriptor will be the combination of the submodules' descriptors. The method is not very memory efficient; however you can still write a library which plugs on PluggableUSB directly, embedding in it an HID implementation and only the functions you need.

the PR now contains @matthijskooijman C++11 patches to get rid of a nasty warning

next step: enable lto (reduces flash occupation by 400 bytes) https://gist.github.com/facchinm/432fbb8ca4d9705644bd

@NicoHood @matthijskooijman @obra @follower @cmaglie @madrang @damellis @PaulStoffregen @weizenspreu and all people interested in USB on AVR, please test the latest @ArduinoBot image to make sure that nothing breaks spectacularly before merging

Thank you very much!


Reply to this email directly or view it on GitHub.

@NicoHood

This comment has been minimized.

Show comment
Hide comment
@NicoHood

NicoHood Jul 4, 2015

Contributor

@yyyc514 of course it is interrupt driven. Not sure if your idea then still makes sense.

@ facchinm Also see this:
NicoHood/HID#42

Going to test this sooner or later... Sounds good at least!

Contributor

NicoHood commented Jul 4, 2015

@yyyc514 of course it is interrupt driven. Not sure if your idea then still makes sense.

@ facchinm Also see this:
NicoHood/HID#42

Going to test this sooner or later... Sounds good at least!

facchinm added some commits Jun 26, 2015

save RAM content overridden by bootloader magic
and restore it in case of aborted reboot
use RAMEND-1 as suggested by @yyyc514 in PR #2474

of course it's not a real solution but we cannot force everyone to update the bootloader using an external programmer
allow HID submodules to create runtime descriptors
with this PR you can add

\#include Keyboard.h
\#include Mouse.h
\#include HID.h

in the top of the sketch and you will expose a Mouse+Keyboard

From the library pow, simply add

static HID_Descriptor cb = {
	.length = sizeof(_hidReportDescriptor),
	.descriptor = _hidReportDescriptor,
};
static HIDDescriptorListNode node(&cb);
HID.AppendDescriptor(&node);

in the class' constructor and you are done!
remove stub MIDIUSB library
revert this commit when it's time to integrate this library
remove CompleteHID library
expect way more interesting user-generated libraries

@cmaglie cmaglie merged commit af290fc into arduino:master Jul 16, 2015

@NicoHood

This comment has been minimized.

Show comment
Hide comment
@NicoHood

NicoHood Jul 16, 2015

Contributor

huh? that was quick. I didnt even had a chance to test this yet.
Did you got feedback from somewhere else? I am a little surprised that nothing happened related to USB for so long, and now this fix is merged so fast.

Not that I feel bad about it, I should really go ahead and try this now!

Contributor

NicoHood commented Jul 16, 2015

huh? that was quick. I didnt even had a chance to test this yet.
Did you got feedback from somewhere else? I am a little surprised that nothing happened related to USB for so long, and now this fix is merged so fast.

Not that I feel bad about it, I should really go ahead and try this now!

@cmaglie

This comment has been minimized.

Show comment
Hide comment
@cmaglie

cmaglie Jul 16, 2015

Member

I think that this is one of the most-desired patch ever (just look the number of PR/Issues), so we decided to merge it upstream to accelerate tests and adoption.

On our side we tested it on everything, BTW this patch is not yet released, so we are still in time to fix bugs or (in the worst case but I hope not :)) completely revert it.

Member

cmaglie commented Jul 16, 2015

I think that this is one of the most-desired patch ever (just look the number of PR/Issues), so we decided to merge it upstream to accelerate tests and adoption.

On our side we tested it on everything, BTW this patch is not yet released, so we are still in time to fix bugs or (in the worst case but I hope not :)) completely revert it.

@NicoHood

This comment has been minimized.

Show comment
Hide comment
@NicoHood

NicoHood Jul 16, 2015

Contributor

Good idea, I think its a good idea to improve the usb stack.

Contributor

NicoHood commented Jul 16, 2015

Good idea, I think its a good idea to improve the usb stack.

@yahesh

This comment has been minimized.

Show comment
Hide comment
@yahesh

yahesh Jul 16, 2015

What do you mean by "quick"? This topic took now 1.5 years to come any further.

Am 16.07.2015 um 15:36 schrieb Nico notifications@github.com:

huh? that was quick. I didnt even had a chance to test this yet.
Did you got feedback from somewhere else? I am a little surprised that nothing happened related to USB for so long, and now this fix is merged so fast.

Not that I feel bad about it, I should really go ahead and try this now!


Reply to this email directly or view it on GitHub.

yahesh commented Jul 16, 2015

What do you mean by "quick"? This topic took now 1.5 years to come any further.

Am 16.07.2015 um 15:36 schrieb Nico notifications@github.com:

huh? that was quick. I didnt even had a chance to test this yet.
Did you got feedback from somewhere else? I am a little surprised that nothing happened related to USB for so long, and now this fix is merged so fast.

Not that I feel bad about it, I should really go ahead and try this now!


Reply to this email directly or view it on GitHub.

@cmaglie cmaglie added this to the Release 1.6.6 milestone Jul 16, 2015

@NicoHood

This comment has been minimized.

Show comment
Hide comment
@NicoHood

NicoHood Jul 16, 2015

Contributor

Laugh out loud :D

Yes I know. And this PR started about a month ago and is now merged. Nevermind, its GOOD that there IS progress.

Contributor

NicoHood commented Jul 16, 2015

Laugh out loud :D

Yes I know. And this PR started about a month ago and is now merged. Nevermind, its GOOD that there IS progress.

@facchinm

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Jul 16, 2015

Member

Initial howto on writing a PluggableUSB or PluggableHID based library

https://github.com/arduino/Arduino/wiki/PluggableUSB-and-PluggableHID-howto

Member

facchinm commented Jul 16, 2015

Initial howto on writing a PluggableUSB or PluggableHID based library

https://github.com/arduino/Arduino/wiki/PluggableUSB-and-PluggableHID-howto

@NicoHood

This comment has been minimized.

Show comment
Hide comment
@NicoHood

NicoHood Jul 16, 2015

Contributor

May you eliminate the u8 which only causes problems?

Edit: Will have a loom of course ;) Lookin good.

Contributor

NicoHood commented Jul 16, 2015

May you eliminate the u8 which only causes problems?

Edit: Will have a loom of course ;) Lookin good.

@NicoHood

This comment has been minimized.

Show comment
Hide comment
@NicoHood

NicoHood Jul 18, 2015

Contributor

Any chance to see a pluggable keyboard layout?
NicoHood/HID#33

Contributor

NicoHood commented Jul 18, 2015

Any chance to see a pluggable keyboard layout?
NicoHood/HID#33

@facchinm

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Jul 20, 2015

Member

Well Nico, a library providing multiple layouts is now possible, you can write it by yourself and we'll be glad to include it in the library manager 馃槈

Member

facchinm commented Jul 20, 2015

Well Nico, a library providing multiple layouts is now possible, you can write it by yourself and we'll be glad to include it in the library manager 馃槈

@NicoHood

This comment has been minimized.

Show comment
Hide comment
@NicoHood

NicoHood Jul 21, 2015

Contributor

Looking at the code now, adding notes here via edit:

  • Overall it looks very nice to me and a simple example just works fine.
  • Found a typo in the instruction: Then you can then perform USB writes calling, for example,
  • begin() should return a bool and if possible it should wait till the HID is initialized. Maybe this is impossible, but I noticed problems like this: NicoHood/HID#29 What about while(!HID); like the Serial? Or remove it completely.
  • u8 should be replaced with uint8_t if you ask me.
  • Keyboard/Mouse.begin() should reset the report to zero. It should be inlined into the header to save flash.
  • It would be very helpful to set the EP Size flexible, not only 64
Contributor

NicoHood commented Jul 21, 2015

Looking at the code now, adding notes here via edit:

  • Overall it looks very nice to me and a simple example just works fine.
  • Found a typo in the instruction: Then you can then perform USB writes calling, for example,
  • begin() should return a bool and if possible it should wait till the HID is initialized. Maybe this is impossible, but I noticed problems like this: NicoHood/HID#29 What about while(!HID); like the Serial? Or remove it completely.
  • u8 should be replaced with uint8_t if you ask me.
  • Keyboard/Mouse.begin() should reset the report to zero. It should be inlined into the header to save flash.
  • It would be very helpful to set the EP Size flexible, not only 64
@facchinm

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Jul 21, 2015

Member

Adding the bool() operator is a good idea!
About u8, they will be replaced in the libs when PR #3542 will be merged.
Typo fixed, thanks 馃槈

Endpoint flexible size is a bit of a mess, but if you come up with a solution I'll be very happy to review it. Also, feel free to open PRs for Keyboard and Mouse libraries modifications.

Member

facchinm commented Jul 21, 2015

Adding the bool() operator is a good idea!
About u8, they will be replaced in the libs when PR #3542 will be merged.
Typo fixed, thanks 馃槈

Endpoint flexible size is a bit of a mess, but if you come up with a solution I'll be very happy to review it. Also, feel free to open PRs for Keyboard and Mouse libraries modifications.

@NicoHood

This comment has been minimized.

Show comment
Hide comment
@NicoHood

NicoHood Jul 21, 2015

Contributor

I am currently working on a PR to backport all my HID Project fixes. Already done with the major core fixes, HID libraries will follow.

The flexible EP_SIZE needs a bit more core changes, however I applied my simple fix for now in the PR (soon).

What is more important is this:
NicoHood/HID@daa80ec...devdiff-5478f073c4245fb29e2a0975e99c647eR101

At the moment you set those values wrong. USB cannot be pluggable in every situation. So I dont know how to solve this the best way, but we should at least try to add a correct CDC descriptor.

Let me give my fixes a last check and I'll open a new PR for this.

Contributor

NicoHood commented Jul 21, 2015

I am currently working on a PR to backport all my HID Project fixes. Already done with the major core fixes, HID libraries will follow.

The flexible EP_SIZE needs a bit more core changes, however I applied my simple fix for now in the PR (soon).

What is more important is this:
NicoHood/HID@daa80ec...devdiff-5478f073c4245fb29e2a0975e99c647eR101

At the moment you set those values wrong. USB cannot be pluggable in every situation. So I dont know how to solve this the best way, but we should at least try to add a correct CDC descriptor.

Let me give my fixes a last check and I'll open a new PR for this.

@NicoHood

This comment has been minimized.

Show comment
Hide comment
@NicoHood

NicoHood Jul 21, 2015

Contributor

Edit:

  • Also think about making the whole USB core a library if possible. Then one can deactivate it if not needed since it uses a lot of flash and ram. You need a fix for the 32u4 since the Arduino Bootloader doesnt clear the USB Interrupts correct. I have this in my HID project, the empty ISR of the usb clock just needs to be added (maybe weak) to the core. Maybe it would make sense here to update the bootloader once more with the RAMEND fix. So people who really want to deactivate USB have to update the bootloader but we then won't need those workarounds.
Contributor

NicoHood commented Jul 21, 2015

Edit:

  • Also think about making the whole USB core a library if possible. Then one can deactivate it if not needed since it uses a lot of flash and ram. You need a fix for the 32u4 since the Arduino Bootloader doesnt clear the USB Interrupts correct. I have this in my HID project, the empty ISR of the usb clock just needs to be added (maybe weak) to the core. Maybe it would make sense here to update the bootloader once more with the RAMEND fix. So people who really want to deactivate USB have to update the bootloader but we then won't need those workarounds.
@NicoHood

This comment has been minimized.

Show comment
Hide comment
@NicoHood

NicoHood Jul 21, 2015

Contributor

There we go:
#3640

Edit: very nice work overall. Took some time to understand the whole thing. Still looking through the code. Just tell me how we should continue.

Contributor

NicoHood commented Jul 21, 2015

There we go:
#3640

Edit: very nice work overall. Took some time to understand the whole thing. Still looking through the code. Just tell me how we should continue.

@NicoHood

This comment has been minimized.

Show comment
Hide comment
@NicoHood

NicoHood Aug 4, 2015

Contributor

@facchinm Hey, you did a lot of great work. Would you be so kind to have a look at the PR to apply my small fixes?

If you disagree with the CDC Events and control line stuff I can remove it. There is another PR my mathijs I think who did something different, maybe better than that. But the rest of my PR is not much to look at but would make the core more flexible and fixes a few problems.

Removed, now this PR can be used:
#3640

Contributor

NicoHood commented Aug 4, 2015

@facchinm Hey, you did a lot of great work. Would you be so kind to have a look at the PR to apply my small fixes?

If you disagree with the CDC Events and control line stuff I can remove it. There is another PR my mathijs I think who did something different, maybe better than that. But the rest of my PR is not much to look at but would make the core more flexible and fixes a few problems.

Removed, now this PR can be used:
#3640

@cuitoldfish

This comment has been minimized.

Show comment
Hide comment
@cuitoldfish

cuitoldfish Sep 19, 2016

Contributor

hello, why there are two device descriptor for only one device --- if the wLength is 8 then USB_DeviceDescriptorB otherwise USB_DeviceDescriptor?
and, according to USB Interface Association Descriptor Device Class Code and Use Modelin July23, 2003, if Interface Association Descriptor is used, then the device class code must be 0xEF, but in USB_DeviceDescriptor the code is 0x00.

hello, why there are two device descriptor for only one device --- if the wLength is 8 then USB_DeviceDescriptorB otherwise USB_DeviceDescriptor?
and, according to USB Interface Association Descriptor Device Class Code and Use Modelin July23, 2003, if Interface Association Descriptor is used, then the device class code must be 0xEF, but in USB_DeviceDescriptor the code is 0x00.

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Sep 19, 2016

Member

This piece of code is intended to handle the case when the host OS is not aware of composite devices (like pre SP2 XP and vintage OSX) so USB_DeviceDescriptor "advertises" the board as a CDC only device.

Member

facchinm replied Sep 19, 2016

This piece of code is intended to handle the case when the host OS is not aware of composite devices (like pre SP2 XP and vintage OSX) so USB_DeviceDescriptor "advertises" the board as a CDC only device.

This comment has been minimized.

Show comment
Hide comment
@cuitoldfish

cuitoldfish Sep 20, 2016

Contributor

Arduino is using multi-interface and the IAD, but give the class code as CDC(02H), it seems this doesn't conform to the USB Interface Association Descriptor Device Class Code and Use Mode, which says

Devices that us the IAD must use the device class, subclass and protocol codes as defined in the example device descriptor illustrated in Table 1-1

I can not upload that table picture to here, but in that table, the bDeviceClass is EFH, bDeviceSubClass is 02H, bDeviceProtocol is 01H.
this unconformity will let some OS who conform to that specification failed to enumerate arduino board, such as Nucleus's usb host.

is arduino intended to support other embedded OS as the host? or only Windows/Linux/iOS?

Contributor

cuitoldfish replied Sep 20, 2016

Arduino is using multi-interface and the IAD, but give the class code as CDC(02H), it seems this doesn't conform to the USB Interface Association Descriptor Device Class Code and Use Mode, which says

Devices that us the IAD must use the device class, subclass and protocol codes as defined in the example device descriptor illustrated in Table 1-1

I can not upload that table picture to here, but in that table, the bDeviceClass is EFH, bDeviceSubClass is 02H, bDeviceProtocol is 01H.
this unconformity will let some OS who conform to that specification failed to enumerate arduino board, such as Nucleus's usb host.

is arduino intended to support other embedded OS as the host? or only Windows/Linux/iOS?

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Sep 20, 2016

Member

Sorry Lui but I can't get your point. We are currently using USB_DeviceDescriptorB (0xEF,0x02,0x01 as per IAD specifications) on all OS which support composite devices, so it shouldn't be problematic in any design.
Our main target for the USB related code are Linux/Win/OSX since we use it primarily for CDC, but if you have a proposal please open a pull request which solves the problems in your system and we'll take care of testing it in all the OSs combinations.

Member

facchinm replied Sep 20, 2016

Sorry Lui but I can't get your point. We are currently using USB_DeviceDescriptorB (0xEF,0x02,0x01 as per IAD specifications) on all OS which support composite devices, so it shouldn't be problematic in any design.
Our main target for the USB related code are Linux/Win/OSX since we use it primarily for CDC, but if you have a proposal please open a pull request which solves the problems in your system and we'll take care of testing it in all the OSs combinations.

This comment has been minimized.

Show comment
Hide comment
@cuitoldfish

cuitoldfish Sep 20, 2016

Contributor

because the wLength for device descriptor should not be 8 in normal case, the following code pieces shows arduino generally send USB_DeviceDescriptor but not USB_DeviceDescriptorB which conform with IAD specification..

    if (USB_DEVICE_DESCRIPTOR_TYPE == t)
    {
        if (setup.wLength == 8)
            _cdcComposite = 1;
        desc_addr = _cdcComposite ?  (const u8*)&USB_DeviceDescriptorB : (const u8*)&USB_DeviceDescriptor;
    }
Contributor

cuitoldfish replied Sep 20, 2016

because the wLength for device descriptor should not be 8 in normal case, the following code pieces shows arduino generally send USB_DeviceDescriptor but not USB_DeviceDescriptorB which conform with IAD specification..

    if (USB_DEVICE_DESCRIPTOR_TYPE == t)
    {
        if (setup.wLength == 8)
            _cdcComposite = 1;
        desc_addr = _cdcComposite ?  (const u8*)&USB_DeviceDescriptorB : (const u8*)&USB_DeviceDescriptor;
    }

This comment has been minimized.

Show comment
Hide comment
@cuitoldfish

cuitoldfish Sep 21, 2016

Contributor

then why you say Arduino use USB_DeviceDescriptorB?

Contributor

cuitoldfish replied Sep 21, 2016

then why you say Arduino use USB_DeviceDescriptorB?

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Sep 30, 2016

Member

I double checked and, indeed, we are sending USB_DeviceDescriptor . I always trusted the _cdcComposite flag to be correct but it looks I was wrong. Would you mind submitting a PR to fix this while testing it on your platform? The updated code should work on all Windows (from XP to 10) , recent Linux and OSX and enumerate correctly when plugging PluggableUSB modules.

Member

facchinm replied Sep 30, 2016

I double checked and, indeed, we are sending USB_DeviceDescriptor . I always trusted the _cdcComposite flag to be correct but it looks I was wrong. Would you mind submitting a PR to fix this while testing it on your platform? The updated code should work on all Windows (from XP to 10) , recent Linux and OSX and enumerate correctly when plugging PluggableUSB modules.

@cuitoldfish

This comment has been minimized.

Show comment
Hide comment
@cuitoldfish

cuitoldfish Oct 6, 2016

Contributor

yes, I'd like to fix this and submit a PR. At the meantime, I'm using Leonardo which is an old version Arduino board and not supported anymore, I'm afraid I can't take my board back if I download updated firmware to it if there is something wrong with the firmware. could you please give some advice on how to find the matching firmware for Leonardo REV 3B board?

Contributor

cuitoldfish commented on 2c5dd20 Oct 6, 2016

yes, I'd like to fix this and submit a PR. At the meantime, I'm using Leonardo which is an old version Arduino board and not supported anymore, I'm afraid I can't take my board back if I download updated firmware to it if there is something wrong with the firmware. could you please give some advice on how to find the matching firmware for Leonardo REV 3B board?

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Oct 6, 2016

Member

Leonardo is still supported, and its core is 100% compatible with Micro. If you should run into trouble, simply re-upload a sketch by using an older version of the IDE. All the code you can upload without a programmer won't modify the bootloader so you are completely safe

Member

facchinm replied Oct 6, 2016

Leonardo is still supported, and its core is 100% compatible with Micro. If you should run into trouble, simply re-upload a sketch by using an older version of the IDE. All the code you can upload without a programmer won't modify the bootloader so you are completely safe

This comment has been minimized.

Show comment
Hide comment
@cuitoldfish

cuitoldfish Oct 6, 2016

Contributor

well, since I need to change the USBCore.cpp, I have to use DFU to update the updated firmware. If I really run into trouble, I think re-upload will not let me pass because the sketch doesn't include the usb-serial functionality. what I need is to chose the matching fimware fetched from https://github.com/arduino/Arduino/tree/master/hardware/arduino/avr/firmwares/atmegaxxu2, such as Arduino-usbserial-uno.hex. and My question is where can I get it for 32u4? it seems there isn't any version for 32u4 in that folder, only 16u2 mega, 16u2 uno available there.

Contributor

cuitoldfish replied Oct 6, 2016

well, since I need to change the USBCore.cpp, I have to use DFU to update the updated firmware. If I really run into trouble, I think re-upload will not let me pass because the sketch doesn't include the usb-serial functionality. what I need is to chose the matching fimware fetched from https://github.com/arduino/Arduino/tree/master/hardware/arduino/avr/firmwares/atmegaxxu2, such as Arduino-usbserial-uno.hex. and My question is where can I get it for 32u4? it seems there isn't any version for 32u4 in that folder, only 16u2 mega, 16u2 uno available there.

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Oct 6, 2016

Member

Wait a second, let me explain a couple of things.
If you are on a stock Leonardo (thus 32u4 processor) you have a bootloader (NON dfu capable and NON user writable) and a sketch (user programmable). The bootloader is called Caterina and can't be flashed via DFU, only via programmer. The 16u2 firmwares are, name says it, firmwares 馃槃 and they are used in UNO and similar boards as USB-to-serial converters.
As long as you want to change USBCore.cpp the modification will only apply to user sketch, and in case you break something you can safely boot into bootloader by double clicking the reset button (the LED will start a fade animation)

Member

facchinm replied Oct 6, 2016

Wait a second, let me explain a couple of things.
If you are on a stock Leonardo (thus 32u4 processor) you have a bootloader (NON dfu capable and NON user writable) and a sketch (user programmable). The bootloader is called Caterina and can't be flashed via DFU, only via programmer. The 16u2 firmwares are, name says it, firmwares 馃槃 and they are used in UNO and similar boards as USB-to-serial converters.
As long as you want to change USBCore.cpp the modification will only apply to user sketch, and in case you break something you can safely boot into bootloader by double clicking the reset button (the LED will start a fade animation)

This comment has been minimized.

Show comment
Hide comment
@cuitoldfish

cuitoldfish Oct 6, 2016

Contributor

so if my understanding is right, USBCore.cpp is not compiled and run on Leonardo? the Caterina is responsible for the usb CDC communication and virtual serial functionality with computer?

Contributor

cuitoldfish replied Oct 6, 2016

so if my understanding is right, USBCore.cpp is not compiled and run on Leonardo? the Caterina is responsible for the usb CDC communication and virtual serial functionality with computer?

This comment has been minimized.

Show comment
Hide comment
@facchinm

facchinm Oct 6, 2016

Member

No, the microcontroller is unique, so Caterina handles the bootloader/sketch loading phase, then launches the sketch (which runs the Arduino core so uses USBCore.cpp)

Member

facchinm replied Oct 6, 2016

No, the microcontroller is unique, so Caterina handles the bootloader/sketch loading phase, then launches the sketch (which runs the Arduino core so uses USBCore.cpp)

This comment has been minimized.

Show comment
Hide comment
@cuitoldfish

cuitoldfish Oct 6, 2016

Contributor

If I upload one SPI related user sketch, then will it overwritten the default sketch including Arduino core?

Contributor

cuitoldfish replied Oct 6, 2016

If I upload one SPI related user sketch, then will it overwritten the default sketch including Arduino core?

This comment has been minimized.

Show comment
Hide comment
@NicoHood

NicoHood Oct 6, 2016

Contributor

you are just dealing with the firmware. ignore the bootloader (FYI: no dfu, 32u4 does not even has a dfu bootloader is has a so called caternia cdc bootloader).

When you compile an arduino sketch the whole firmware including usb core is always compiled and used. so whatever source file you change, the leonardo will also change its usb behaviour. thatswhy we can code keyboard, mouse and gamepads. that is 100% independant from the bootloader. thatshwy also you sketch is always >4kb as the usb core takes so much space at a minimum.

If your usb core has a critical bug that it does not work at all that is also no problem. Then you can hit upload with an unplugged device, press the reset button, insert the device, release the button and the bootloader will run. the arduino ide recognizes the new device and start the upload.

Contributor

NicoHood replied Oct 6, 2016

you are just dealing with the firmware. ignore the bootloader (FYI: no dfu, 32u4 does not even has a dfu bootloader is has a so called caternia cdc bootloader).

When you compile an arduino sketch the whole firmware including usb core is always compiled and used. so whatever source file you change, the leonardo will also change its usb behaviour. thatswhy we can code keyboard, mouse and gamepads. that is 100% independant from the bootloader. thatshwy also you sketch is always >4kb as the usb core takes so much space at a minimum.

If your usb core has a critical bug that it does not work at all that is also no problem. Then you can hit upload with an unplugged device, press the reset button, insert the device, release the button and the bootloader will run. the arduino ide recognizes the new device and start the upload.

This comment has been minimized.

Show comment
Hide comment
@cuitoldfish

cuitoldfish Oct 7, 2016

Contributor

guys thanks for all the clarification :)

Contributor

cuitoldfish replied Oct 7, 2016

guys thanks for all the clarification :)

This comment has been minimized.

Show comment
Hide comment
@cuitoldfish

cuitoldfish Oct 17, 2016

Contributor

@facchinm , one PR is submitted to try to solve this, let's have a discussion

Contributor

cuitoldfish replied Oct 17, 2016

@facchinm , one PR is submitted to try to solve this, let's have a discussion

@facchinm facchinm deleted the facchinm:testPlugUSB_PR branch Jan 4, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment