Pen Customizations API

[darktears]

Introduction

Newest generation of stylus/pens have more capabilities that is currently supported by the Pointer Events specification. They have some built-in memory which allows them to store and retrieve extra attributes or customizations to speed up drawing. They can store a preferred inking color (e.g. red), a preferred inking width (e.g. 2.0) and a preferred inking style (e.g. pencil vs brush). This API is mostly targeting digital artists who work with various pens and allow them to set them up like real pens. Then as they touch the screen the application can retrieve the properties and select them as drawing tools to speed up drawing. The other main use case is leveraging the pens which have the capability to scan real world colors and store them into the memory. Using this API web applications can retrieve the color scanned and use it. Think of it as a real world color picker.

I have a video showing the API in action in a mini drawing application that we built to showcase the API (watch with sound).

Read the complete Explainer.

Please provide all feedback below.

[darktears]

Pasting the WICG discourse feedback for history.

JS: This is a great feature that not only I can get the colors from the app but also get from the real objects back the app. Look forward to that!

Hugo: Colors among all applications should be easy to access and manage. This is the core value of what Darttears purposed. This Color API is what I’d like to see in the market!

Bubuche87: I am a newcomer and certainly not a professional. Don’t take anything I’ll say too seriously. Also, English is not my native language.
This seems interesting, but I have a ton of questions.
Are the pen painting colors or materials ? By that I mean two things: are colors just overwriting what is beneath them, or do they mix ( and how they mix will be influenced by the material). The second is the export of the painting to any other support than a screen. You can have a painting representing a river and a rock and the river will reflect light differently.
This is especially true for materials in 3D: the metal will not reflect light like the wood or some plastic.
From the point of view of the pen/surface, it would only be another data to store, alongside the color.
The natural evolution of that is to separate the presentation of the information and the information itself. You would have the layer(s) containing the data, the layer displaying it and a formula to mix the incoming data from the pen, and what is already there in the layer.
In it’s basic form it would look like “data=rgba, display=show data directly, mix=erase” A slightly different one would be “data=hsl, display=hls_to_rgb”, with a smart mix ( the data sent by the pen can indicate what is the desired action: darken, brighten, desaturate etc ) And a more complex one would just pull the fragment shader, so you can composed your informations and see the result you’ll have at the end.
But it was originally a question and it became that …
Anyway, the question was: what kind of data is the pen painting? (Sorry for the long post)
What kind of color space is supported ? ( RGB of course but: hsl, HSV, cymk, ciel, paleted colors … ) Is the notion of layers built-in that ?
I have difficulties seeing where the API starts and ends and where the software starts and ends. What in the video is built on top the API but is not part of it, and what is actually part of the API ?
What are the typical use cases of this API ? Isn’t that a bit redundant with already existing softwares proposing that kind of functionality?

darktears : Hi Bubuche87. thanks for the long message.
We’re talking about regular digital pen nothing that special with them. They just have a tiny memory inside of them where we can store some information.
So I touched a bit on that in the explainer but currently USI 1.0 pens supports only 140 colors or so, USI 2.0 pens supports 24bits colors and I personally requested the USI forum to extend this to more color space like you mentioned. Ideally we should cover what the web covers and what’s available on the market. The discussion has not yet started at USI but when I do I can update that thread so you can provide input. Back to the kind of data the pen is storing, it is just bits nothing really more than this.
Did you take a look at the explainer about the proposed APIs? There are not that many APIs that we’re adding. It’s on top of existing Pointer Events capabilities. The video is showing a tiny drawing application that we put ourselves as a showcase to demonstrate the API. We don’t intend to make it a full fledge drawing application it was the easiest way to test the API. Ideally the API would be adopted by https://beta.tldraw.com/, Sketchpad - Draw, Create, Share! etc etc.