Graphics tablet input

[elmindreda]

Position, pressure, tilt and buttons.

[dmitshur]

Can it be treated like a joystick and use that API?

It also seems similar to multitouch input, except that typically doesn't have extra degrees of freedom like pressure and tilt.

[fogfx]

Treating a stylus as a joystick would be odd to say the least.

Stylus rotation support might also be something to consider (Wacom's art pens use it, for example), if that information is easily available.

[Cooldude234]

I know stylus's are usually treated as a mouse with an axis for pressure or angles.
But if you are going to do this I would highly recommend allowed raw input of the graphics pen its self, as screen resolutions are usually much lower than the resolution of input provided by the pen.

Having raw input from a graphics tablet would be highly useful (and would make this library much more beneficial to use than others).

[dmitshur]

Since this isn't mentioned already, I would highly recommend looking at and considering Pointer Events that Microsoft has worked on and brought forward as a W3C recommendation for browsers.

http://www.w3.org/TR/pointerevents/

[RotemShi]

For future reference, I was able to get pressure readings working under Windows in a GLFW app by implementing the suggestions on this blog post:

http://backworlds.com/under-pressure/

Note that the links to the code samples on that page are broken, and the correct link seems to be:
http://backworlds.com/wp-content/uploads/2013/07/pressure_app.zip

I do not know if this is the best or most efficient method, just one that worked.

[lennart]

has there been any work done for tablet support yet?

and

how is this related to the touch branch regarding touch events/gestures from touchpads. (If I understand PointerEvents correctly this would be a standard for both tablet-events (wacom) and touch-events (trackpads, surface, ipad)

I have been looking through the examples by wacom to read out additional data from mouse events and it seems like an easy way to get out pressure (and maybe tilt) information from the events received in the mouseDown callback that is already implemented in cocoa_window.m

However I am not sure about the complexity to get this working cross-platform.

[camoy]

I wrote a patch a few years ago that implements @lennart's suggestion: pressure on OSX through mouse events. Alas, I believe that OSX is likely the easiest platform to support tablet input.

Tablet input is definitely a feature I would love to see implemented in GLFW.

[openGLnoob]

Has anyone work with GLFW and Wintab? I'm struggling the last few days to make it, but not Easytab or any library can't work with GLFW. Guys, need your help. For example Easytab needs Winapi's parameters like WPARAM, LPARAM, MESSAGE and another. How can I get it? HWND I get from glfwGetWin32Window().

[felselva]

I will check what is the status of APIs available to add this support on X11, Wayland and Windows. Gdk does provide events to check tablet inputs (pressure, tilt, rotation, and tablet buttons). What I'm not sure is the nature of the dependencies (on X11 seems to be just the extended input protocol, so not a big deal, but I'm not sure). Wayland has support for tablet input (seems to be recent). As for Windows, if Wintab is a requirement, what is the license (is it a third party or is it part of the win32 API)? If not, it would be simpler to just use the events.

I have a tablet (wacom), so I can make and test some implementations.

[fogfx]

Windows is going to be interesting as I believe as of Windows 10 Anniversary, you're supposed to use the Ink API (or whatever it's called), whereas for previous versions of Windows you should use wintab (as apparently older versions of the Ink API suck). IIRC some newer graphics tablets don't even have wintab drivers? Fun.

I actually tried my hand at a wintab implentation a couple years ago and got a partial implementation working. Unfortunately I developed it in a VM and no longer have that VHD.

It would probably be a good idea to come up with an API design. I didn't bother with explicit support for the tablet itself, just the stylus; usually the tablet buttons are mapped to hotkeys anyway. Maybe mapping and absolute/relative coordinates settings could be exposed though?

Some stylus-specific things that should be supported:

• pressure, of course (normalized)
• stylus X and Y tilt (radians)
• stylus rotation support would be nice but I don't know if anything other than Wacom's art pens (or their apparently deprecated cursor mice) actually support it (radians)
• stylus unique ID; useful for per-stylus tool settings, common in art apps (int)
• stylus button states
• which of the tip or eraser is being used
• whether a stylus is within the tablet's active area

Would this need a separate stylus position callback (for backwards compatibility reasons)? A drawback to having a separate callback is that some people actually use the mouse that some tablets come with; does that use the stylus position callback or the cursor position callback? From the standpoint of the tablet, both of these are tablet input devices but the programmer would probably conceptually treat them as completely different things. Or a separate callback could be abandoned and the user would instead need to call the getters for pressure, tilt, etc.

IMO it would be nice if GLFW 4 unified mouse, stylus, and maybe even touch under a single input API, with the concept of "pressed" and "unpressed" being generalized as pressure where the former is 1.0 and the latter 0.0, along with sane defaults for things like tilt that are currently only used by styluses. Especially if there are ever plans to support peripherals like the Surface Dial.

[felselva]

Some stylus-specific things that should be supported:

pressure, of course (normalized)
stylus X and Y tilt (radians)
stylus rotation support would be nice but I don't know if anything other than Wacom's art pens (or their apparently deprecated cursor mice) actually support it (radians)
stylus unique ID; useful for per-stylus tool settings, common in art apps (int)
stylus button states
which of the tip or eraser is being used
whether a stylus is within the tablet's active area

Which raises the question: what about other tablets? I wonder if they are detectable by the Wacom API (maybe they were built to be).

I'm trying to start the work on X11, but the lack of documentation is slowing me down. I found /usr/include/libwacom (homepage) on my system, but I don't know if I should use it directly, or if there's a proxy library for X11. On Wayland, at first glance it seems easier. I found some libinput_* functions that probably expose the tablets.

I think the tablet support should be enabled/disabled by a flag (disabled by default) at build time, because not all computers have the (dynamic) libraries installed.

IMO it would be nice if GLFW 4 unified mouse, stylus, and maybe even touch under a single input API, with the concept of "pressed" and "unpressed" being generalized as pressure where the former is 1.0 and the latter 0.0, along with sane defaults for things like tilt that are currently only used by styluses. Especially if there are ever plans to support peripherals like the Surface Dial.

I don't know how this would work with support for multiple mouse, though (#827).

[gracicot]

I don't know how this would work with support for multiple mouse, though (#827).

Since we talk about unified api, wouldn't be logical to handle multiple mouse the same way multi-touch is handled?

[fogfx]

Which raises the question: what about other tablets? I wonder if they are detectable by the Wacom API (maybe they were built to be).

Wintab's supposed to be the de-facto cross-vendor tablet API, but again the rift in tablet input APIs on Windows complicates things; realistically we'd want a variety of tablets to test with.

I think the tablet support should be enabled/disabled by a flag (disabled by default) at build time, because not all computers have the (dynamic) libraries installed.

You should just be able to try loading the libraries, and if that fails then you don't get tablet input; I don't think this requires any compile-time handling.

Also, unified input discussion should probably go into its own issue; that was just an aside.

[eloraiby]

@elmindreda Support for pen/tablet can be added easily, but it will only be supported with Windows 8 and later, is that OK for you ?

[elmindreda]

@eloraiby You mean WM_POINTER? That's fine for initial support.

[anael-seghezzi]

For those interested, here is one way to get tablet pressure without modifying glfw's code for Windows and Linux. You will need glfw3native.h and easytab.h.

#ifdef WIN32
   HWND w32window = glfwGetWin32Window(ctoy__window); MSG msg;
   if (PeekMessageW(&msg, NULL, 0, 0, PM_NOREMOVE)) {
      if (EasyTab_HandleEvent(msg.hwnd, msg.message, msg.lParam, msg.wParam) == EASYTAB_OK)
         float pressure = EasyTab->Pressure;
   }
#elif defined(__linux__)
   Display *xdisplay = glfwGetX11Display();
   int count = XPending(xdisplay);
   if (count > 0) {
      XEvent event;
      XPeekEvent(xdisplay, &event);
      if (EasyTab_HandleEvent(&event) == EASYTAB_OK)
         float pressure = EasyTab->Pressure;
   }
#endif
glfwPollEvents();

I'm not sure how to do the same for OSX without modifying cocoa_window.m (need to access [event pressure]). Also this code can only read the first event in the queue.

[eloraiby]

@anael-seghezzi, @RajaLehtihet has already integrated the windows support: pull request. I think, she's waiting on @elmindreda to proceed with other platforms.

[anael-seghezzi]

@elmindreda is there a way to access all native events ? Because even with glfw3native I can only access the first event from the queue. If not what do you recommend would be the minimal code change I could do ?

[r-lyeh]

For reference:
Dunno if following project was discussed somewhere before...
https://github.com/ApoorvaJ/EasyTab

edit:
oops, overlooked anael's code :D

[kantoniak]

Will this feature be supported in the future? Last comment is over a year old.

[jitspoe]

I managed to get this working with easytab by peeking at a specific message range before the glfw event polling.

First, after creating the GLFWwindow:

	HWND hwnd = glfwGetWin32Window(window);
	EasyTab_Load(hwnd);

Then:

		float pen_x = 0.0, pen_y = 0.0, pen_pressure = 0.0;
#ifdef WIN32
		MSG msg;
		while (PeekMessageW(&msg, NULL, WT_PACKET, WT_MAX, PM_REMOVE))
		{
			if (EasyTab_HandleEvent(msg.hwnd, msg.message, msg.lParam, msg.wParam) == EASYTAB_OK)
			{
				pen_pressure = EasyTab->Pressure;
				pen_x = EasyTab->PosX;
				pen_y = EasyTab->PosY;
			}
		}
#endif
		glfwPollEvents();

The positions are incorrect (scaled wrong and/or offset), though, so I still need to figure that out, but at least I'm getting input.

[ric-a-tic]

The positions are incorrect (scaled wrong and/or offset), though, so I still need to figure that out, but at least I'm getting input.

@jitspoe
Did you continue pursuing this code? I'm having trouble getting it to work. My tablet is recognized by Easytab, but PeekMessageW never returns true.

[jitspoe]

The positions are incorrect (scaled wrong and/or offset), though, so I still need to figure that out, but at least I'm getting input.

@jitspoe Did you continue pursuing this code? I'm having trouble getting it to work. My tablet is recognized by Easytab, but PeekMessageW never returns true.

Honestly, I completely forgot I even did this. I might have missed something. Here's my whole main function:

int main(int, char**)
{
	// Setup window
	glfwSetErrorCallback(glfw_error_callback);
	if (!glfwInit())
		return 1;
	GLFWwindow* window = glfwCreateWindow(1280, 720, "VectorSketch", NULL, NULL);
	if (window == NULL)
		return 1;
	glfwMakeContextCurrent(window);
	glfwSwapInterval(1); // Enable vsync
	HWND hwnd = glfwGetWin32Window(window);
	EasyTab_Load(hwnd);


	// Setup Dear ImGui context
	IMGUI_CHECKVERSION();
	ImGui::CreateContext();
	ImGuiIO& io = ImGui::GetIO(); (void)io;
	//io.ConfigFlags |= ImGuiConfigFlags_NavEnableKeyboard;  // Enable Keyboard Controls
	//io.ConfigFlags |= ImGuiConfigFlags_NavEnableGamepad;   // Enable Gamepad Controls

	// Setup Dear ImGui style
	ImGui::StyleColorsDark();
	//ImGui::StyleColorsClassic();

	// Setup Platform/Renderer bindings
	ImGui_ImplGlfw_InitForOpenGL(window, true);
	ImGui_ImplOpenGL2_Init();

	// Load Fonts
	// - If no fonts are loaded, dear imgui will use the default font. You can also load multiple fonts and use ImGui::PushFont()/PopFont() to select them.
	// - AddFontFromFileTTF() will return the ImFont* so you can store it if you need to select the font among multiple.
	// - If the file cannot be loaded, the function will return NULL. Please handle those errors in your application (e.g. use an assertion, or display an error and quit).
	// - The fonts will be rasterized at a given size (w/ oversampling) and stored into a texture when calling ImFontAtlas::Build()/GetTexDataAsXXXX(), which ImGui_ImplXXXX_NewFrame below will call.
	// - Read 'misc/fonts/README.txt' for more instructions and details.
	// - Remember that in C/C++ if you want to include a backslash \ in a string literal you need to write a double backslash \\ !
	//io.Fonts->AddFontDefault();
	//io.Fonts->AddFontFromFileTTF("../../misc/fonts/Roboto-Medium.ttf", 16.0f);
	//io.Fonts->AddFontFromFileTTF("../../misc/fonts/Cousine-Regular.ttf", 15.0f);
	//io.Fonts->AddFontFromFileTTF("../../misc/fonts/DroidSans.ttf", 16.0f);
	//io.Fonts->AddFontFromFileTTF("../../misc/fonts/ProggyTiny.ttf", 10.0f);
	//ImFont* font = io.Fonts->AddFontFromFileTTF("c:\\Windows\\Fonts\\ArialUni.ttf", 18.0f, NULL, io.Fonts->GetGlyphRangesJapanese());
	//IM_ASSERT(font != NULL);

	bool show_demo_window = true;
	bool show_another_window = false;
	ImVec4 clear_color = ImVec4(0.45f, 0.55f, 0.60f, 1.00f);


	// Generate spectral image
	glGenTextures(1, &g_SpectralTexture);
	glBindTexture(GL_TEXTURE_2D, g_SpectralTexture);
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
	glPixelStorei(GL_UNPACK_ROW_LENGTH, 0);


	glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, g_image_width, g_image_height, 0, GL_RGB, GL_UNSIGNED_BYTE, g_image_data);
	//(ImTextureID)(intptr_t)g_FontTexture


		// Main loop
	while (!glfwWindowShouldClose(window))
	{
		float pen_x = 0.0, pen_y = 0.0, pen_pressure = 0.0;
		// https://github.com/glfw/glfw/issues/403
#ifdef WIN32
		//HWND w32window = glfwGetWin32Window(ctoy__window);
		MSG msg;
		while (PeekMessageW(&msg, NULL, WT_PACKET, WT_MAX, PM_REMOVE))
		{
			if (EasyTab_HandleEvent(msg.hwnd, msg.message, msg.lParam, msg.wParam) == EASYTAB_OK)
			{
				pen_pressure = EasyTab->Pressure;
				pen_x = EasyTab->PosX;
				pen_y = EasyTab->PosY;
			}
		}
#elif defined(__linux__)
		Display* xdisplay = glfwGetX11Display();
		int count = XPending(xdisplay);
		if (count > 0) {
			XEvent event;
			XPeekEvent(xdisplay, &event);
			if (EasyTab_HandleEvent(&event) == EASYTAB_OK)
				float pressure = EasyTab->Pressure;
		}
#endif
		// Poll and handle events (inputs, window resize, etc.)
		// You can read the io.WantCaptureMouse, io.WantCaptureKeyboard flags to tell if dear imgui wants to use your inputs.
		// - When io.WantCaptureMouse is true, do not dispatch mouse input data to your main application.
		// - When io.WantCaptureKeyboard is true, do not dispatch keyboard input data to your main application.
		// Generally you may always pass all inputs to dear imgui, and hide them from your application based on those two flags.
		glfwPollEvents();
		wintab_loop();

		// Start the Dear ImGui frame
		ImGui_ImplOpenGL2_NewFrame();
		ImGui_ImplGlfw_NewFrame();
		ImGui::NewFrame();

		// 1. Show the big demo window (Most of the sample code is in ImGui::ShowDemoWindow()! You can browse its code to learn more about Dear ImGui!).
		if (show_demo_window)
			ImGui::ShowDemoWindow(&show_demo_window);

		// 2. Show a simple window that we create ourselves. We use a Begin/End pair to created a named window.
		{
			static float f = 0.0f;
			static int counter = 0;

			ImGui::Begin("Hello, world!");                          // Create a window called "Hello, world!" and append into it.

			ImGui::Text("This is some useful text.");               // Display some text (you can use a format strings too)
			ImGui::Checkbox("Demo Window", &show_demo_window);      // Edit bools storing our window open/close state
			ImGui::Checkbox("Another Window", &show_another_window);

			ImGui::SliderFloat("float", &f, 0.0f, 1.0f);            // Edit 1 float using a slider from 0.0f to 1.0f
			ImGui::ColorEdit3("clear color", (float*)& clear_color); // Edit 3 floats representing a color

			if (ImGui::Button("Button"))                            // Buttons return true when clicked (most widgets return true when edited/activated)
				counter++;
			ImGui::SameLine();
			ImGui::Text("counter = %d", counter);

			ImGui::Text("Application average %.3f ms/frame (%.1f FPS)", 1000.0f / ImGui::GetIO().Framerate, ImGui::GetIO().Framerate);
			ImGui::End();
		}

		// 3. Show another simple window.
		if (show_another_window)
		{
			ImGui::Begin("Another Window", &show_another_window);   // Pass a pointer to our bool variable (the window will have a closing button that will clear the bool when clicked)
			ImGui::Text("Hello from another window!");
			if (ImGui::Button("Close Me"))
				show_another_window = false;
			ImGui::End();
		}
		/*
		float test[NUM_POINTS];
		for (int i = 0; i < NUM_POINTS; ++i)
		{
			double freq_test = 440.0;
			int sample_rate = 48000;
			double sample = sin((i / (double)sample_rate) * freq_test * 2.0 * M_PI) * .25;
			test[i] = sample;
		}*/

		ImGui::PlotLines("Input", g_test_input, NUM_POINTS, 0, NULL, FLT_MAX, FLT_MAX, ImVec2(0, 64));
		ImGui::PlotLines("DST(real)", g_test_output1, NUM_POINTS, 0, NULL, FLT_MAX, FLT_MAX, ImVec2(0, 64));
		ImGui::PlotLines("DST(imaginary)", g_test_output2, NUM_POINTS, 0, NULL, FLT_MAX, FLT_MAX, ImVec2(0, 64));
		ImGui::PlotLines("DST Magnitude", g_test_mag, NUM_POINTS, 0, NULL, FLT_MAX, FLT_MAX, ImVec2(0, 64));
		ImGui::PlotLines("Reversed DST (real)", g_test_points, NUM_POINTS, 0, NULL, FLT_MAX, FLT_MAX, ImVec2(0, 64));
		ImGui::PlotLines("Reversed DST (imaginary)", g_test_points2, NUM_POINTS, 0, NULL, FLT_MAX, FLT_MAX, ImVec2(0, 64));
		ImDrawList* DrawList = ImGui::GetBackgroundDrawList();
		DrawList->AddCircle(ImVec2(pen_x, pen_y), 100 * pen_pressure, 0xffffffff);

		ImGui::Begin("Image");
		ImGui::Image((ImTextureID)(intptr_t)g_SpectralTexture, ImVec2(g_image_width, g_image_height));
		ImGui::End();
		//ImGui::Image((ImTextureID)(intptr_t)g_SpectralTexture, ImVec2(g_image_height, g_image_width), ImVec2(0.0, 1.0), ImVec2(1.0, 0.0));

		// Rendering
		ImGui::Render();
		int display_w, display_h;
		glfwGetFramebufferSize(window, &display_w, &display_h);
		glViewport(0, 0, display_w, display_h);
		glClearColor(clear_color.x, clear_color.y, clear_color.z, clear_color.w);
		glClear(GL_COLOR_BUFFER_BIT);

		// If you are using this code with non-legacy OpenGL header/contexts (which you should not, prefer using imgui_impl_opengl3.cpp!!), 
		// you may need to backup/reset/restore current shader using the commented lines below.
		//GLint last_program; 
		//glGetIntegerv(GL_CURRENT_PROGRAM, &last_program);
		//glUseProgram(0);
		ImGui_ImplOpenGL2_RenderDrawData(ImGui::GetDrawData());
		//glUseProgram(last_program);

		glfwMakeContextCurrent(window);
		glfwSwapBuffers(window);
	}

	// Cleanup
	ImGui_ImplOpenGL2_Shutdown();
	ImGui_ImplGlfw_Shutdown();
	ImGui::DestroyContext();

	glfwDestroyWindow(window);
	glfwTerminate();

	return 0;
}

Maybe you're missing a call to wintab_loop();?

[ric-a-tic]

@jitspoe
I am missing a call to wintab_loop()! I bet that's it. So sorry to be a bother, but could you post the code to that function too? (Or if it's obvious/generic code copied from somewhere, where might that be?) Google isn't helping me find references for wintab_loop(), and it doesn't seem to be part of the Easytab or Wintab headers.

[jitspoe]

@jitspoe I am missing a call to wintab_loop()! I bet that's it. So sorry to be a bother, but could you post the code to that function too? (Or if it's obvious/generic code copied from somewhere, where might that be?) Google isn't helping me find references for wintab_loop(), and it doesn't seem to be part of the Easytab or Wintab headers.

Apparently it does nothing, so that wasn't relevant at all. Sorry, I thought it was part of one of the libraries.

void wintab_loop()
{

}

[ric-a-tic]

Rats, I thought that seemed too easy. To better describe my problem: I can acquire one message at the start of the program: an EASYTAB_EVENT_NOT_HANDLED. If I PM_REMOVE, I only get the one instance of the message and then nothing, as if the message queue is not being replenished (?) (but I don't know why that would be). At this point I'm lost, so I'm just posting for others with a similar issue.

[anael-seghezzi]

You should try this pull request if you want good multi-platform tablet support in glfw:
#1445

Or here as a fork (the pull request is not added yet to the official repo, so use at your own risk):
https://github.com/anael-seghezzi/glfw/tree/tablet

[ric-a-tic]

It works! Thanks a ton - you're the MVP of GLFW art programs. Looking forward to incorporation into 3.4.