Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add touch input support for web shell #2426

Merged
merged 14 commits into from Dec 19, 2020
Merged

Add touch input support for web shell #2426

merged 14 commits into from Dec 19, 2020

Conversation

johannschopplich
Copy link
Contributor

@johannschopplich johannschopplich commented Dec 15, 2020

This pull requests adds buttons to the player shell for touch input capable devices. The following keyboard inputs are bound:

  • Arrow up
  • Arrow right
  • Arrow down
  • Arrow left
  • Enter
  • Escape

Thus, RM2k games are now playable on e.g. mobile devices. For RM2k3 games it get's trickier, since Shift and Numpad keys are possibly used as well. The web player is unusable on touch-based devices at the moment, so in my opinion this is a step up even if some keys are still missing. Do you think further buttons shall be added? Speaking for myself, I'd like to keep the layout as clean as possible.

You may see and test it in action here: https://realtroll.de/player/?game=el-dorado

el-dorado-easy-rpg

Further features:

  • Buttons are highlighted when touched.
  • Operating the d-pad while holding down also works. Inspired by @pulsejet's lines from his mkxp-web project. Thank you!

Notes:

  • Implementation is future-proof: key and code keyboard event properties are used along deprecated keyCode (like 13 for enter) and which. Later ones are still required for emscripten to interpret the keyboard events.
  • SVG for the d-pad is hand-crafted, therefore no license to add to the readme.
  • Media query @media all and … has been reduced to @media …, since type all is implied by defining a media query in the first place.

@Ghabry
Copy link
Member

Ghabry commented Dec 15, 2020

Jenkins: test this please <3

This sounds awesome

@johannschopplich
Copy link
Contributor Author

Is it possible to place files along the shell HTML and copy them over after building? I'd like to outsource styles into index.css and the touch-input script block into controls.js or something like that. The shell file gets confusing since its line number has grown... 😄

@fdelapena fdelapena added the Emscripten WebAssembly/JavaScript port for web browsers label Dec 15, 2020
@fdelapena fdelapena added this to the 0.6.3 milestone Dec 15, 2020
@Ghabry
Copy link
Member

Ghabry commented Dec 15, 2020

For your JavaScript you can modify emscripten-pre.js. this one is embedded.

For installing the CSS you must modify the CMakeLists at various places. Likely not something you want to figure out. But when you commit the CSS file I can write a patch for you.

@Ghabry
Copy link
Member

Ghabry commented Dec 15, 2020

@johannschopplich
Copy link
Contributor Author

Wonderful news, good to know. No rush, let's put that into another pull request and leave this one as a pure feature request.

I've just added one last commit to make buttons unselectable. In some cases when holding down for a while they got selected, what we want to avoid of course.

PR build looks good! Nothing to add. 🙂

@Ghabry
Copy link
Member

Ghabry commented Dec 15, 2020

What I noticed: Nothing major but for further enhancements:

Compared to the Android app the buttons are flipped. We have confirm left and cancel right. Though no idea why. This was based on the Gameboy but the Gameboy has confirm/A on the right, so we messed up here 😅.

In full screen mode the touch buttons disappear.


There is no button editor like we have on Android. Games that needs Shift or the numpad (yume Nikki) are unplayable. Also no way to move buttons

@fdelapena
Copy link
Contributor

fdelapena commented Dec 15, 2020

Great feature, thank you!

It seems there's a regression when the Player runs from an iframe (arrows not working after getting focus). This is good to keep it working because is used on some game hosting websites (itch.io, etc.).

Tests:

https://easyrpg.org/play/iframetest.html?path=master/?game=default
https://easyrpg.org/play/iframetest.html?path=pr2426/?game=default


The full screen button merged recently in master also disappears after initializing inside iframe, this was working in the 0.6.2 release:
https://easyrpg.org/play/iframetest.html?path=/?game=default

@johannschopplich
Copy link
Contributor Author

johannschopplich commented Dec 16, 2020

@Ghabry I intuitively inherited the Gameboy layout A/B button layouts as well. Didn't know it is switched on the Android ports (never tested). 😄 Should I chance them to match the Android player?

We could add a Shift button specifically for RM2k3 games. The engine gets logged anyway, so we could parse that before writing to the console.
For the Numpad, I guess we'll find a solution on the future.

Regarding buttons in fullscreen mode — I'll look for a simple solution to paint them on top of the canvas when fullscreen has been initialised.

@fdelapena Good catch. Will look into it.

@Ghabry
Copy link
Member

Ghabry commented Dec 16, 2020

I intuitively inherited the Gameboy layout A/B button layouts as well. Didn't know it is switched on the Android ports (never tested). smile Should I chance them to match the Android player?

Would be good. I guess most users are already used to the Android layout. At least I am. Pressed lots of times the wrong buttons in the web ui xD

We could add a Shift button specifically for RM2k3 games. The engine gets logged anyway, so we could parse that before writing to the console.

That's not a bad idea. But also not generic enough, I think we will post-bone this. Now that the basic ui is there it will be possible to implement a settings ui based on it.

@Ghabry
Copy link
Member

Ghabry commented Dec 16, 2020

Jenkins: test this please

@Ghabry
Copy link
Member

Ghabry commented Dec 16, 2020

@johannschopplich I think you broke the touch input. I see the buttons but nothing happens

@johannschopplich
Copy link
Contributor Author

Might be. I'll fix it. Having trouble compiling emscripten from the source.

@johannschopplich
Copy link
Contributor Author

I've used the chance to slightly refactor the shell: All Module-related code has been moved from the HTML file to the pre-script. The script handling button simulation is inside the index.html. Resulting in better SOC.

Also the CSS content has been expanded for increased readability.

Will build locally and tackle the remaining points.

@johannschopplich
Copy link
Contributor Author

johannschopplich commented Dec 16, 2020

@fdelapena Arrow keys are working again inside iframe. Fullscreen button is visible again as well.

@Ghabry Touch input fixed. Escape and Enter reversed.

It's pretty easy to register new buttons with the new setup. Apart from some additional CSS, you just create a new element. They get automatically registered. id, data-key and data-key-code attributes are required:

<div id="apad-shift" data-key="Shift" data-key-code="16"></div>

The key and key code may be retrieved via tools like keycode.info.

@EasyRPG EasyRPG deleted a comment from johannschopplich Dec 16, 2020
Copy link
Member

@Ghabry Ghabry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks now good to me. I will ask some testers if all there devices work. Mobile is always a bit... special

EDIT: hu? Githbu tells me I deleted a comment from you but don't know which one xD

@Ghabry
Copy link
Member

Ghabry commented Dec 16, 2020

Got one bug report: The touch controls are not shown on Android Samsung Galaxy S9+ (Android 10).

Somehow all bugs are Samsung related. They patch/mess around in Android and randomly break stuff. -_-

Will also test this evening if Touch works on my Microsoft Surface Pro 👍

@johannschopplich
Copy link
Contributor Author

@Ghabry Thanks, I've enhanced the device checks. I guess you're speaking of your Samsung Galaxy. Please test again.

Touch controls will now show only on screens with primary touch support, while they are hidden if the device supports mouse or touchpad primarily.

@Ghabry
Copy link
Member

Ghabry commented Dec 16, 2020

Nah is not my device. I don't buy expensive samsung hardware :) Waiting for the response.

Maybe a nice enhancement: Somebody else suggested: Just listen to a touch event and when one is received just show the buttons. Would work around this problem (and useful for devices that are not primary touch devices)

@johannschopplich
Copy link
Contributor Author

johannschopplich commented Dec 16, 2020

All right. Thanks.

True, 'ontouchstart' in window works, but is ugly in comparison to a sweet media query like (hover: none) and (pointer: coarse)') in my opinion. The later one is broadly supported, therefore I'm wondering why it wouldn't work.
Let's try to make it clean. If it won't work and I won't find a solution, well, then we gotta use the ugly one.

Copy link
Member

@Ghabry Ghabry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed working on Samsung so gets my approval. thanks

@johannschopplich
Copy link
Contributor Author

@Ghabry Regarding buttons disappearing in fullscreen mode: Making that working is a whole new topic. Solely the canvas element get's expanded. Will have to find a way to overlay the buttons on top of the canvas in fullscreen mode. Mabye a wrapper around the canvas. Dunno. Leaving that for another pull request. 😉

@rygos
Copy link
Contributor

rygos commented Dec 17, 2020

Yesterday I noticed another problem with iPhones, which was not the case at first.
When I press the control pad or the buttons, the player is highlighted. As if I were marking text. When I press the fullscreen button, the blue flashes only briefly. But the fullscreen button doesn't work for me anyway. But this is another story.

@johannschopplich
Copy link
Contributor Author

johannschopplich commented Dec 17, 2020

Text selection should be inhibited through additional user-select: none rules. I've added them yesterday. They work as intended on iOS for me. Could you please refresh the window some times, so we ensure the browsers uses the latest pr build? I guess it's cached and thus text selection still visible. Please retest with the latest build. If text selection still occurs, I'll look into it.

The fullscreen button is still selectable tho, good catch. Will fix that asap.

Yeah, fullscreen mode is unsupported in the Safari Mobile. It keeps on getting the new Internet Explorer... A pain for all web devs.

@Ghabry
Copy link
Member

Ghabry commented Dec 17, 2020

@rygos can you retest with the latest version?

@rygos
Copy link
Contributor

rygos commented Dec 19, 2020

Sorry for my late reply.

The Text Selection bug is fixed for me :)

@fdelapena fdelapena merged commit 8d4ef15 into EasyRPG:master Dec 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Emscripten WebAssembly/JavaScript port for web browsers Enhancement
Development

Successfully merging this pull request may close these issues.

None yet

4 participants