Skip to content

Conversation

@grorp
Copy link
Member

@grorp grorp commented Aug 7, 2024

closes #14456

This PR implements a touchscreen layout editor. Features:

editor.mp4

The UI design is inspired by Minecraft's control editor.

P.S. I have used "ChatGPT" while working on an early version of the touch control editor (but this PR doesn't include code generated by it)

To do

This PR is a Ready for Review.

Revert commit "Temp: Unbreak mainmenu security" before merging

Features that would be cool, but are not planned for this initial PR:

  • Implement dig/place buttons (Android: Add more touchscreen buttons #13229). This is already implemented, but the new SVG icons would blow up the diff too much ;)
  • Allow having multiple saved layouts, possibly per-game
  • Automatically align buttons to each other while dragging, possibly optional
  • Make the joystick customizable like a regular button
  • Allow resizing buttons

How to test

Play Minetest on Android. Use the touchscreen layout editor (accessible via the pause menu or the settings menu).

APKs for testing (signed with a debug keystore)

app-armeabi-v7a-release.zip
app-arm64-v8a-release.zip
app-x86-release.zip
app-x86_64-release.zip

The APKs are inside ZIP archives, so you have to extract the ZIP first. You have to delete any existing Minetest installation including data beforehand for installation to work.

@grorp grorp added Android Feature ✨ PRs that add or enhance a feature @ Client / Controls / Input UI/UX Waiting (on dependency) Waiting on another PR or external circumstances (not for rebases/changes requested) labels Aug 7, 2024
@grorp grorp force-pushed the touch-layout-editor branch from 5f174a8 to 2b12121 Compare August 12, 2024 16:12
@grorp grorp removed the Waiting (on dependency) Waiting on another PR or external circumstances (not for rebases/changes requested) label Aug 12, 2024
@RokeJulianLockhart

This comment was marked as duplicate.

@grorp
Copy link
Member Author

grorp commented Aug 16, 2024

is there a way to enable a snapping grid, so that buttons can be aligned?

No, but I might add one in a future PR. See the "Features that would be cool, but are not planned for this initial PR" section.

@rubenwardy
Copy link
Contributor

rubenwardy commented Aug 16, 2024

I get the following crash when dragging a button after adding it from "Add" in the editor

           A  Cmdline: net.minetest.minetest
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A  pid: 13585, tid: 13681, name: Main  >>> net.minetest.minetest <<<
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A        #02 pc 0000000000e8b01c  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (ButtonMeta::setPos(irr::core::vector2d<int>, irr::core::vector2d<unsigned int>, int)+488) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A        #03 pc 0000000000e6a950  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (GUITouchscreenLayout::updateDragState(irr::core::vector2d<unsigned int>, irr::core::vector2d<int>)+160) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A        #04 pc 0000000000e6b0dc  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (GUITouchscreenLayout::OnEvent(irr::SEvent const&)+912) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A        #05 pc 0000000000daf484  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (irr::gui::IGUIElement::OnEvent(irr::SEvent const&)+68) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A        #06 pc 0000000000e5cc4c  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (GUIModalMenu::simulateMouseEvent(irr::ETOUCH_INPUT_EVENT, bool)+388) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A        #07 pc 0000000000e5d6b8  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (GUIModalMenu::preprocessEvent(irr::SEvent const&)+1864) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A        #08 pc 0000000000b8b390  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (MainMenuManager::preprocessEvent(irr::SEvent const&)+200) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A        #09 pc 0000000000c66c38  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (MyEventReceiver::OnEvent(irr::SEvent const&)+1056) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A        #10 pc 000000000128694c  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (irr::CIrrDeviceStub::postEventFromUser(irr::SEvent const&)+72) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A        #11 pc 000000000127a2cc  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (irr::CIrrDeviceSDL::run()+2544) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A        #12 pc 0000000000b3df14  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (RenderingEngine::run()+32) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A        #13 pc 0000000000c0bfc8  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (Game::run()+848) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A        #14 pc 0000000000c23b44  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (the_game(bool*, InputHandler*, RenderingEngine*, GameStartData const&, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >&, ChatBackend&, bool*)+128) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A        #15 pc 0000000000b8742c  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (ClientLauncher::run(GameStartData&, Settings const&)+2576) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A        #16 pc 0000000001132b60  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (main+2992) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.943 14490-14490 DEBUG                   pid-14490                            A        #17 pc 0000000001199600  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (SDL_Main+120) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.944 14490-14490 DEBUG                   pid-14490                            A        #18 pc 00000000018104e0  /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/lib/arm64/libminetest.so (Java_org_libsdl_app_SDLActivity_nativeRunMain+712) (BuildId: e1349a6a1ba1d4bc3b5a740a43a6cf7345b35bc0)
2024-08-16 20:47:39.944 14490-14490 DEBUG                   pid-14490                            A        #24 pc 000000000000c500  [anon:dalvik-classes3.dex extracted in memory from /data/app/~~WFjKVNmSvS5oRum9KTSWCg==/net.minetest.minetest-ph3qyjTNK4rnV5F230Pugg==/base.apk!classes3.dex] (org.libsdl.app.SDLMain.run+0)

@rubenwardy rubenwardy added the Action / change needed Code still needs changes (PR) / more information requested (Issues) label Aug 17, 2024
@grorp grorp force-pushed the touch-layout-editor branch from 2b12121 to 22e005b Compare August 19, 2024 10:04
@grorp
Copy link
Member Author

grorp commented Aug 19, 2024

I get the following crash when dragging a button after adding it from "Add" in the editor

Cannot reproduce, but I assume it's this assertion failing:

https://github.com/minetest/minetest/blob/5aeeff0f1415c708d18090d04d0cab53bfbbec40/src/gui/touchscreenlayout.cpp#L137

Replaced it with some printing for debugging (22e005b), can you tell me what it outputs? And more importantly, does it work now? If it does, I'll just remove the assertion.

@grorp grorp removed the Action / change needed Code still needs changes (PR) / more information requested (Issues) label Aug 19, 2024
@SmallJoker
Copy link
Member

FWIW: Yesterday I tested this PR on Android 6 and could not reproduce any such crash. Works on my side.

@rubenwardy
Copy link
Contributor

rubenwardy commented Aug 19, 2024

Replaced it with some printing for debugging, can you tell me what it outputs? And more importantly, does it work now? If it does, I'll just remove the assertion.

Works now

....
WARNING[Main]: Irrlicht: GL: 8246 8250 Performance:glFlush::Submission has been flushed
WARNING[Main]: Irrlicht: GL: 8246 8250 Performance:glFlush::Submission has been flushed
WARNING[Main]: Irrlicht: GL: 8246 8250 Performance:glFlush::Submission has been flushed
ERROR[Main]: [mismatch]
ERROR[Main]: screensize = (3120,1440)
ERROR[Main]: button size = 193
ERROR[Main]: target pos = (984,732)
ERROR[Main]: actual pos = (983,732)
WARNING[Main]: Irrlicht: GL: 8246 8250 Performance:glFlush::Submission has been flushed
WARNING[Main]: Irrlicht: GL: 8246 8250 Performance:glFlush::Submission has been flushed
WARNING[Main]: Irrlicht: GL: 8246 8250 Performance:glFlush::Submission has been flushed
...

@grorp grorp force-pushed the touch-layout-editor branch from 22e005b to ece32a9 Compare August 19, 2024 13:23
@rubenwardy
Copy link
Contributor

rubenwardy commented Aug 20, 2024

something for a follow up PR but might be nice to be able to move HUD elements like the minimap and hotbar as part of this: #15021
although I suppose that is inconsistent with mod HUD

@grorp grorp added the Rebase needed The PR needs to be rebased by its author label Aug 28, 2024
@grorp grorp force-pushed the touch-layout-editor branch from ece32a9 to ff072c7 Compare August 29, 2024 12:15
@grorp grorp added Rebase needed The PR needs to be rebased by its author and removed Rebase needed The PR needs to be rebased by its author labels Aug 29, 2024
@grorp grorp force-pushed the touch-layout-editor branch from ff072c7 to ee6b6d8 Compare September 1, 2024 09:34
@grorp grorp removed the Rebase needed The PR needs to be rebased by its author label Sep 1, 2024
@grorp grorp requested a review from rubenwardy September 1, 2024 10:09
@grorp grorp force-pushed the touch-layout-editor branch from 9a1fe10 to da80a40 Compare September 1, 2024 14:51
@grorp grorp added the Rebase needed The PR needs to be rebased by its author label Sep 6, 2024
@grorp grorp force-pushed the touch-layout-editor branch from da80a40 to 0d04cb6 Compare September 6, 2024 12:39
@grorp grorp removed the Rebase needed The PR needs to be rebased by its author label Sep 6, 2024
@SmallJoker
Copy link
Member

@grorp I would've done that if I had an Android build system set up at all. Currently I am testing the ARM v7a builds from the buildbot and sign them manually.

@grorp
Copy link
Member Author

grorp commented Nov 18, 2024

Cannot test due to #15452.
I tried to fix it, but at this point it would be easier to have this PR base on an older commit.

Actually, looks like this PR doesn't even include ea4ae55 yet ...?

@grorp grorp force-pushed the touch-layout-editor branch from 01415b8 to d8ff60e Compare November 18, 2024 21:09
Copy link
Member

@SmallJoker SmallJoker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

☑️ The layout is reset to default if any Json::RuntimeError is thrown. The user does not see any error. Luanti continues to work.
☑️ Move/add/remove buttons from the Settings menu

A few minor comments regarding code quality. Works well as-is.

At some point I tested with SDL_HINT_MOUSE_TOUCH_EVENTS=1 to be able to test
the touch controls with a mouse on desktop.
That resulted in the touch controls working, but mouse events arriving twice in GUIs:
- once directly via mouse
- once via mouse -> touch (converted by SDL) -> mouse (converted by Minetest,
  regular conversion that's necessary for menus to work on touchscreens at all)
This resulted in a crash here because the code had more side effects iirc. Now
it seems fine, and having a workaround for this specific case is questionable anyway.
@SmallJoker
Copy link
Member

Checked the diff since my last review. Still LGTM. Thanks for the update. 👍

@grorp grorp merged commit 6a1d22b into luanti-org:master Nov 24, 2024
18 checks passed
@grorp grorp deleted the touch-layout-editor branch November 24, 2024 10:33
@maplemedley
Copy link
Contributor

maplemedley commented Feb 18, 2025

Just tested this out on Android, this is such a massive improvement! Tysm for implementing it, this fixes multiple gripes I've had with the mobile experience all at once, and is so good for accessability and customizability!

Just being able to remove the zoom button and add the inventory button in the controls is huge for me lol

By the way, do you have any thoughts about how #13229 could be implemented, adding arbitrary internal keybindings as buttons through your new add-buttons prompt?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Android @ Client / Controls / Input Feature ✨ PRs that add or enhance a feature One approval ✅ ◻️ Roadmap The change matches an item on the current roadmap UI/UX

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Android: Allow players to change the button layout

7 participants