-
-
Notifications
You must be signed in to change notification settings - Fork 122
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable dynamic input switching #97
Comments
Wow, thank you for the detailed design and test description! I wish
everyone would send me such detailed info when they contribute.
I'll look over this in detail tonight when I've got some time and let you
know if we need any changes before merging, but this looks great to me so
far.
Thanks again - many users have been requesting this ability so really
looking forward to getting this in.
…On Thu, Oct 18, 2018, 11:47 PM utzcoz ***@***.***> wrote:
Hi, @pdsouza <https://github.com/pdsouza> , I have do some work to enable
dynamic input switching for maruos, but it is split into three
projects, so I create a issue to describe it.
Firstly, I add a property called sys.maruos.linux.enable to identify the
state whether the desktop is running and remove bluetooth keyboard/mouse
filter for Android(vendor_maruos: Enable input dynamic input switching
<utzcoz/vendor_maruos@373129a>
).
Secondly, I remove your patch to filter bluetooth keyboard/mouse in
frameworks/native to make InputFlinger works fine as normal, and disable
mouse surface in SurfaceFlinger(supports HWC1 and HWC2) which will remove
noise view when we use keyboard/mouse in Linux, when desktop is running(frameworks_native:
Enalbe dynamic input switching
<utzcoz/android_platform_frameworks_native@3625671>
).
And then, I drop all input events except touch screen/navigation/pad
events in frameworks/base when desktop is running. It will disable
bluetooth/physical keyboard/mouse input events in Android, but Linux will
read input events from /dev/input(frameworks_base: Enable dynamic input
switching
<utzcoz/android_platform_frameworks_base@cef52d1>).
For touch screen/navigation/pad events, the Linux has disabled it, so it
will only work in Android, and it will help stop the desktop when we use
Android cast to show desktop.
Lastly, I enhance the PerspectiveService's auto start for public
presentation not only HDMI to make PerspectiveService works fine when we
use Android cast(frameworks_base: Control desktop state based on public
presentation not only HDMI
<utzcoz/android_platform_frameworks_base@e5e018a>
).
I have tested the function, and below is my test steps:
1. Connect Android phone with bluetooth keyboard/mouse.
2. Connect Android phone with Android cast to show desktop, and the
desktop will auto start when cast is connected, then Android will
disable the mouse surface, and Linux will use bluetooth keyboard/mouse
correctly.
3. If I close the Android cast in physical cast device(such as
removing the battery) or in Android settings manually with touch screen,
the desktop will auto stop and bluetooth keyboard/mouse works normally in
Android.
4. For test different HWC version, just change TARGET_USES_HWC2 to true
for HWC2 and false for HWC1. It works fine in bullhead.
@pdsouza <https://github.com/pdsouza> If you think it can help community,
I will do real pulling request for it.
Thanks.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#97>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AATGcqgPgsaYK6uRS3In9F1Lb2bS6qHqks5umUszgaJpZM4Xvny2>
.
|
Hi @utzcoz, I just looked over your changes in detail and it looks good overall! There is just one issue which is with your addition of the auto-stopping behavior in PerspectiveService: frameworks_base: Control desktop state based on public presentation not only HDMI L#397. I think we can leave the auto-stop behavior out for now, because it is possible that the user wants the desktop to continue running in the background so it is easy to return to to it when the user re-connects to a display. Ideally, we need a Settings option so that users can either enable or disable auto-start and auto-stop behavior - until we have that, we should just leave it running in case there is important work still running on the desktop when the user disconnects. For example, imagine there is a document open that the user has not saved yet - we don't want to shut down until the user explicitly requests it. Other than that, this is awesome. Please go ahead and send me the pull requests for each project and we can work together to get your work merged in for the next release. Thanks! |
@pdsouza I got your concerns. I will remove auto-stop from patch. But for bluetooth/physical mouse/keyboard working in Android when disconnecting HDMI or Android Lastly, thanks for your review. |
@pdsouza If we let desktop running when we disconnect HDMI or Andoird Thanks. |
@utzcoz Oh, I see what you mean. Now that Android can use BT devices it will create noise on the desktop side. Thanks for bringing this up, I forgot about that. I think Proposal v2Ideally, we need a way to dynamically switch input events on both Android and the desktop container depending on whether the desktop is in interactive or headless mode (display available or no display available, respectively). @utzcoz's work implements dynamic input switching on Android already, but we need to figure out how to do dynamic input switching within Linux too. Otherwise, we must either shutdown or freeze the desktop to prevent noise from Here is one idea to implement dynamic input switching on the desktop:
To do this:
I can help implement the dynamic input switching within the desktop container. Then we can use both of our patches together to solve this problem. |
@pdsouza Thanks for your solution, and it looks good to me. I will use your proposal to modify my patch to let it clear the input noise aon desktop side. |
@pdsouza I have done some work to implement your proposal, and below is my implement steps. Firstly, I rename the property Then, I remove directly auto-stop from Thirdly, I rename property name in And then, I remove property changing logic from Lastly, I add real script in container to enable/disable input dynamic based on After above work, when I disconnect the Android For testing noise, I open a terminal in desktop, and change input focus on it, and then disconnect Android Thanks. |
@utzcoz Thanks! I will do a detailed review and get back to you soon! |
OK, I just read through all the patches. This looks really good. Just one edge case I think: what happens if the user starts the desktop manually (via Settings > Desktop > Dashbboard) in the background when no display is attached? In this case, it looks like input will still be enabled since the hook to disable input only happens if a display is removed (https://github.com/utzcoz/android_platform_frameworks_base/commit/be0b4f2ec204e6cd5bf8cab59c98dab0beaf3d5a#diff-85fab8f12f3a3954e15f12dedcd4b711R421). We need to check if a display is available when the user starts and disable input if no display is attached. Please correct me if I am wrong and you already handle this case. Once that's fixed, this should be good to go and you can send me the PRs! Thanks again for all the great work @utzcoz! |
@pdsouza You're correct. I will add another patch to fix it. Thanks for your detailed review. |
@pdsouza I add a patch to support dynamic input switching for desktop running without public presentation(frameworks_base: Add public presentation connecting check for dynamic input switching). So now the logic to enable/disable dynamic input is:
There is a missed occasion that starting desktop without public presentation, and connect public presentation later. With my nexus 5x and Android Thanks. |
For testing input status, I add a script into Linux, and use uname=$(/usr/bin/getent passwd 901000 | /usr/bin/cut -f1 -d:)
export DISPLAY=:0.0
export XAUTHORITY=/home/$uname/.Xauthority
ids=$(/usr/bin/xinput list --id-only)
for id in $ids;do /usr/bin/xinput list-props $id;done |
@utzcoz Looks good to me!
The |
@pdsouza I have sent all patches with four PRs for sign-off and building checking:
I will do more detailed testing for the Thanks. |
Hi @pdsouza, I use Thanks. |
@utzcoz OK thanks. I'll review the PRs soon and we can create a separate issue for |
@utzcoz I have reviewed each PR. Two of them are good and two of them need a little bit more work. Please see my review for details. I realized that these PRs touch almost every critical component of Maru except |
@pdsouza I have fixed the problem, and squashed commits to a main commit for cleaning up, which I think will make commit message clean and readable. I remove Also I test again, below is my test steps:
I use above script to use Thanks. |
@utzcoz Thanks for resolving the issues. I will run some test builds with these patches to verify since this is an important change. |
@utzcoz Just tested with my Nexus 5X and it works great! I am going to test on the Nexus 5 as well just to make sure everything is working well on older hw libs (hwcomposer especially) and with HDMI, but I expect it to be good. Edit: I tested the 5X using Chromecast and there was no issue connecting to Chromecast again even while the desktop was running in the background. So it appears that the Chromecast does not have the same issue as your cast device. We will still fix your |
@utzcoz I found two issues while testing my Nexus 5X some more.
|
@pdsouza I have fixed the problem.
|
@utzcoz Thanks for the quick response! I will build and test the new patches on the 5X again and get back to you soon. |
@utzcoz Great, looks like both issues are fixed! I will run a build for the Nexus 5 and do a quick test on that. |
@utzcoz On the Nexus 5, it looks like the native call to
This causes noise on the desktop. I tried running I tried to debug by running
I discovered that the PID namespace file does not exist in
I think this is due to this kernel patch I had to apply to fix building PID namespaces on hammerhead. If I run
I can also run I guess I will have to get PID namespaces working on hammerhead to prevent this issue. For devices with older kernels that do not support PID namespaces like hammerhead, perhaps we can set the I will try to fix the PID namespaces in the hammerhead kernel first though. I will hold merging the patches until I work on getting PID namespaces to compile in the hammerhead kernel and then test this again. Thanks for your patience on getting these patches merged in! |
@utzcoz No luck on patching the hammerhead 3.4 kernel to have a working I may still be able to really dig down and see what's wrong in the kernel at some point but I don't have the time right now. Using a workaround until that happens is probably the best solution to get your patches in ASAP. For now, can we try to drop the PID flag in |
@pdsouza I have added a new patch to drop Thanks. |
@utzcoz Tested on the Nexus 5 and it now works! We should be able to still support older devices now. Thanks for the quick patch. I noticed one more thing that may be confusing to users: when the desktop is in interactive mode (input enabled), the phone's virtual keyboard does not show up, so the user cannot type while the desktop is interactive unless they tap the keyboard icon in the system button tray at the bottom (or go to Settings > System > Languages & input) and select "Show virtual keyboard". We also need to set the virtual keyboard to disable "Show correction suggestions". We need to look into whether there is a config option that can be set to enable these settings by default - or, ideally, to enable the virtual keyboard when the desktop is interactive, and then disable it when it is not. Do you have any ideas on how to do this? I can help take a look at this too. I also noticed that one nice side-effect of your patches is that the phone's display (and presentation display) will be prevented from sleeping when the user is interacting with the desktop. It works really nicely when the phone is connected to power and Daydream is enabled. To sum it up: your patches are good! I'm just going to hold them till we get the virtual keyboard fixed. |
@utzcoz I did some digging and I found out how the Settings app is enabling the virtual keyboard even when a hard (physical) keyboard is attached. The Settings keyboard fragment calls InputMethodUtils to set the system preference Maybe we can toggle this setting "ON" when the desktop is in interactive mode? |
@pdsouza I'm sorry for late response, I will try to fix it, thanks your help for digging.
From: pdsouza
Sent: Saturday, November 10, 4:19 AM
Subject: Re: [maruos/maruos] Enable dynamic input switching (#97)
To: maruos/maruos
Cc: utzcoz, Mention
@utzcoz<https://github.com/utzcoz> I did some digging and I found out how the Settings app is enabling the virtual keyboard even when a hard (physical) keyboard is attached. The Settings<https://github.com/maruos/android_platform_packages_apps_Settings/blob/1aa74d3a9b2b42ae08f46f26c47311a1ff65785c/src/com/android/settings/inputmethod/PhysicalKeyboardFragment.java#L273> keyboard fragment calls InputMethodUtils<https://github.com/maruos/android_platform_frameworks_base/blob/4443fc32b9b03be01f3d9aa97129844ebf462e35/core/java/com/android/internal/inputmethod/InputMethodUtils.java#L1341> to set the system preference Settings.Secure.SHOW_IME_WITH_HARD_KEYBOARD.
Maybe we can toggle this setting "ON" when the desktop is in interactive mode?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#97 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ApGHRPATdvw-trTALIKY51NyhsOiTDAiks5uteNUgaJpZM4Xvny2>.
|
@pdsouza I add a new patch to force showing IME when desktop is interactive(maruos/android_platform_frameworks_base@a26f43c). Test steps:
Thanks. |
@pdsouza For this nice-side-effect, I think it is because of we just drop input events in top level( |
@utzcoz Just tested on the 5X and the IME patch works great! I will do a final test on the Nexus 5 and this should be good to merge. Thank you for patching this so quickly!
Yeah, this is definitely the right way to do input in Maru. Thanks for indirectly fixing this as well! |
@utzcoz Tested on the Nexus 5 and all is good. I just merged in all your PRs. Thank you so much for your effort and time to make dynamic input switching happen for Maru! |
@pdsouza Thanks. |
Hi, Do we have a RC or binary image that include this fix? Thanks, |
@oscarvalenzuelab Yes, apologies for no official releases for a while. We haven't made an official release since Maru 0.4 as we were transitioning from Android 6 to Android 8 over the past year and had some changes in project management, but we are back in action now. You can find our test builds for Maru 0.6 (not yet officially released) here: https://github.com/maruos/builds/releases. This particular patch will be available in the next build (0.6.3) which should be out by the end of the week. |
Yes I saw that, but no build for -flo :( |
@oscarvalenzuelab Yes, sadly flo's /system partition is too small to support Maru according to our N7 maintainer @TMartinPPC... I hope we can figure out a workaround for this at some point, but the focus is on newer devices now. |
Any plan on porting just this change? |
@pdsouza Maybe it's time to move the rootfs management to a app like Samsung's Linux on Dex. If we can do it, the system image's size is not a problem to port maruos. Also it's a good start to make maruos as a app. |
@utzcoz Yes, that's the right solution here. This has been discussed on our forum for a while now and @dianaxxyyzz is looking into possibly prototyping a simple app for installing the desktop image. I've created an issue for it for further discussion: #99. |
@dianaxxyyzz I will test it. |
@dianaxxyyzz I have fixed the |
Hi, @pdsouza , I have done some work to enable dynamic input switching for maruos, but it is split into three projects, so I create a issue to describe it.
Firstly, I add a property called
sys.maruos.linux.enable
to identify the state whether the desktop is running and remove bluetooth keyboard/mouse filter for Android(vendor_maruos: Enable input dynamic input switching).Secondly, I remove your patch to filter bluetooth keyboard/mouse in
frameworks/native
to makeInputFlinger
work fine as normal, and disable mouse surface inSurfaceFlinger
(supportsHWC1
andHWC2
) which will remove noise view when we use keyboard/mouse in Linux, when desktop is running(frameworks_native: Enalbe dynamic input switching).And then, I drop all input events except touch screen/navigation/pad events in
frameworks/base
when desktop is running. It will disable bluetooth/physical keyboard/mouse input events in Android, but Linux will read input events from/dev/input
(frameworks_base: Enable dynamic input switching). For touch screen/navigation/pad events, the Linux has disabled it, so it will only work in Android, and it will help stop the desktop when we use Androidcast
to show desktop.Lastly, I enhance the
PerspectiveService
's auto start for public presentation not onlyHDMI
to makePerspectiveService
works fine when we use Androidcast
(frameworks_base: Control desktop state based on public presentation not only HDMI).I have tested the function, and below is my test steps:
cast
to show desktop, and the desktop will auto start whencast
is connected, then Android will disable the mousesurface
, and Linux will use bluetooth keyboard/mouse correctly.cast
in physicalcast
device(such as removing the battery) or in Android settings manually with touch screen, the desktop will auto stop and bluetooth keyboard/mouse works normally in Android.HWC
version, just changeTARGET_USES_HWC2
totrue
forHWC2
andfalse
forHWC1
. It works fine inbullhead
.@pdsouza If you think it can help community, I will do real pulling request for it.
Thanks.
The text was updated successfully, but these errors were encountered: