AutoFill in Apps behaving odd

[magebarf]

I’m unsure if this is a case of KeePassium doing the AutoFill methods in a different way than the iCloud Keychain, or if it simply is that Apple is not using their own API for their AutoFill. As I’m using the iCloud Keychain as the ground truth here, it may all be null and void based on the assumption that it should behave the same.

In Safari it seems the behavior is as expected, but when entering credentials inside Apps there is some odd behavior in the UI.

In a gist; When using AutoFill and selecting the entry I’m returned to the App, with the focus on one of the fields, and sometimes also outside the fields, with a blinking caret indicating the field is active. If I tap on any of the fields related to the login they stay blank, and also if I push outside, the fields now losing focus. As soon as I tap one of the user or pass fields again, after their focus has been lost, the credentials simply pop in.

In the cases where the fields are not filled this pattern can be repeated over and over without the credentials showing in the fields, invoking AutoFill multiple times. As soon as you tap outside and then in one of the fields again they “automagically” appear.

With iCloud Keychain I’ve tried this about 20 times, never seeing the same behavior. When using KeePassium it is roughly 50/50 on a run of 20 tests, with the longest streak without being able to reproduce the issue being 5 times.

Not done a proper profile how this differs between apps, but I’ve had the same behavior in multiple apps. From my memory I recall a handful of them, sadly local apps only on the Swedish App Store. Will hunt for a international one that can be used to reproduce.

I’ve captured the behavior in one App using iOS screen capture. Can’t manage to attach them through my iPhone, but I’m linking the clips on dropbox. Good to know:
Användarnamn = User name
Lösenord = Password

As the iOS Screen Capture does not indicate where taps happen the best way is to look where the caret (text entry marker) is placed, as that is the clearest indication of what field is in focus.

Also, the whole on-screen keyboard goes away whenever the user or pass field is not in focus.

The following link should go to a Dropbox folder where you can find 5 clips:
https://www.dropbox.com/sh/gr0840td25sdl0x/AAA3ZvWvcUPRWgiFRDGidqGpa?dl=0

Clips 1-3 are different variations on altering field in focus before taping outside of formed and then back in the login form.

Clip 4 is a case where returning to the app none of the fields are focused.

Clip 5 is when doing multiple AutoFills after each other without the fields getting filled until “onblur” and then focusing on the username again.

[keepassium]

Thanks, Patrik.

I have tried this with the same app on two phones (both running 12.3.1, though), but could not reproduce this. It just worked as it should (except for the double Touch ID popup, but this is different).

In general, the system completely isolates AutoFill from the host app (the one with the login form). When you press "Passwords" above the keyboard:

• The system launches the AutoFill process, giving it some info about the hosting app (domain name or URL if we are lucky, sometimes nothing at all)
• You select an entry
• AutoFill reports to the system "here are the credentials"
• The system kills the AutoFill process, then populates the login form with the credentials

So the whole form-filling process is completely out of my hands...

Can you try to reboot the phone and see if the issue persist? Just to minimize the chances of a random system glitch.

If this does not work out, it would be useful to know the iOS version, if only for statistics...

P.S. As an off-topic, enabling "Remember master keys" with a short database timeout might really save you quite a few taps... It almost hurts to see you had to type the master password every single time :)

[magebarf]

Hmm... Odd...

I’m currently out traveling for the weekend, thus changing my usage patterns away from laptop and webpages to phone apps, so I’m encountering this with quite a few apps the last few days.

I’m running an iPhone Xr on iOS 12.3.1, official build (I’m trying to stay away from iOS betas), so it shouldn’t be any difference there. Could it be related to localization and/or input language?

Just gave my phone a restart, and then launched 17TRACK and similar behavior was reproduced again.

In the same Dropbox folder as before you now have a 6th clip, this screen recording was started just after being returned from the KeePassium UI, and the fields not being populated. As suspected, as soon as I tap on the fields they directly get the credentials in the right place.

I don’t know if I have any other AutoFill enabled apps using the API to compare it to... Have you encountered any that may be useful to contrast with?

[magebarf]

I stuck some dummy entries into SAASPASS to be able to compare, and it seems that the issue is not happening there. Out of 10-15 tries, I could not produce the same behavior.

What did strike me when testing with SAASPASS is that when their “overlay” when invoking AutoFill disappears the credentials is already entered and in place when the UI “slides out”. With KeePassium it feels like the values “pop in” as soon as the UI dialog goes away, when things works.

Could it be related to UI layers, maybe stealing focus from the App?

One suspicion I had was that it could be related to the FaceID overlay, and I tried removing the biometric AppLock. I think it slightly improved the behavior, reducing the probability, but it still did not eliminate it.

[keepassium]

Indeed, odd...

Face ID overlay sounds plausible. From earlier experience, the system handles Face ID and Touch ID overlays a bit differently. This also explains why I cannot reproduce this on Touch ID devices. (I tried SAASPASS, and it worked the same as KeePassium — just fine :)

Unfortunately, I don't have a FaceID-enabled device around. In a simulator (12.2) everything works fine, but simulators are not reliable for such tests anyway.

I am out of ideas for the moment... This issue will probably have to hang around until there is more info...

[keepassium]

@magebarf, can you please check if the issue remains in beta 30?

There was a weird bug in AutoFill, that could be related to the odd behavior.

According to the documentation, the system should kill the AutoFill process after the extension is done. I relied on this statement quite heavily.

Apparently, starting from iOS 12.2, Apple decided to keep the extension process alive for a while, but forgot to document the change. This caused a bit of time-travel paradox in AutoFill (meeting your former self), and wrecked massive havoc in AutoFill state machine. (Like loading a new instance of the database without deallocating the old one, because it was not supposed to be there in the first place. Or asking for Touch ID only on 1st, 3rd, 5th... invocations.)

I'm not quite sure if this issue is related, but the scale of oddities introduced by the bug does suggest such possibility.

[magebarf]

Just updated to beta 30, and gave it a go; sadly the odd behavior is still there.

As far as I could tell it has not changed anything noticeably, even though it took a few tries to reproduce it the first time after a few more tries I’d bet it’s about the same percentage of tries that ends up with the same behavior.

I did try to contrast it a bit against iCloud Keychain, and I noticed that compared to SAASPASS iCloud is also a lot slower; SAASPASS has the details prefilled before the UI slides out while iCloud as well as KeePassium (when working correctly) pops in during or just after their UI goes away.

[keepassium]

Thanks for the update! So, back to square 1: observation...

[magebarf]

Did an attempt last weekend to reproduce in the simulator without any success.
So, if to reproduce this, I’m guessing one may need to make a more advanced app to test it against than a simple dummy form with a username and password field.

Same behavior is still present in latest beta (of KeePassium).

[keepassium]

Thanks for the update, Patrik!

I took a closer look at the test apps you mentioned. For what it's worth:

• Nacho Libre is a Xamarin app;
• 17TRACK contains quite a few Flutter frameworks.

I am not sure how thick are the UI abstraction layers added by Xamarin/Flutter, but if the system has to guess which control does what — this might explain why AutoFill is a hit-and-miss experience triggered by input focus...

Can you check if the issue happens with native apps? (MiniKeePass is a sure-fire choice :)

[magebarf]

I've attempted to trigger the same behavior using MiniKeePass for quite a while now, 5-10 times each time I've done so, but have so far not been able to reproduce the same type of behavior. So, maybe 20-30 attempts without triggering it.

I'm in line with your thinking as well, believing the nesting of UIs in and out of each other may be a big contributor to the way this behaves.

[keepassium]

Thanks for checking and confirming!

So, if the cause is indeed the hybrid/non-standard UI of the target apps, the solution is then completely on the system's side and out of my hands.

I'm quite new to GitHub etiquette... Should we keep the issue open for further details, or close it as being Apple's problem?

[magebarf]

I guess it could be good to keep around, for two reasons (that I can think of): Better visibility, and keeping track of the issue over time (checking up on it over various iOS releases).

I'm guessing a lot more users who manage to find their way here may still be a few more steps off before they start checking through closed issues, so the chances increase they will find the open issue.

A question out of curiosity, and which may be helpful in further studying the underlying reasons; What utility/method did you use to peek under the hood of the apps you listed above? Is it otool included with Xcode?

I've only tried running KeePassium in the Simulator, but when debugging on the device do you get any useful stack trace or information about the App that invoked the autofill operation? I'm guessing the extension is spawned in a isolated process, without any insight into whoever did the invocation, or is there any useful things that can be learnt by doing things this way?

My current plan of action is to start logging which apps I can see this behavior in, as I use them, and also to retest this set of apps upon newer versions of both KeePassium and iOS, which may or may not be useful at some point. :)

[keepassium]

Fair enough, let's keep it open.

What utility/method did you use to peek under the hood of the apps you listed above? Is it otool included with Xcode?

I had their .ipa packages downloaded using this method, then just looked at the files inside.

I've only tried running KeePassium in the Simulator, but when debugging on the device do you get any useful stack trace or information about the App that invoked the autofill operation? I'm guessing the extension is spawned in a isolated process, without any insight into whoever did the invocation, or is there any useful things that can be learnt by doing things this way?

In Xcode, I can see some log messages of the host app, but it does not seem useful (see below). The extension indeed runs in a separate process and does not know anything about the host, only the list of needed credentials.

Example of host app log from device

2019-09-01 22:37:49.284951+0200 MobileSafari[3560:1198140] WF: === Starting WebFilter logging for process MobileSafari
2019-09-01 22:37:49.288137+0200 MobileSafari[3560:1198140] [Bookmarks] Safari has already imported built-in bookmarks; No need to import them again
2019-09-01 22:37:49.985393+0200 MobileSafari[3560:1198140] [UserInteraction] Application did become active
2019-09-01 22:37:50.909688+0200 MobileSafari[3560:1198164] [RemotePlistController] The downloaded plist could not be loaded: Error Domain=NSCocoaErrorDomain Code=260 "The file couldn’t be opened because it doesn’t exist."
2019-09-01 22:37:52.654860+0200 MobileSafari[3560:1198164] [Warning] Bad response from daemon for setup info
2019-09-01 22:38:05.888974+0200 MobileSafari[3560:1198140] API error: <_UIKBCompatInputView: 0x113b90600; frame = (0 0; 0 0); layer = > returned 0 width, assuming UIViewNoIntrinsicMetric
2019-09-01 22:38:05.889096+0200 MobileSafari[3560:1198140] API error: <_UIKBCompatInputView: 0x113b90600; frame = (0 0; 0 0); layer = > returned 0 width, assuming UIViewNoIntrinsicMetric
2019-09-01 22:38:05.934348+0200 MobileSafari[3560:1198140] [LayoutConstraints] Unable to simultaneously satisfy constraints.
...

My current plan of action is to start logging which apps I can see this behavior in, as I use them, and also to retest this set of apps upon newer versions of both KeePassium and iOS, which may or may not be useful at some point. :)

KeePassium would still use the standard AutoFill API, so I would not expect any changes from the app updates. However, new iOS will most definitely change something (for better or otherwise :)

[A13BioniciOS6]

@magebarf
Have you ever contacted the app providers (Nacho Libre, 17TRACK, etc.)? Maybe it's an app problem?

[magebarf]

@A13BioniciOS6 if it was only those two I would probably have gone down that path.
The week leading up to this ticket was a week of travel, with a new phone to boot, so I logged in to quite a few apps, and essentially half of the apps, roughly estimated to 10 apps, all showed the same.

Out of those, I did a detailed comparison with iCloud Keychain and SAASPASS for two of them, the two you mention here, which did not show the same behavior as either of the other autofill options.

Other apps where I've seen the same behavior of having to unfocus and then tap back in the login fields includes; Untappd (2 out of 8 attempts), Fuelly (6 out of 10). More regional/Swedish apps that shows the issue: Blocket, Sunfleet (4 out of 10), Nordic Choice Hotels (8 out of 10), Västtrafik To Go (4 out of 10).

Apps working nicely includes; MiniKeePass, Instagram, Waze. Swedish/Regional ones in the same category: SJ Prio.

So, it's a hit or miss. I believe the lead on the UI stack is likely the hottest one, and I would guess that quite a few of the ones listed above uses some type of cross platform library that definitely adds to the UI "depth" (stack wise). I will probably follow the hints above in this ticket and do some kind of summary of what Libraries I can find traces from the apps I just listed.

[magebarf]

Nah, no pattern did emerge here;

Blocket - A few frameworks, nothing really UI heavy, tonnes of nib files, main binary ~12MB.
Fuelly - No frameworks, only swift dylibs, quite a lot of nibs, main binary ~75MB.
Nordic Choice Hotels - A bit framework heavy with Reactive Cocoa/Swift and nfc key library being heaviest, no cross platform framework. Almost no nibs/storyboards, main binary ~10MB
Untappd - Cordova, and a photoediting framework. Not much nibs/storyboards but a lot of "web content". Main binary ~23MB.
Sunfleet - Hell's bells; mono and a metric ton of imperial crud, including essentially triple copies of all libraries. Main binary ~60MB.

[keepassium]

Thank you for the update, @magebarf!

Hopefully, iOS 13 will make the experience smoother. (Although it does wreck some havoc in my UI :)

[magebarf]

Bearer of good news for once; iOS 13 seems to have resolved this issue!

I took the two worst culprits for this issue, Nacho Libre and Nordic Choice Hotels, and I can happily report that both of them have 10 out of 10 tries where everything works nicely! No oddities about taping elsewhere and back.

I'm a bit unsure if everything of this behavior is fixed, but statistically this looks like great improvement, will see more apps as I go on.

On Apple's behalf however I see quite a bit of odd UI behavior...
First, I noticed they remove the password button once you've selected the first entry. This may be by design, but differs from iOS 12.
However, interacting with credential forms/fields causes some visual glitching.
Uploaded two video examples to dropbox; https://www.dropbox.com/sh/ais6ua1zwfca7sq/AAA5SoS-srDwta6DECFyUhXba?dl=0
Check the password bar, on the left end the password button is jumping in and out.

The easiest way to trigger is by switching focus to a password field where a password is already filled in, and push backspace to delete. This causes the visual glitch in 99% of the case.
The second way, slightly lower reproducibility, is by simply taping back and forth between username and password fields, where it will also glitch.

Should probably be off to bug report the UI glitching to Apple...

[keepassium]

Thanks for the good news :) As for the glitch, it feels like 13.1 is just the next beta of 13.0; they are still polishing it.

[magebarf]

Now I believe the window where the risk of this happening has passed, seeing as Apple no longer signs the iOS versions where this could happen.

On the glitchiness front (still within Apple's domain) it's gradually turned better, now the only time I can make it behave erratically in the UI is when clearing an already entered password.

Closing this, hoping it will never have to be dug out of the dusty tomes of the internet.