New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider whether to generate state updates for each composition update #543

Open
marijnh opened this Issue Jan 25, 2017 · 25 comments

Comments

Projects
None yet
8 participants
@marijnh
Copy link
Member

marijnh commented Jan 25, 2017

When composed text is only flushed to the state at the end of a composition, the UI can get out of sync with the visible document state during composition.

One solution could be to re-parse and update constantly during composition. This would be somewhat more expensive (but since this is still happening on human-interaction timescales, and wouldn't be much different from fast non-composed typing, that may not be an argument).

@owjsub

This comment has been minimized.

Copy link

owjsub commented Feb 8, 2017

Looks like draft-js needs to solve this problem as well. I found a comment that provides some context.

On Android when the word is underlined it doesn't actually appear to be in the editor - i.e. the word is not sent to my server from the onChange callback until I hit space or select an autocorrect option. facebook/draft-js#911 (comment)

@marijnh

This comment has been minimized.

Copy link
Member Author

marijnh commented Feb 24, 2017

I did some work on this but haven't finished anything yet. The main issue is that, if we want people to be able to treat the view as consistent (state and DOM match, so that coordsAtPos etc work), we need to somehow create a view on the DOM that is consistent with its updated state, but without actually mutating the DOM, since that'll reset the composition. This is tricky, especially if you add the fact that there could be other (programmatic, collab) transactions coming in during a composition, which, if we want to display them, would require the DOM to be mutated.

@owjsub

This comment has been minimized.

Copy link

owjsub commented Feb 24, 2017

I don't plan on using coordsAtPos or posAtCoords on iOS or Android. Is there anything else that would be affected if the state and DOM don't match?

@marijnh

This comment has been minimized.

Copy link
Member Author

marijnh commented Feb 26, 2017

Any decorations you added/removed won't be updated. What did you want to use the updated state for? Wasn't it some kind of UI update?

@owjsub

This comment has been minimized.

Copy link

owjsub commented Feb 28, 2017

Android w/ spellcheck or suggestions enabled causes the following issues with composition:

  1. Unable to render result of wrapInList command until composition is finished. For a composition to finish, a user must complete the current word (e.g. add space, return, or autofill) or add a special character (e.g. period or comma).
  2. I have a custom placeholder element that shows when the editor is empty. It is stuck in the visible state until the first composition finishes.
  3. Cursor jumping to prev state issue #552

In testing, I found that disabling autocapitalize, autocomplete, and autocorrect will avoid these issues per flarnie/draft-js@321a9cf, but only on certain devices. (e.g. works on a Nexus 5 w/ Chromium 56, but not on a Note 5 w/ Chromium 56)

@marijnh

This comment has been minimized.

Copy link
Member Author

marijnh commented Feb 28, 2017

Unable to render result of wrapInList command until composition is finished.

This goes for any command, right? Or are you talking about inputrule-based list wrapping?

I have a custom placeholder element that shows when the editor is empty. It is stuck in the visible state until the first composition finishes.

Yes, that's a similar problem to the coordsAtPos one -- if the view can't update its DOM representation, there'll be noticeable discrepancies.

This is a major issue that I'll be putting some more energy into at some point, but right now I have a bunch of deadlines coming up that'll have to take priority.

@owjsub

This comment has been minimized.

Copy link

owjsub commented Feb 28, 2017

I'm testing with prosemirror-commands::toggleMark and prosemirror-schema-list::wrapInList. Both have issues during a composition.

@forresto

This comment has been minimized.

Copy link
Contributor

forresto commented Feb 28, 2017

I have a custom placeholder element that shows when the editor is empty. It is stuck in the visible state until the first composition finishes.

Ditto.

@adrianheine

This comment has been minimized.

Copy link
Member

adrianheine commented Oct 17, 2017

I don't know what the current status of this is, but maybe it's a good case for an RfC?

@marijnh

This comment has been minimized.

Copy link
Member Author

marijnh commented Oct 18, 2017

Yes, it would be. If I get to this, I'd probably see what I can implement first, and then draft an RFC. If anyone is interested in picking this up, that'd probably be the way to go for them too (the main work would be finding a way to keep view descs consistent during composition—without resetting the composition by touching the wrong parts of the DOM).

@rafalp

This comment has been minimized.

Copy link

rafalp commented Apr 22, 2018

I'm interested in using prosemirror as messages editor in my forum software. Obviously editor crashing on some android devices is a no-no for internet forum, but I've started wondering, is there a way to test client for supporting required behavior before loading Prosemirror, or I have no choice but to go nuclear on all devices with android in their user agent and push fallback on them?

I would still have to implement fallback editor that's just textarea with buttons for shortcut actions for users that don't like WYSIWYM experience, so "making dumbed down fallback is not worth it" doesn't matter for me ;)

@steveccable

This comment has been minimized.

Copy link

steveccable commented Aug 2, 2018

@marijnh Any updates on this? I am encountering (what appears to be) this problem as well on Android, even on the demos on prosemirror.net (once they are edited to not contain any initial content, anyway). I've noticed other Android bugfixes that went into some of the more recent releases, but haven't seen anything which addresses the problems here or in the related closed issues.

@marijnh

This comment has been minimized.

Copy link
Member Author

marijnh commented Aug 2, 2018

@steveccable Are you sure the problem you're seeing is related to this issue? Could you describe the actual problem?

@steveccable

This comment has been minimized.

Copy link

steveccable commented Aug 2, 2018

@marijnh I've been seeing a few issues, all specifically on Android:

  1. When editing a block of text, I would apply some mark to it (let's say bold). The text I type wouldn't become bold. When I rotate the screen or lock/unlock the phone or do just about anything else to make the UI refresh, it would reappear as bold. Related, if I got the actual value out of the editor, the text would properly be bolded. That is the thing which seems most clearly this issue.

This second issue is presenting differently between my application rendering in a WebView and running the sample in Chrome on Android. Both display problems, but the specific behavior is what was happening in my app. If it is unrelated to this issue, I can open a new issue and place some sample code in there. Main reason I didn't already is because I'm unsure how tied together all the problems I'm seeing are, and the above definitely looks like part of this issue.

  1. Perhaps separate, and perhaps a whole set of problems all together, but when I start in a blank field and select inside to bring up the keyboard and start typing (specifically with the software keyboard), weird things happen. When I first click on one of the buttons to apply a mark, the cursor jumps to the beginning of the editing. Anything I type at that point would be at the start and not have the mark applied. If I toggle the mark again and type a character, that character gets bolded, and then the cursor jumps forward one character and if I continue typing, it is not bold. If I click back to the bolded character and type from there, new text is still bolded.

What I have found through debugging is that on Android when you focus into an empty editor, the selection state is AllSelection, and it stays that way until you either click inside of some text or add whitespace. This differs from on a desktop web browser, where it goes from AllSelection to TextSelection as soon as you start typing. Oddly enough, when using a hardware keyboard instead of a soft keyboard, Android is behaving the same as web.
I can also confirm that the cursor jump when a new mark is applied occurs even in the sample when running on Android and using a software keyboard. Again, hardware keyboard shows no such issues.

@steveccable

This comment has been minimized.

Copy link

steveccable commented Aug 2, 2018

An addendum: the main difference between my code and the sample is just that after toggling the mark I call editor.focus() to make sure that we are properly focused on the editor. Without that, it still has the problems with the view not reflecting changes, but the cursor no longer jumps back to the start of the editor.

@marijnh

This comment has been minimized.

Copy link
Member Author

marijnh commented Aug 3, 2018

@steveccable I don't think these are really related to this issue, sounds more like cursor wrappers aren't working as intended on Android—could you please file separate issues?

@steveccable

This comment has been minimized.

Copy link

steveccable commented Aug 3, 2018

Created #829 and #830 as requested.

@marijnh marijnh referenced this issue Aug 6, 2018

Closed

Android: Software Keyboard Jumping Behavior #829

1 of 4 tasks complete
@marijnh

This comment has been minimized.

Copy link
Member Author

marijnh commented Aug 6, 2018

Status update on this: Yes, it's probably something that should be implemented, but it's really hard (updating the DOM without disrupting the composition is tricky, and sometimes impossible).

We're working on a new version of CodeMirror where we try to get this right from the beginning, and if that works out, I hope to be able to port that solution to ProseMirror.

But for the time being, I'm very busy, and it will be a while before there will be progress on this.

@stevenfuzz

This comment has been minimized.

Copy link

stevenfuzz commented Aug 15, 2018

I'm interested in helping solve this issue. I've written a react component as part of my slashr-react library. It works fantastic, except for this. These issues make it a tough sell for production. If you give me some background and point me in the right direction I'd be more than happy to at least try to figure out a hack.

@marijnh

This comment has been minimized.

Copy link
Member Author

marijnh commented Aug 15, 2018

The problem is that if we programmatically set the selection during a composition, it'll reset the composition. This means we can't redraw the DOM around the cursor. Right now, this is solved by putting the view on hold during composition—it won't read from the DOM or update the DOM until the composition finishes.

I do not know what a solution would look like. We could read from the DOM during composition, and update the state accordingly, but not redraw the DOM. But that wouldn't really solve the issue—even if plugins can respond to state changes, if the DOM stays in its old state, the interface will still appear unresponsive.

@stevenfuzz

This comment has been minimized.

Copy link

stevenfuzz commented Aug 15, 2018

When I get a chance I'll do some debugging in an emulator. Research why Android is behaving differently and if there are any vendor specific tricks. If pm isn't really the issue, then I don't think trying to change pm logic is a viable solution. Is this chrome specific, or does it persist in webview?

@marijnh

This comment has been minimized.

Copy link
Member Author

marijnh commented Aug 15, 2018

If pm isn't really the issue

ProseMirror is absolutely the issue, I think. What Android does is use composition for all virtual keyboard input, which makes the issue extremely visible there, but the same problem exists for IME input for languages like Japanese and Korean on other platforms.

@stevenfuzz

This comment has been minimized.

Copy link

stevenfuzz commented Aug 15, 2018

Understood, so it's an architectural compatibility issue? ProseMirror is already way more usable on mobile than draft. I'll see what I can do to help, because getting this 100% working on mobile devices would be huge. I'll let you know what I find, and if I have any questions.

@jordoh

This comment has been minimized.

Copy link

jordoh commented Jan 24, 2019

In case it is useful here, the Slate editor project has made some great strides in understanding Android's composition events, culminating in an open PR that addresses many Android input issues, and a plan for an alternate approach to further address lingering issues.

@marijnh

This comment has been minimized.

Copy link
Member Author

marijnh commented Feb 19, 2019

The Slate threads are informative (and huge), but on the whole <insufferably-smug>it seems like Slate is still struggling to get to the point where pre-1.0 ProseMirror was</insufferably-smug>—the recent work builds towards a solution that freezes the editor during composition, whereas this issue is about moving beyond that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment