Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ace: add external keyboard support on iOS (except history) #3172

Closed
wants to merge 5 commits into from

Conversation

@etamponi
Copy link
Contributor

@etamponi etamponi commented Dec 20, 2016

This adds some basic external keyboard handling and fixes part of #37 .

What is supported:

  • Arrow keys
  • Selection and navigation using the keyboard
  • Copy and paste using keyboard

What is not supported:

  • History management using Cmd+Z and Shift+Cmd+Z
  • Touch (except for moving the caret to a certain position)

However, this provides a pretty usable experience.

How does it work?

The only event that iOS sends to textareas and other editable elements when any "special" character is pressed is selectionchange. If we pick the text in the textarea in the correct way, it is possible to use this event to detect what special character has been pressed.

It is possible to modify the code to also support history and touch, but I'll leave it for another PR :)

/* ----------- iPad mini ----------- */

/* Portrait and Landscape */
@media only screen

This comment has been minimized.

@pketh

pketh Dec 20, 2016

@etamponi I don't know if using css media queries based on device width is a good idea for this. what happens if my desktop browser window is smaller than that? Is there a way to do this with js based user agent detection + conditional stylesheet loading or class name appending?

This comment has been minimized.

@rocketinventor

rocketinventor Dec 20, 2016

I agree. It would be much better to do useragent detection via js and to add a css class to the page.


var iOS = /iPad|iPhone|iPod/.test(navigator.userAgent) && !window.MSStream;
if (iOS) {
this.textInput = new TextInput_iOS(renderer.getTextAreaContainer(), this);

This comment has been minimized.

@pketh

pketh Dec 20, 2016

@etamponi re comment above, could you get away with appending a class to the editor here? and using the css to target based on that classname rather than viewport size?

@pketh

This comment has been minimized.

Copy link

@pketh pketh commented on e8e8616 Dec 20, 2016

this looks awesome now ❤️

@nightwing
Copy link
Member

@nightwing nightwing commented Dec 20, 2016

This looks awesome!
Though i don't quite understand why this works 🤔. I have seen this approach tried a couple of times and usually ios stops sending selectionchange events once cursor gets to the end of text in the textarea and resetting selection after that doesn't help.

I only found a couple of small issues:

  • why is selectionchange listener attached to the document instead of textarea? would be good to have a comment with explaination
  • when typing fast cursor often moves down a line
@etamponi
Copy link
Contributor Author

@etamponi etamponi commented Dec 21, 2016

Thanks @nightwing :)

Ok, so this works by: 1) using a specific placeholder to detect special keys and 2) resetting the selection after every selectionchange event that has been handled. I don't know why this approach didn't work before though... 🤔 It's my first attempt, and it worked :D

Anyway, for the sake of explanation, let's make an example:

the hidden textarea contains: "\n aaaa a\n", and the initial selection is (4, 5), that is, the third a is selected. What happens when the user hits ArrowRight? that the selection is now empty, and the cursor is at position 5 (right after the third a). We have successfully detected ArrowRight! Let's reset the selection to (4, 5). Now the user hits Alt+ArrowRight. What happens? that now the cursor is at position 7, that is, right before the last a... we detected Alt+ArrowRight! And so on and so on...

Needing to attach it to the document was a surprise for me too! It's part of iOS quirks... selectionchange is only fired on document... I couldn't make it work on any other element. That's why I need to check activeElement. I'll add a comment.

Thanks for discovering the fast typing bug :) I think the reason is that I reset typing too fast. Better add a timeout... I'll let you know if this works!

@etamponi
Copy link
Contributor Author

@etamponi etamponi commented Dec 21, 2016

Ok, as I suspected, adding a timeout fixes it.

@@ -74,7 +75,13 @@ var Editor = function(renderer, session) {
this.renderer = renderer;

this.commands = new CommandManager(useragent.isMac ? "mac" : "win", defaultCommands);
this.textInput = new TextInput(renderer.getTextAreaContainer(), this);

var iOS = /iPad|iPhone|iPod/.test(navigator.userAgent) && !window.MSStream;

This comment has been minimized.

@nightwing

nightwing Jan 9, 2017
Member

why does this check for MSStream?

This comment has been minimized.

@tqc

tqc Apr 13, 2017
Contributor

That is because MS tried to break browser detection - see http://stackoverflow.com/a/9039885/101970

this.textInput = new TextInput(renderer.getTextAreaContainer(), this);

var iOS = /iPad|iPhone|iPod/.test(navigator.userAgent) && !window.MSStream;
if (iOS) {

This comment has been minimized.

@nightwing

nightwing Jan 9, 2017
Member

would be better to move navigator.userAgent test to https://github.com/ajaxorg/ace/blob/master/lib/ace/lib/useragent.js#L104 and use useragent.isIOS here.

var text = dom.createElement("textarea");
text.className = "ace_text-input ace_text-input-ios";

if (useragent.isTouchPad)

This comment has been minimized.

@nightwing

nightwing Jan 9, 2017
Member

this can be removed

.ace_text-input-ios {
position: absolute !important;
top: -100000px !important;
left: -100000px !important;

This comment has been minimized.

@nightwing

nightwing Jan 9, 2017
Member

why is this needed? this will break composition handling.

This comment has been minimized.

@tqc

tqc Apr 13, 2017
Contributor

Moving the text box off screen prevents iOS rendering selection handles, which are not affected by opacity settings.

return text.selectionStart === 0 && text.selectionEnd === text.value.length;
};
// IE8 does not support setSelectionRange
if (!text.setSelectionRange && text.createTextRange) {

This comment has been minimized.

@nightwing

nightwing Jan 9, 2017
Member

could you remove ie8 code from here?

return text.selectionStart === 0 && text.selectionEnd === text.value.length;
};
// IE8 does not support setSelectionRange
if (!text.setSelectionRange && text.createTextRange) {

This comment has been minimized.

@nightwing

nightwing Jan 9, 2017
Member

could you remove ie8 code from here?

break;
}
}
text.style.height = "100px";

This comment has been minimized.

@nightwing

nightwing Jan 9, 2017
Member

why do we need to set height? this may be better to do somewhere else

@tqc
Copy link
Contributor

@tqc tqc commented Apr 13, 2017

Not sure what best to do with it since this isn't my PR, but I have a cleaned up version of this at
https://github.com/tqc/ace/tree/ipadkb

@nightwing
Copy link
Member

@nightwing nightwing commented Apr 21, 2017

@tqc your branch looks good, could you confirm in a comment here that the changes you've made are under the terms of the project's BSD license. I can merge after that, thanks!

@tqc
Copy link
Contributor

@tqc tqc commented Apr 21, 2017

@nightwing Confirmed.

@etamponi
Copy link
Contributor Author

@etamponi etamponi commented Apr 24, 2017

Unfortunately I don't have time to keep working on this, sorry for the inconvenience. It was meant as a simple proof of concept and as someone stated in a different bug report, it might cause problems with other plugins (like vim), but I am happy that @tqc cleaned it up :)

@tqc
Copy link
Contributor

@tqc tqc commented Apr 24, 2017

@etamponi Do you have a link to the specific bug report? If you mean #3157, I don't think there is a problem - this update doesn't fully support vim on iOS, but it doesn't make anything worse either - vim needs a lot more special keys than can be detected with selectionChange.

@etamponi
Copy link
Contributor Author

@etamponi etamponi commented Apr 24, 2017

@tqc perhaps I was dreaming... :D I was pretty sure there was a bug somewhere related to my changes that asked about support for plugins like the vim one. Since I can't find it anymore, I guess it is fine 👍

@aaronvegh
Copy link

@aaronvegh aaronvegh commented May 21, 2017

I, for one, would love to see this get finished and merged... :-)

@nightwing nightwing mentioned this pull request Jun 5, 2017
@idchlife
Copy link

@idchlife idchlife commented Jun 7, 2017

Hey guys, didn't they fix this in new iOS 11? I didn't test it yet, but maybe someone knows. Arrow keys.

@aaronvegh
Copy link

@aaronvegh aaronvegh commented Jun 7, 2017

I've just checked on an iPad running iOS 11. The Kitchen Sink demo https://ace.c9.io/build/kitchen-sink.html does not work with the arrow keys. This merge is still very much needed. Thanks!

@idchlife
Copy link

@idchlife idchlife commented Jun 7, 2017

@aaronvegh thanks for checking! I didn't yet buy iPad because of this. Also, some offtopic, did you by any chance use your ipad with keyboard for coding in terminal vim as ide or something? Very interested in experience. If you even have video of this process somewhere public, would be very interested to see.

@aaronvegh
Copy link

@aaronvegh aaronvegh commented Jun 7, 2017

Here's a video you can look at: I recorded this using iOS 11's screen recorder. It's Panic's Coda running in the terminal with a keyboard. You'll see nano, and using the arrow keys to navigate history.

http://d.pr/v/lVwUlA

@idchlife
Copy link

@idchlife idchlife commented Jun 7, 2017

@aaronvegh thank you very much. If you have reddit account, I can gild it 🙇

@nightwing
Copy link
Member

@nightwing nightwing commented Jun 16, 2017

merged as #3310

@nightwing nightwing closed this Jun 16, 2017
@nightwing
Copy link
Member

@nightwing nightwing commented Jun 17, 2017

It is possible to modify the code to also support history and touch, but I'll leave it for another PR :)

@etamponi what approach did you have in mind for these? Do not leave us in mystery like Fermat did:)

@etamponi
Copy link
Contributor Author

@etamponi etamponi commented Jun 17, 2017

@tqc I found the mention about vim support, it is on #37 on which I cited this PR. I thought the comment meant that my PR broke vim support on iOS, but apparently that wasn't the case :)

@nightwing: you now put me in a hard situation... Should I do like Fermat, so that I have at least something in common with him, or should I answer your question and lose this great chance? :)

I'm writing from my phone right now, so I can't elaborate a lot, I'll be more precise tomorrow. To make history work, it should be enough to track when the content of the hidden textbox changes without any other event has been detected (keypress, copy/paste etc). This should be pretty easy to do for Cmd+Z (undo), less so for Cmd+Shift+Z (redo).

I'll try to recollect my original thoughs and elaborate more on this tomorrow :) right now I don't remember anything on how to make touch selection work too.

@etamponi
Copy link
Contributor Author

@etamponi etamponi commented Jun 19, 2017

More details on history support:

  1. the hidden textarea starts with "\n aaaa a\n"; the user types a letter, say b, and the text becomes \n aaba a\n; a keypress event is triggered, so we know from where the b comes. We handle the event and reset the textarea using KeyboardEvents in order to preserve history (if this is still feasible. I attempted at using keyboard events to write to a textarea in Chrome and I couldn't figure out how to do it). For example, we can make sure that the history is:
\n aaua a\n   - for undo
\n aaaa a\n   - current value
\n aara a\n   - for redo
  1. The user hits Cmd+Z or Cmd+Shift+Z: the text in the textarea changes, selectionchange is triggered, but no keypress event. It can't be an arrow key event since the content of the textarea has changed, so it has to be an history event. Is it Cmd+Z or Cmd+Shift+Z? It's enough to check if the value is \n aaua a\n or \n aara a\n.

This would be my plan, but again, it depends on the possibility of making programmatic changes to a textarea preserving/changing its history.

@eacabbi
Copy link

@eacabbi eacabbi commented Sep 24, 2017

guys in my experience the arrow keys on an external keyboard still break typing in sharelatex on ios...

@nightwing
Copy link
Member

@nightwing nightwing commented Sep 24, 2017

@ildodo12 sharelatex still uses 1.2.5 (check by typing ace.version in the browser console), ask them to update to 1.2.8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

8 participants