ace: add external keyboard support on iOS (except history) #3172

etamponi · 2016-12-20T19:06:29Z

This adds some basic external keyboard handling and fixes part of #37 .

What is supported:

Arrow keys
Selection and navigation using the keyboard
Copy and paste using keyboard

What is not supported:

History management using Cmd+Z and Shift+Cmd+Z
Touch (except for moving the caret to a certain position)

However, this provides a pretty usable experience.

How does it work?

The only event that iOS sends to textareas and other editable elements when any "special" character is pressed is selectionchange. If we pick the text in the textarea in the correct way, it is possible to use this event to detect what special character has been pressed.

It is possible to modify the code to also support history and touch, but I'll leave it for another PR :)

pketh · 2016-12-20T19:17:01Z

lib/ace/css/editor.css

+/* ----------- iPad mini ----------- */
+
+/* Portrait and Landscape */
+@media only screen 


@etamponi I don't know if using css media queries based on device width is a good idea for this. what happens if my desktop browser window is smaller than that? Is there a way to do this with js based user agent detection + conditional stylesheet loading or class name appending?

I agree. It would be much better to do useragent detection via js and to add a css class to the page.

pketh · 2016-12-20T19:18:37Z

lib/ace/editor.js

+
+    var iOS = /iPad|iPhone|iPod/.test(navigator.userAgent) && !window.MSStream;
+    if (iOS) {
+      this.textInput = new TextInput_iOS(renderer.getTextAreaContainer(), this);


@etamponi re comment above, could you get away with appending a class to the editor here? and using the css to target based on that classname rather than viewport size?

nightwing · 2016-12-20T22:56:58Z

This looks awesome!
Though i don't quite understand why this works 🤔. I have seen this approach tried a couple of times and usually ios stops sending selectionchange events once cursor gets to the end of text in the textarea and resetting selection after that doesn't help.

I only found a couple of small issues:

why is selectionchange listener attached to the document instead of textarea? would be good to have a comment with explaination
when typing fast cursor often moves down a line

etamponi · 2016-12-21T00:04:50Z

Thanks @nightwing :)

Ok, so this works by: 1) using a specific placeholder to detect special keys and 2) resetting the selection after every selectionchange event that has been handled. I don't know why this approach didn't work before though... 🤔 It's my first attempt, and it worked :D

Anyway, for the sake of explanation, let's make an example:

the hidden textarea contains: "\n aaaa a\n", and the initial selection is (4, 5), that is, the third a is selected. What happens when the user hits ArrowRight? that the selection is now empty, and the cursor is at position 5 (right after the third a). We have successfully detected ArrowRight! Let's reset the selection to (4, 5). Now the user hits Alt+ArrowRight. What happens? that now the cursor is at position 7, that is, right before the last a... we detected Alt+ArrowRight! And so on and so on...

Needing to attach it to the document was a surprise for me too! It's part of iOS quirks... selectionchange is only fired on document... I couldn't make it work on any other element. That's why I need to check activeElement. I'll add a comment.

Thanks for discovering the fast typing bug :) I think the reason is that I reset typing too fast. Better add a timeout... I'll let you know if this works!

etamponi · 2016-12-21T00:15:44Z

Ok, as I suspected, adding a timeout fixes it.

nightwing · 2017-01-09T17:58:54Z

lib/ace/editor.js

@@ -74,7 +75,13 @@ var Editor = function(renderer, session) {
    this.renderer = renderer;

    this.commands = new CommandManager(useragent.isMac ? "mac" : "win", defaultCommands);
-    this.textInput  = new TextInput(renderer.getTextAreaContainer(), this);
+
+    var iOS = /iPad|iPhone|iPod/.test(navigator.userAgent) && !window.MSStream;


why does this check for MSStream?

That is because MS tried to break browser detection - see http://stackoverflow.com/a/9039885/101970

nightwing · 2017-01-09T18:01:20Z

lib/ace/editor.js

-    this.textInput  = new TextInput(renderer.getTextAreaContainer(), this);
+
+    var iOS = /iPad|iPhone|iPod/.test(navigator.userAgent) && !window.MSStream;
+    if (iOS) {


would be better to move navigator.userAgent test to https://github.com/ajaxorg/ace/blob/master/lib/ace/lib/useragent.js#L104 and use useragent.isIOS here.

nightwing · 2017-01-09T18:02:07Z

lib/ace/keyboard/textinput_ios.js

+    var text = dom.createElement("textarea");
+    text.className = "ace_text-input ace_text-input-ios";
+
+    if (useragent.isTouchPad)


this can be removed

nightwing · 2017-01-09T18:06:51Z

lib/ace/css/editor.css

+.ace_text-input-ios {
+  position: absolute !important;
+  top: -100000px !important;
+  left: -100000px !important;


why is this needed? this will break composition handling.

Moving the text box off screen prevents iOS rendering selection handles, which are not affected by opacity settings.

nightwing · 2017-01-09T18:07:48Z

lib/ace/keyboard/textinput_ios.js

+        return text.selectionStart === 0 && text.selectionEnd === text.value.length;
+    };
+    // IE8 does not support setSelectionRange
+    if (!text.setSelectionRange && text.createTextRange) {


could you remove ie8 code from here?

nightwing · 2017-01-09T18:07:48Z

lib/ace/keyboard/textinput_ios.js

+        return text.selectionStart === 0 && text.selectionEnd === text.value.length;
+    };
+    // IE8 does not support setSelectionRange
+    if (!text.setSelectionRange && text.createTextRange) {


could you remove ie8 code from here?

nightwing · 2017-01-09T18:09:33Z

lib/ace/keyboard/textinput_ios.js

+            break;
+        }
+      }
+      text.style.height = "100px";


why do we need to set height? this may be better to do somewhere else

tqc · 2017-04-13T02:49:10Z

Not sure what best to do with it since this isn't my PR, but I have a cleaned up version of this at
https://github.com/tqc/ace/tree/ipadkb

nightwing · 2017-04-21T20:08:31Z

@tqc your branch looks good, could you confirm in a comment here that the changes you've made are under the terms of the project's BSD license. I can merge after that, thanks!

tqc · 2017-04-21T20:36:31Z

@nightwing Confirmed.

etamponi · 2017-04-24T13:59:19Z

Unfortunately I don't have time to keep working on this, sorry for the inconvenience. It was meant as a simple proof of concept and as someone stated in a different bug report, it might cause problems with other plugins (like vim), but I am happy that @tqc cleaned it up :)

tqc · 2017-04-24T23:02:27Z

@etamponi Do you have a link to the specific bug report? If you mean #3157, I don't think there is a problem - this update doesn't fully support vim on iOS, but it doesn't make anything worse either - vim needs a lot more special keys than can be detected with selectionChange.

etamponi · 2017-04-24T23:14:23Z

@tqc perhaps I was dreaming... :D I was pretty sure there was a bug somewhere related to my changes that asked about support for plugins like the vim one. Since I can't find it anymore, I guess it is fine 👍

aaronvegh · 2017-05-21T00:05:49Z

I, for one, would love to see this get finished and merged... :-)

idchlife · 2017-06-07T19:33:01Z

Hey guys, didn't they fix this in new iOS 11? I didn't test it yet, but maybe someone knows. Arrow keys.

aaronvegh · 2017-06-07T19:41:31Z

I've just checked on an iPad running iOS 11. The Kitchen Sink demo https://ace.c9.io/build/kitchen-sink.html does not work with the arrow keys. This merge is still very much needed. Thanks!

idchlife · 2017-06-07T19:46:57Z

@aaronvegh thanks for checking! I didn't yet buy iPad because of this. Also, some offtopic, did you by any chance use your ipad with keyboard for coding in terminal vim as ide or something? Very interested in experience. If you even have video of this process somewhere public, would be very interested to see.

aaronvegh · 2017-06-07T19:56:16Z

Here's a video you can look at: I recorded this using iOS 11's screen recorder. It's Panic's Coda running in the terminal with a keyboard. You'll see nano, and using the arrow keys to navigate history.

http://d.pr/v/lVwUlA

idchlife · 2017-06-07T20:02:45Z

@aaronvegh thank you very much. If you have reddit account, I can gild it 🙇

nightwing · 2017-06-16T14:15:49Z

merged as #3310

nightwing · 2017-06-17T22:48:50Z

It is possible to modify the code to also support history and touch, but I'll leave it for another PR :)

@etamponi what approach did you have in mind for these? Do not leave us in mystery like Fermat did:)

etamponi · 2017-06-17T23:17:00Z

@tqc I found the mention about vim support, it is on #37 on which I cited this PR. I thought the comment meant that my PR broke vim support on iOS, but apparently that wasn't the case :)

@nightwing: you now put me in a hard situation... Should I do like Fermat, so that I have at least something in common with him, or should I answer your question and lose this great chance? :)

I'm writing from my phone right now, so I can't elaborate a lot, I'll be more precise tomorrow. To make history work, it should be enough to track when the content of the hidden textbox changes without any other event has been detected (keypress, copy/paste etc). This should be pretty easy to do for Cmd+Z (undo), less so for Cmd+Shift+Z (redo).

I'll try to recollect my original thoughs and elaborate more on this tomorrow :) right now I don't remember anything on how to make touch selection work too.

etamponi · 2017-06-19T09:34:09Z

More details on history support:

the hidden textarea starts with "\n aaaa a\n"; the user types a letter, say b, and the text becomes \n aaba a\n; a keypress event is triggered, so we know from where the b comes. We handle the event and reset the textarea using KeyboardEvents in order to preserve history (if this is still feasible. I attempted at using keyboard events to write to a textarea in Chrome and I couldn't figure out how to do it). For example, we can make sure that the history is:

\n aaua a\n   - for undo
\n aaaa a\n   - current value
\n aara a\n   - for redo

The user hits Cmd+Z or Cmd+Shift+Z: the text in the textarea changes, selectionchange is triggered, but no keypress event. It can't be an arrow key event since the content of the textarea has changed, so it has to be an history event. Is it Cmd+Z or Cmd+Shift+Z? It's enough to check if the value is \n aaua a\n or \n aara a\n.

This would be my plan, but again, it depends on the possibility of making programmatic changes to a textarea preserving/changing its history.

eacabbi · 2017-09-24T16:43:59Z

guys in my experience the arrow keys on an external keyboard still break typing in sharelatex on ios...

nightwing · 2017-09-24T20:57:58Z

@ildodo12 sharelatex still uses 1.2.5 (check by typing ace.version in the browser console), ask them to update to 1.2.8

ace: add external keyboard support (except history)

4e8a7f5

etamponi mentioned this pull request Dec 20, 2016

Arrow Keys and other function keys don't work on iOS (Chrome, Safari) microsoft/monaco-editor#293

Closed

pketh reviewed Dec 20, 2016

View reviewed changes

css: do not use media queries but add another class to element

e8e8616

fix bug: if typing fast, cursor moves down.

96e0991

ios: accept characters that count as two (like emojis)

9d98b83

nightwing requested changes Jan 9, 2017

View reviewed changes

disable gray tap highlight on ipad

59c214e

nightwing mentioned this pull request Jun 5, 2017

Ios keyboard #3310

Merged

nightwing closed this Jun 16, 2017

straley mentioned this pull request Jun 26, 2017

Add iOS Arrow Key Support (Like Ace) spiffcode/ghedit#22

Open

nightwing mentioned this pull request Aug 2, 2017

iOS compatibility issue on iPad pro, and CKJ input method issues #3157

Closed

exsilium mentioned this pull request Aug 7, 2017

iPad Pro basic compatibility pylonide/pylon#33

Closed

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ace: add external keyboard support on iOS (except history) #3172

ace: add external keyboard support on iOS (except history) #3172

etamponi commented Dec 20, 2016 •

edited

pketh Dec 20, 2016

rocketinventor Dec 20, 2016 •

edited

pketh Dec 20, 2016

nightwing commented Dec 20, 2016

etamponi commented Dec 21, 2016 •

edited

etamponi commented Dec 21, 2016

nightwing Jan 9, 2017

tqc Apr 13, 2017

nightwing Jan 9, 2017

nightwing Jan 9, 2017

nightwing Jan 9, 2017

tqc Apr 13, 2017

nightwing Jan 9, 2017

nightwing Jan 9, 2017

nightwing Jan 9, 2017

tqc commented Apr 13, 2017

nightwing commented Apr 21, 2017

tqc commented Apr 21, 2017

etamponi commented Apr 24, 2017

tqc commented Apr 24, 2017

etamponi commented Apr 24, 2017

aaronvegh commented May 21, 2017

idchlife commented Jun 7, 2017

aaronvegh commented Jun 7, 2017

idchlife commented Jun 7, 2017

aaronvegh commented Jun 7, 2017

idchlife commented Jun 7, 2017

nightwing commented Jun 16, 2017

nightwing commented Jun 17, 2017

etamponi commented Jun 17, 2017

etamponi commented Jun 19, 2017

eacabbi commented Sep 24, 2017

nightwing commented Sep 24, 2017

ace: add external keyboard support on iOS (except history) #3172

ace: add external keyboard support on iOS (except history) #3172

Conversation

etamponi commented Dec 20, 2016 • edited

Choose a reason for hiding this comment

rocketinventor Dec 20, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nightwing commented Dec 20, 2016

etamponi commented Dec 21, 2016 • edited

etamponi commented Dec 21, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tqc commented Apr 13, 2017

nightwing commented Apr 21, 2017

tqc commented Apr 21, 2017

etamponi commented Apr 24, 2017

tqc commented Apr 24, 2017

etamponi commented Apr 24, 2017

aaronvegh commented May 21, 2017

idchlife commented Jun 7, 2017

aaronvegh commented Jun 7, 2017

idchlife commented Jun 7, 2017

aaronvegh commented Jun 7, 2017

idchlife commented Jun 7, 2017

nightwing commented Jun 16, 2017

nightwing commented Jun 17, 2017

etamponi commented Jun 17, 2017

etamponi commented Jun 19, 2017

eacabbi commented Sep 24, 2017

nightwing commented Sep 24, 2017

etamponi commented Dec 20, 2016 •

edited

rocketinventor Dec 20, 2016 •

edited

etamponi commented Dec 21, 2016 •

edited