eww integration

[axyz]

quick web browsing inside emacs is very useful (especially with search-engines like google, stackoverflow, wikipedia, etc SPC a /).

However eww keybindings are almost non existent and quite buggy on spacemacs.

I may create my own bindings, but I think that being eww included by default on emacs they should be integrated on the spacemacs core and also they should be chose wisely to work correctly with evil and spacemacs keybindings philosophy.

as a side note, by default it seems that you can have only one eww buffer, we do not want a full featured browser and for the moment it may be ok as it is, but in the future we may think to something that works with layers or workspaces or similar.

I'm going to list an unordered list of things I've noticed and ideas about this topic:

• pressing o over links trigger ace-link-eww to quickly jump to a link, but if you are not over a link o will just insert a new line below. This do not make sense as you cannot actually edit the buffer and insert mode won't work. So I guess ace-link-eww should always work.
• any accidental buffer editing (such as o and dd) should be disabled
• o sounds good for open link, however we may try to follow vimperator keys and use f for follow link and maybe in the future F to follow on new buffer
• the mode leader , (or SPC m) is not defined here so we have a lot of keys available for common operations.
• we should have a quick way to open a new eww buffer other than using SPC a /, maybe something like SPC a w opening a default page or maybe asking for an url and if empty just going about:blank to be ready to accept o or p commands...
• d or q may be used to kill the eww buffer: q is similar to popups strengthening the idea of quick browsing and then close, however macro may be useful as well for some kind of dynamical scraping or similar so d may be a better alternative (used on vimperator to close a tab, also not being a popup q may seems misleading)

ideas for keybindings (similar to vimperator):

• H page back
• L page forward
• f follow link (using ace-link-eww)
• o open an url
• y in visual mode will copy text, in normal mode would be nice to be smart and copy an url if over a link or an image or copy a paragraph or a word otherwise (to be discussed)
• p open the url in the clipboard (or specified register)
• r reload
• b add to bookmarks
• B remove from bookmarks?

SPC m keys may be used for helm awesomeness like:

• SPC m l helm buffer with links on the page to be followed
• SPC m h helm buffer to jump to an header or maybe a for anchors?
• SPC m i helm buffer to jump to images by their title or filename
• SPC m b helm buffer with bookmarks

[Andre0991]

I like your idea. It would be nice to have eww integration, either by default or via a layer.

I think using most of Vimperator's keybindings is a good idea.

[syl20bnr]

I think using most of Vimperator's keybindings is a good idea.

+1

[axyz]

I'd like to prototype a pull request for this on coming days.

I should be able to manage to do it on a custom layer, however as there are no external packages to be installed and also no default keys being overwritten I guess it should be put on the core.

For this second option I'd need some guidance: where is the correct place to put custom keybindings on spacemacs core? Some example of well written code doing similar things?.

Alternatively I can proceed with a separate layer (something like +vim/vimeww) and then if it work well think of a way to integrate on the core.

I use vim keybindings, so I don't know what keybindings may sound good for emacs style guys. Having some suggestions at least for the basic operations would be really useful.

[fintelkai]

This is very timely for me. Just this morning, I decided to try using eww for at least some reading on the web. I'd like to point out that eww defines a set of useful modal keybindings (such as "q" for quitting the browser), which are active in spacemacs when in emacs or hybrid mode. It might be good to copy these as much as possible into whatever keybindings you decide on. There's some documentation at https://www.gnu.org/software/emacs/manual/html_mono/eww.html.

[fintelkai]

Here's a list of keybindings in the order they appear on the manual page:

• q eww-quit
• g eww-reload
• w eww-copy-page-url
• d eww-download
• l eww-back-url
• r eww-forward-url
• H eww-list-histories
• b eww-add-bookmark
• B eww-list-bookmarks
• & eww-browse-with-external-browser
• v eww-view-source
• C url-cookie-list

[fintelkai]

Here's another set of keybindings: http://oremacs.com/2014/12/30/ace-link-eww/

[TheBB]

@axyz in that case it sounds like it would fit better in the Spacemacs layer, rather than in core. It works much like any other layer.

[axyz]

@TheBB then I'll start with a layer

@fintelkai honestly I find vimperator keys more easy to remember, but as stated before I'm not used to emacs keys (except C x C c, C a and C e :P ) so if those are commonly used keys we could use them when edit style is set to emacs

[syl20bnr]

@axyz Go for vimperator bindings with evilification of eww keymaps. This way people using Vim style will get vimperator like bindings, people using Emacs style will get default eww bindings.

[tuhdo]

You should consider adding this eww-lnum package. It's like ace-jump, but is specialized for eww to jump around links quickly. Here is video demo.

[axyz]

@tuhdo Originally I was thinking to use ace-jump to maintain consistency, however I think I'm going to give a look at eww-lnum as well.

Mainly because of the additional features it has (F and selecting inputs).

I'm also encountering some issues like randomly loosing the evilified-state (especially when dealing with text inputs, or sometimes when loading a new link) That's probably a problem with some timings I can guess, however pressing o (default key for ace-link-eww) seems to reactivate the evilified state (then f will work again. Quite wierd... however I cannot exclude it may be a problem oface-link-eww` and maybe using eww-lnum could help.

[axyz]

when I go in evil-insert state and then I press esc, evil-normal-state is activated and I loose all my keybindings.

Probably it could work using simply evil-define-key, but I'm trying to use evilified state to no interfere with emacs editing style as suggested by @syl20bnr.

Is there a way to map all escapes (ESC and fg) to go back in evilified state instead of normal? or maybe I'm missing something idiomatic here?

that's the basic structure of what I'm doing right now:

(defun eww/init-eww ()
  (use-package eww
    :defer t
    :init
    (evil-leader/set-key "aw" 'eww)
    :config
    (evilified-state-evilify-map eww-mode-map
      :mode eww-mode
      :bindings
      "i" 'evil-insert ;; is this correct?
      "H" 'eww-back-url
      "L" 'eww-forward-url
      ....)))

actually going over an input in normal state allows you to write, but if you press a binded key, such as f eww-lnum or ace-eww-jump will fire... so I've mapped i to evil-insert, but I'm not sure it is the cleanest way to handle with this problem.

a possible alternative, but I'm not sure about it, may be to remove the new bindings from specific keymaps such as eww-text-map or eww-textarea-map.

I also fear the problem could be related to #1874 as well (but not going in normal mode should avoid it eventually?)

[syl20bnr]

Currently reverting to evilified state in insert state is not supported, this is also an issue with other states like lisp state etc...
I have yet to find a clean way to support this. Seems that eww requires it.

[axyz]

just to know, what are the problems that would arise if I simply map the evil normal state instead of evilified? (maybe adding a control to check if the user have the emacs style option)

[syl20bnr]

Evilified state keymap is very small to limit the number of shadowed original bindings, then we apply a set of rules automatically to rebind those shadowed keys.

If your intent is to remap everything like with evil-magit then you can use the normal state.

But I feel that Eww is the first app that would ideally require both evilified state and normal state. I mean when you are navigating you want evilified state, when you are editing you want full insert + normal state access.

Maybe renaming the evilified state navigation state and providing an easy way to switch to it when edition is finished is viable UX.

I think you cannot use motion state instead of evilified because this keymap is inherited by normal state (summoning @justbur to confirm or not :-))

[syl20bnr]

You can also create your own evil state from scratch, check how it is done with evilified.

[axyz]

assuming that I'm going to create something like vimperator state or web-motion state... won't I end up having the same problem to go back in vimperator mode from the normal or edit mode as happens with evilified?

The best thing would be to be able to use insert and normal states on text inputs, however it would also mean to have to "escape" 2 times to go back in evilified(or motion or vimperator).

This is an interesting topic, however I also think that I could remap everything as eww do not have a huge amount of keys and probably a full "vim" experience do not need to rely on default emacs bindings.

[justbur]

@axyz I haven't looked at all of the keybindings for eww, but here are two tools you can try

(evil-make-overriding-map eww-mode-map 'normal)

will make all of the bindings in eww-mode-map take priority over the default bindings in normal state. It's a quick way to elevate all of the default bindings in eww-mode-map.

Then you can selectively use

(evil-define-key 'normal eww-mode-map 
  KEY1 'DEF1
  KEY2 'DEF2
;; etc
)

to override any bindings in either eww-mode-map or evil-normal-state-map that you don't like after step one.

Basically, I would stick to normal state in this case and shadow the keybindings in evil-normal-state-map as you see appropriate using the above tools.

[axyz]

@justbur great, seems to work quite well with inputs.

just a small problem: l is set by default to go to the previous page, I want it to simply move 1 character on right as usual. How can I do that? is there a way to remove the previous binding? Or do I have to shadow it with some function to move 1 char on right?

j, k works as usual because, I guess they are not redefined on eww-mode, so probably the best things would be to somehow "undefine" l instead to try to override it or am I missing something?

[justbur]

that's step two 😄

(evil-define-key 'normal eww-mode-map "l" 'evil-forward-char)

[axyz]

thanks, I was missing 'evil-forward-char 😛

[justbur]

@axyz there's also this shortcut if you like

(evil-add-hjkl-bindings eww-mode-map 'normal)

[syl20bnr]

If there is not so much eww bindings it can work but I fear that some useful normal state key bindings will be missing while editing text, this is why I propose full normal/insert states and a special navigation state. To be more accurate about what I wrote above:

• once in editing state the user loops between normal and insert with i/o etc.. and ESC, the user typically enters it with i to go into insert state
• we add a simple way to go to navigate state at any time, i.e. we add a key binding like C-z or something which unconditionally switch you in navigate/evilified depending on how you want to implement it.

So normal state is left untouched.

[syl20bnr]

All in all the paradigm I propose is roughly the same as a terminal one with a normal/insert prompt and a key bindings to escape from the prompt.

[a13ph]

in case it wasn't mentioned yet, consider that EWW will have multi-buffer support soon ™️

[axyz]

@a13ph nice! that should make easy dealing with vimperator functions such as O to open url in new buffer (as well as F, P, etc...)

at the moment in the eww-lnum-universal (binded to F) it is possible to open a link on a new buffer as well, but it will create an eww<1> buffer (quite hard to figure out what is inside from helm-buffer...)

with a proper multibuffer support we may end up avoiding to manually hack buffer names to make it work correctly.

[axyz]

as a side note (far beyond my actual elisp and emacs internals knowledge...) it seems to me that eww is quite slow to load a new url.

javascript is not executed, so I guessed it may be a problem with http requests, but loading really plain html websites such as http://motherfuckingwebsite.com is fairly fast.

So the problem is probably on complex css rendering.

duckduckgo is an example: on a normal browser it is as fast as google (maybe a little slower) but on ess it is much much slower and this is why I've decided to default to google as a default search engine.

probably eww layout rendering may be optimized (where to open an issue?) ignoring or "polyfilling" some really slow css rules hard to implement on an emacs buffer.

[Andre0991]

where to open an issue?

I think it's supposed to be addressed on emacs-bugs.

Edit: emacs-bugs, not emacs-devel

[Andre0991]

A draft of the keybindings for the navigation state that @syl20bnr proposed:

;; keybindings are exactly the same as in vimperator unless otherwise stated
  (evil-define-key 'normal eww-mode-map
    "&" 'eww-browse-with-external-browser ;; default in eww-mode
    "q" 'eww-quit ;; different in vimperator (run macro)
    "a" 'eww-add-bookmark
    "yy" 'eww-copy-page-url
    "f" 'eww-lnum-follow
    "F" 'eww-lnum-universal ;; in vimperator open new tab
    "gu" 'eww-up-url
    "gt" 'eww-top-url
    "f" 'eww-lnum-follow
    "F" 'eww-lnum-universal
    "h" 'eww-back-url ;; H in vimperator, because h is :help, but I think lowercase is better for us
    "l" 'eww-forward-url ;; in vimperator, L is used for consistency, but again I think lower case is nicer for us
    "r" 'eww-reload
    )

Leader keys:

  (spacemacs/set-leader-keys-for-major-mode 'eww-mode
    "h"     'eww-history
    "ba"    'eww-add-bookmark ;; also "a" in normal state
    "bb"    'eww-list-bookmarks
    "s"     'eww-view-source
    "c"     'url-cookie-list)

Random note: Have you guys heard that eww will get a readability command? It's gonna be pretty usable with it.

[ahyatt]

BTW, I created #4975 to at least add eww files to .gitignore.

[jdriordan]

Did anyone look at this since 2016? Could these bindings form the basis of a web-browsing layer?(AFAICT there is currently no such thing)

[syl20bnr]

There is a pending layer for this, I plan to finish to merge it at some point :-)

[zendeavor]

Any news on the pending layer? A brief browse through the issues reveals that you are working since March on a rewrite of the layer system which stalled progress on any potential eww-layer.

[lexxish]

In the meantime, there is also https://github.com/coljamkop/eww-layer

[duianto]

This is the PR that syl20bnr was referring to (above): Add eww layer #9871
It was opened by the author of that external layer.

[nfedyashev]

@lexxish thanks for the link! It works great

[ckopsa]

Hey! I'm glad you guys are finding some use from my eww layer and I am grateful for the excitement around it. TBH, I don't have a whole lot of useful contributions on GitHub, so this really brightens the rest of my day. 😀

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Please let us know if this issue is still valid!

[jdriordan]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Please let us know if this issue is still valid!

This issue has never been more valid.

[lebensterben]

Please review #9871

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Please let us know if this issue is still valid!