Highlighting & indenting embedded XML (E4X)?

[nbaksalyar]

Hello,

It seems that js2-mode already has a reasonable parser of embedded XML in JavaScript - but I can't see if there's highlighting & indenting.

To be more precise, AFAIK js2-record-face function should be called for each highlighted token, and I can't see if it's called for XML tokens too (js2-XML and js2-XMLATTR) - so did I understand correctly that there's no highlighting for E4X yet?

If so, I'd try to add it - IMO it would be a very useful feature. I know that E4X was deprecated, but embedded XML is heavily used by React in form of JSX, and the syntax essentially is the same as E4X, so no changes would be required in the parser.

Or, perhaps, that's a problem on my side, and js2-mode already works great with React? :)

Please let me know.
Thanks!

[dgutov]

Following the deprecation, I was thinking of removing E4X support altogether, to make the code simpler. But React would be a surprising use case for it, yes.

so did I understand correctly that there's no highlighting for E4X yet?

Unfortunately, there's none. And if you look into the code, a plain XML snippet is parsed as one js2-XMLEND token (js2-XML is used when the snippet has an expression inside). js2-XMLATTR is supposed to be used for E4X attribute access expressions (apparently, @attr, @ns::attr, @ns::* or @ns::[expr]), not XML attribute names.

So to highlight tag names and attributes, one would have to add the relevant parsing logic somewhere around js2-get-next-xml-token.

As for indentation, probably delegate to sgml-indent-line, although there's a question how one would determine whether point is inside an XML expression (and expression's bounds) without using the AST (using it was rejected in the past on the grounds of that being slow).

[ustun]

Just wanted to chime in, and ask you to not remove E4X in the future. The current situation is fine enough for me when writing code for React.

[dgutov]

Yes, no problem. It would be nice to have the situation improved, though.

I'm not motivated enough to do it personally, but a contribution would be welcome (as long as it comes with copyright assignment, though).

[pedroteixeira]

@ustun how did you manage to get js2-mode to highlight/indent jsx? appreaciate any hints.

[ustun]

@pedroteixeira Hi Pedro, I actually didn't manage to do it too, it is just that it indents if there is no { } expressions, that is, if it is just XML.

See

If it is a single {} expression, it indents, but the second one is broken as you see.

[pedroteixeira]

http://web-mode.org seems to have support for jsx, perhaps it can be integrated wit js2-mode?

[dgutov]

perhaps it can be integrated wit js2-mode?

Probably (using mmm-mode, for example), but that'll be pretty weird, considering that web-mode itself provides support for mixed-mode languages.

[mattdeboard]

web-mode does indeed have some support for JSX, but it is not focused on JavaScript. Instead it is a massive and ambitious project focusing on many different languages, templating systems, and other assorted syntaxes. I've got nothing but respect for the maintainer, he's done a lot of good work.

I do think, however, that JSX support would be better suited as an extension or plugin for js2-mode. Is such a thing possible?

[dgutov]

Is such a thing possible?

I don't know, the current parser structure isn't really modular.

[sverrejoh]

http://facebook.github.io/jsx/

They have defined the syntax that expands the grammar from the ECMAScript standard. Should be useful if you want to expand js2-mode to support JSX.

[dgutov]

@sverrejoh If someone writes a parser, I'd consider it.

But actually it's not necessary to create a "real" parse tree for the nodes if the intention is just to add syntax highlighting, so the patch might opt to only perform highlighting inside the XML "tokens".

And as for indentation, we intend to delegate that to js-mode, so it can't use the parser anyway. So doing that would be harder.

[sverrejoh]

Sounds like a fun project to add some kind of optional support for the JSX grammar, but you're saying it's not really useful?

Why do you want to delegate the indention to js-mode? (I'm a little bit confused when it comes to what is what: js2-mode, js-mode, espresso-mode, js3-mode etc, which mode is this?)

[dgutov]

but you're saying it's not really useful?

It might be useful if you have code that will manipulate the resulting AST, do code generation, etc, but that's not what is being discussed here.

I'm a little bit confused when it comes to what is what

js-mode is the one distributed with Emacs right now. In Emacs 24.4, its indentation code had incorporated (almost?) all the important changes from js2-mode, so there's not much point in maintaining that logic in two places anymore.

[xizhao]

+1, I'd also like better support w/ JSX syntax.

[oiwn]

+1 want to use emacs instead of atom.

[XeCycle]

I am trying to implement function bind operators https://github.com/zenparsing/es-function-bind, and found E4X conflicts with this. JSX does not however; do you think it okay to drop E4X and implement JSX?

[dgutov]

do you think it okay to drop E4X and implement JSX?

I think so. Moreso if the proposed patch adds "real" support for JSX, with syntax highlighting. Maybe even indentation support?

[XeCycle]

On 09/10/15 20:57, Dmitry Gutov wrote:

I think so. Moreso if the proposed patch adds "real" support for JSX,
with syntax highlighting. Maybe even indentation support?

Looks like quite some work. I'd first focus on the bind operator as I
use that now; also I'm using React without JSX, have to say I have no
idea about which highlighting & indentation style makes most sense.

A quick and dirty way is to drop XML attributes and leave only inline
elements with interpolation. I guess this should not suprise JSX users
too much.

[dgellow]

At the moment I'm using a quick and dirty solution to format JSX.
I call it webfragment, it's based on emacs' indirect buffers and the web-mode.

Source: https://github.com/dgellow/home-bootstrapping/blob/master/emacs.d/utils/webfragment.el

[jacksonrayhamilton]

I began experimenting with JSX indentation support. It seems possible, and it's much easier if the implementors make assumptions and the users follow conventions.

In the React documentation, I only see two (and a half) styles of instantiation, single-line and multi-line:

var myDivElement = <div className="foo" />;

var App = (
  <Form>
    <FormRow>
      <FormLabel />
      <FormInput />
    </FormRow>
  </Form>
);

React.render(
  <CheckLink href="/checked.html">
    Click here!
  </CheckLink>,
  document.getElementById('example')
);

Single-line never uses parentheses, multi-line always does (except when the element is used as an argument, in which case it's followed by a comma; hence the "half"). Functionally, the parenthesis are a good style, because they're needed for return statements (which are sensitive to newlines), so it's at least a good starting point for support.

My basic algorithm goes like this:

• Keep backtracking until you encounter something like this:

  ( <div ...
  = <div ...
  return <div ...

• Once found, look ahead for a matching:

  ... </div> ;
  ... </div> )
  ... </div> ,

• If you find a match, but the current line isn't inside it, it's not JSX.
• Otherwise, in all likelihood, you're inside a JSX expression.
• Once you figure that out, use sgml-indent-line to indent the current line.

Theoretically this should also work for:

var foo = <div>
  <div></div>
</div>;

var foo = <div>
    <div></div>
  </div>;

var foo =
  <div>
    <div></div>
  </div>;

Though I'm having helluvah time getting the last 3 cases to behave consistently.

Also still not sure how to best handle complex embedded JS:

return (
  <ul>
    {this.props.results.map(function(result) {
      return <ListItemWrapper data={result}/>;
    })}
  </ul>
);

IDK, backtrack to {.*?( and find the matching bracket and switch back to JS indentation for that region, maybe.

[dgutov]

IDK, backtrack to {.*?( and find the matching bracket and switch back to JS indentation for that region, maybe.

Probably. In this case you could just take (nth 9 (syntax-ppss)) and iterate through these positions to find the enveloping brackets and parens.

But come to think of it, in this example you only need to care about the embedded snippet if point is either at the middle line (before return <List...), or at the following line. Neither of these positions should trigger the "we're in JSX!" heuristic to begin with.

[franleplant]

+1 to indentation for jsx!

[xizhao]

With the current implementation, is the indentation of XML and etc... dictated by sgml-indent-line?

[dgellow]

Awesome, thank you guys

[dgutov]

To be clear, though, there's still no highlighting. Just indentation.

[Macmee]

+1 for inline JSX / XML highlighting support

[jakoblind]

Great job with indentation of JSX! It's not 100% complete yet, I have some issues with codes like this:

<div className="delivery-methods-list">
    <h2>Velg leveringsalternativ</h2>
    {_.map(this.props.deliveryMethods, (deliveryMethod, i) => {
        return <DeliveryMethodListItem key={i} deliveryMethod={deliveryMethod} isChecked={(i === 0)} onChange={this.props.onChange} />;
    })}
    {loadingMessage}
</div>

Is any work being done on this? Is there any way I can help?

[dgutov]

@jakoblind See #292.

Is there any way I can help?

You can write a patch, of course. But if there's more than 15 lines of new code, you'll have to sign copyright assignment.

[azer]

Indentation still doesn't work for me, am I missing a configuration ?

[mgrandrath]

@azer Did you M-x js2-jsx-mode?

[azer]

@Grandrath whoah, I didn't even know about it!

[azer]

@Grandrath how do we automatically enable jsx mode for all files ?

[ustun]

@azer

for .react.js files:

(add-to-list 'auto-mode-alist '("\\.react\\.js$" . js2-jsx-mode))

or for all:

(add-to-list 'auto-mode-alist '("\\.js$" . js2-jsx-mode))

[ustun]

But note that it is not perfect by any means. Also see: #292

[azer]

@ustun I tried it before asking but there was a problem on loading it. here is my config; https://github.com/azer/emacs/blob/master/config/js2.el#L4

[azer]

actually got it working, just noticed that I need to load it separately. thanks / teşekkürler @ustun

[AndreaOrru]

Any news on highlighting? Anyone currently working on that / willing to provide hints on how to implement it?

[dgutov]

Nobody is working on it, to my knowledge. As far as hints go, maybe js2-get-next-xml-token is the place to apply the highlighting.

[jacksonrayhamilton]

Could code from sgml-mode be used or adapted for this purpose?

Ideally syntax highlighting would be defined in js-jsx-mode and propagate up to this mode.

[dgutov]

Maybe. As a separate font-lock rule, you could iterate through all XML nodes, narrow to their bounds and apply sgml-mode fontification inside.

[ghost]

web-mode works better (indentation and highlighting), my solution for the moment is adapt this spacemacs layer to my normal emacs configuration.