Make it easier to consume Prism language definitions from Node

[bebraw]

As I want to use Prism for some backend processing, I would be very interested in consuming it entirely through Node. The problem is that the entry point of the project doesn't give direct access to the language definitions. In addition the definitions seem to attach to a global. It would be great if the Node version was easier to consume.

To work around this, I set up a little wrapper. If you could provide an interface like that for Node, that would work for me.

[Golmote]

You can currently use this:

var Prism = require('./components/prism-core.js');
require('./components/prism-markup.js');
require('./components/prism-markdown.js');

console.log(Prism.highlight('_foo_', Prism.languages.markdown, 'markdown'));
// Outputs:
// <span class="token italic" ><span class="token punctuation" >_</span>foo<span class="token punctuation" >_</span></span>

The biggest flaw is that you need to handle the dependencies by yourself.

[bebraw]

Yeah. That is exactly what I wanted to avoid, hence the wrapper. I can just do a lookup against it when highlighting and that's it. 👍

[Golmote]

If I get it right, currently your wrapper is just a concatenation of all components, isn't it? If that is the case, couldn't you build a version of Prism with all languages included and require it? You could then access the definitions using Prism.languages.

[bebraw]

Yeah, that would work. I would still need to publish the result through npm given I consume this as a package dependency (highlighting plugin for a static site generator).

[Golmote]

I'm not sure I understand well what you expect us to do. Should we automatically build a full-featured version of Prism, with all languages included?
We currently don't publish automatically to npm, but it has already been suggested (#578), so this build could be published as well.

[bebraw]

Here's the problem:

> prism = require('prismjs')
...
> Object.keys(prism.languages)
[ 'extend',
  'insertBefore',
  'DFS',
  'markup',
  'css',
  'clike',
  'javascript' ]

The current distribution provides only a subset of languages through package main making it tough to consume. As you mentioned, a consumer could just require every language. That's not ideal.

It would be far better if the listing above would contain all language definitions. That would render by package obsolete (good thing).

Regarding #578, that might be a good idea. Just having an official npm package with the definitions in an easy to consume format would do the trick.

[Golmote]

Ok, so the steps described in my previous comments should do what you want. I agree that the current published package is far from ideal, as it is actually the build used for the website. I'll try to dive into #578 soon.

[bebraw]

Cool, let me know if I can be of any help. Feel free to reuse my script thinger as you want. It just concats the files, deals with the global and exposes the language definitions.

It's possible to do something more sophisticated and generate a separate entry point in addition to a CommonJS wrapper for each. In fact you can push one more step forward from that and go UMD. If you need a library for that, maybe libumd would work. That would make it really easy to consume individual definitions from global/AMD/CommonJS environments.

[Golmote]

Using UMD has been suggested too, but it would require many changes so I won't do it for now.

BTW, would this require publishing an npm package for each and every language definition, separately from the core?

[bebraw]

@Golmote Publishing a package for each and every language definition might be a little much. I would be happy with a single package that's easy to consume in various ways. Just having a nice entry point through Node would do for me personally.

[rrag]

The plugins are handled via hooks and they are called from inside highlightElement which accepts an element. When running from node I am using the highlight (text, grammar, language) { which does not trigger the hooks.

Any ideas on how to use plugins on node?

[Golmote]

Which plugins are you trying to use?

[rrag]

prismjs/plugins/line-numbers/prism-line-numbers

To better explain what I am trying to do, I have some documentation as markdown and build a static site with some custom code. I can pass a highlight function to the markdown parser, and the parser returns an HTML.

My highlight function now looks like this

var Prism = require('prismjs/components/prism-core');
require('prismjs/components/prism-jsx');

...
...

highlight: function (str, lang) {
    var grammar = lang !== undefined ? Prism.languages[lang] : Prism.languages.markup;
    return Prism.highlight(str, grammar, lang);
}

the line-number plugin does not expose a function it only adds a hook, and this function also takes an env object which has a dom element

as for
prismjs/plugins/line-highlight/prism-line-highlight

I have some ideas where markdown will send me the line range as part of the lang and I can slice that string to call the line numbers plugin. But the highlightLines method takes a pre element as an input.

[Golmote]

Like you said, both plugins are made to be used in the DOM, not directly from node.

[glenjamin]

@bebraw I notice you've marked prism-languages as deprecated, but it doesn't look like all the languages are available in the default bundle. Is there a trick to make this work?

[bebraw]

@glenjamin Bummer. Two options. We can fix this on Prism side (preferable) or I can undeprecate the package. I am not maintaining it actively as I don't need it anymore. Of course if you are interested, I could transfer the package to you and you could take over. It's bit of a hack, but the definitions are there.

[glenjamin]

I think it'd be neat if there was a prismjs/all entrypoint which included all the components - ideally via some code generation pre-publish so it will work with browserify/webpack.

I'll probably try doing something similar in my build for now.

[bebraw]

Yeah. prismjs/all would definitely solve it. Feel free to reuse any code you want from my approach in case you make a PR. There was some ordering business I had to hack manually to make it work.

[feimosi]

I've encountered the same problem. Would be great if this issue could be resolved. Currently as a workaround I'm using the global scope instead of the imported Prism instance.

[Golmote]

See #972 (comment) for an easy way to load language definitions in Node.