Custom extensions/processors

[sroach]

This is more of feature request, lack of knowledge. Is there a way to add my own asciidoctorj block macro extension to the plug-in? Like point to a directory with my jar file which packages my extension(s).

[sroach]

Never mind I just saw the wiki instructions.

[sroach]

Reopen as I read further, I see this only supports ruby extensions?

[ahus1]

Currently only Ruby extensions are supported. In #343 a similar question was asked. Back then I replied that writing an "plugin for the AsciiDoc plugin" might be a solution.

Since then some internal have changed, for other reasons I already implemented a customer class loader for AsciidoctorJ. It might now be possible to support also AsciidoctorJ extensions.

I remember AsciidoctorJ supported a service loader mechanism so extensions could self-register once the JAR is in the classpath.

Would you rather check-in the JAR in the project, or would you rather place the JAR as a custom IDE configuration in a user-local folder?

Please be aware that the extension will run in the same process as the IDE. For example too much memory consumption could destabilize the IDE.

[sroach]

Thanks for the info. Yes I am looking at the service loader mechanism.

I would like the jar loaded from a directory as we are blocked from storing binaries in our source repository

Thanks for letting me know the limitations/concerns of the plug-in running in the same process as IDE.

[ahus1]

I've prepared a pre-releae 0.31.20 that will also pick up AsciidoctorJ extensions. Actually it will pick up all JAR files in the .asciidoctor/lib. Extensions need to use the service loader mechanism of AsciidoctorJ.

I've updated the docs and moved the content from the wiki to the website: https://intellij-asciidoc-plugin.ahus1.de/docs/users-guide/features/advanced/asciidoctor-extensions.html

I decided in favor of the project-specific .asciidoctor/lib folder as users will probably want to configure this project specifically. If users don't want to check in the JAR files, they might want to check in a script that downloads the files to this folder. A .gitignore file can ensure that these downloaded files are never checked in to the repository.

Before loading the JAR files the plugin copies the file to a temporary folder. This should avoid locking the files once they are loaded (common problem on Windows).

I tested it used it with a small example from spring rest docs and it picked up the extension. I tried to look for another simple AsciidoctorJ extension but didn't find one. Most tried to include JavaScript in the header or footer which didn't work in the preview. Those worked better when rendering a full HTML page.

Please let me know how this works for you.

The pre-release of the plugin is available from GitHub releases and the IntelliJ AsciiDoc EAP repository.

[sroach]

Hi I’m trying the plug-in, it is loading but I’m getting error loading kotlin Script Engine. It uses a service loader mechanism as well.

If I use this library with asciidoctorj the library loads the extension and Kotlin script engine, but when I use in the lib directory it is not loaded.

The idea was parse a dsl in ascii doc and generate a word cloud, it’s a simple idea as I wanted to combine two things I’m liking right now, DSL and asciidoc.

[ahus1]

When loading the JARs, the plugin will only trigger AsciidoctorJ service loaders, no additional service loaders. I don't know how the Kotlin initialization works; maybe your AsciidoctorJ service loader will need to kick-off the initialization of Kotlin as well?

If this general advice doesn't help you, you might want to debug the plugin and your extension:

• check this plugin's source code in an IntelliJ IDE - see the contribute as a code docs subchapters
• run in debug mode the plugin within a sandboxed IDE
• set a breakpoint in AsciiDoc.java where it calls AsciidoctorJRuby.Factory.create(cl) to step through into the service loader logic.

Please let me know what you think of this.

[sroach]

So I’ve got the ScriptEngine loading now. Unfortunately I am facing a class cast exception. Even though it’s the same class, it is being loading by a different classloader. I tried a couple of different approached like using the parent classloader but that doesn’t work. :(

[ahus1]

The classes are loaded with the IDE providing the parent classloader context. The kind of issues you describe usually get tricky.

Can you publish a minimal example in a public (or private) Git repo?

[sroach]

Give me a little time and I will create a small project

[sroach]

I just wanted to let you know that I’ve changed the design. The extension was doing too much, so I chose to offload the work to a server. So now when the dsl block hits the blockprocessor, I’m passing that string over to the server and it returns content back. In the situation where the output/backend is Html5 I’m generating some JavaScript element and a canvas that renders an image so it’s not visible in the preview. If I have PDF as backend, the server returns an image of type png.

I may need your guidance here on a few items.

1. I am creating a temporary file and storing into the docdir, don’t think this is a good practice as I will like to clean up this file
   a. I see the image generated, the image content is as expected.

2. The issue now is in the block processor I am using the createblock() method. Passing in parent, “image”, new ArrayList(), attributes, new HashMap()
   When I log the attributes I see there is a target attribute with a value for the absolute path to the file. After that I receive a message no method error. Undefined method ‘attributes’

I think I am super close as this idea is similar to the plantuml server idea, but need to get over this small hump

For the preview I will check to see if env is idea and also return an image so the preview will work.

Just wanted to keep this thread updated.

[ahus1]

Offloading to the server is a common pattern. You mention PlantUML server. Kroki takes this several steps further for other diagram types. The maintainer of Kroki submitted a ruby based that connects the plugin to Kroki; see kroki-extension.rb

You're mentioning JavaScript plus canvas. Will the output be interactive, is that why you use JavaScript?

This plugin uses an inline conversion of AsciiDoc and misses additional HTML and JavaScript headers/footers added by plugins. I believe this could be improved given an example use case. For now, you can use the docinfo-capabilities of the plugin to add not only stylesheets, but also JavaScript to the header of the document. See the user guide about stylesheets for more information.

You could also add the necessary JavaScript as pass-through content.

Please be aware that the preview doesn't reload the page every time (unless you disable "replace without flicker" in the plugin's settings). Instead it does an inline-replace of the contents. It calls some JavaScript methods after the inline-replace that you could hook into. This is no official API yet, but again this could be mended.

As you mention PDF, did you consider returning a SVG? Vector graphics are usually nicer for printing.

Creating files in docdir is the default. Plugins should honor the outdir and imagesoutdir for a target folder; see https://github.com/asciidoctor/asciidoctor-diagram/ for examples. Users can set them in their document/build process to steer the output of images. This plugin sets them to direct the contents to a temporary folder.

Instead of creating an image in the file system, you could output a image URL that contains an base64-encoded DSL (like Kroki I mentioned above). This is then lazily evaluated once the image is shown in the preview. No temporary file would be necessary in this case. The JavaFX preview has problems with some SVGs, therefore PNG is preferred. The JCEF preview handles SVGs a lot better.

The second item you mention would be hard to analyze without the source code, sorry. Maybe see kroki-extension.rb can help you there.

I hope these replies contains some bits and pieces that help you continuing your quest.

[sroach]

Your reply put me on the right track! My first use case was to build a simple kotlin dsl for word clouds, see library http://kennycason.com/posts/2014-07-03-kumo-wordcloud.html. The dsl would allow you to write snippets in ascii doc blocks and the plug-in extension takes that code sends it to the server, server parses dsl and generates image. I can report that this is working great. Thank you for guiding me through this.

Couple of observations, when I had all of the processing inside of the extensions 55mb jar files IntelliJ would be running low on memory, 3GB allocated. Once I switched to server and limited the library dependencies it returned to normal, so a good idea to warn future developers who would want to take advantage of this.

My next step will be try the html canvas approach and see how that works, for now I would say this was a success! Cheers 🍻

[ahus1]

@sroach - your comment sounds promising. As your issue seems to be resolved, I close this issue. Please comment/re-open if you have further questions.

About "warning future developers": The docs contain the following statement:

Caveats: Extensions run in the same JVM as the IDE. Errors in extensions might consume lots of memory, CPU and might crash the IDE. Changing extensions at runtime re-instantiates the Asciidoctor and JRuby runtime, which will lead to memory leaks.

If you want to enhance it, please open a pull request. The source of the documentation is below "/doc" in this repository.

EDIT: "enhancements" might include adding links in the right places (not too many people read the docs as long as everything runs smoothly)

[sroach]

Yes the help you provided was crucial to resolving my issues. Thank You. I agree with closing the issue. Funny enough I was reading the docs and saw the caveats.