Ideas for next PF4J versions

[decebals]

Please see https://groups.google.com/forum/#!topic/pf4j/JcdrRcFaVJs

[gearlles]

Maybe it's time to release a stable version (improve unit tests, fix bugs, etc) and improve the documentation (also move it to the wiki). After this version (probably 1.0.0), the number of contributors will probably increase and the community could work on more features.

You also could create a TODO.md file to organize these tasks.

[tuxedo0801]

I agree to gearlles.

Beside that:

• Add better maven build for the plugins.

Problems I faced so far:

I have a main-application which uses plugins (of course). Both, the main application and plugins are build with maven.
I'm now in the process of creating a "distribution project" which depends in the main application and some plugins. The ide is to put the main-application and it's folder structure, start-scripts, libraries,, plus some default-plugins, which are placed into the folder structure to one big zip file.
I tried to use the maven assembly-plugin and depend on the main application as well as on the plugins.

Problem is: The plugins have packaging JAR, but produce a ZIP. The maven assembly plugin can only depend (with dependencyset) on the jar, but not on the zip. I could depend with file-dependency, but then I need to work with wildcards/placeholders for the version etc.
The clean way would be the dependencyset...

I don't know how exactly this can be solved.
Maybe create a maven-build-plugin that provides a ZIP packaging-type, so that one can depend on the zip artifact ?!
Or use JAR instead of ZIP for the plugins (need to try if this can be already achieved with a custom pluginmanager or so....

[tuxedo0801]

I'm trying to write my own pluginmanager, ... etc.

But things are blocking me:

1. Copy&paste + modifying DefaultPluginManager is not possible due to package-private class "DependencyResolver"
2. Extending DefaultPluginManager and overwriting some methods is also not that possible due to field "pluginsDirectory" which has private access and has no get-method at all.

To overcome this problem, I would need to add my custom-classes directly to PF4J. BUt I expect that this is not the way it is meant (blowing up that small library with such a "special-case" is maybe not a good idea)

So, before starting a new release, I would vote for "fine-tuning" the existing version to make it as generic as it should be.

I'll try to fix this a place a pull-request.

[tuxedo0801]

Added pull request: #66

[decebals]

Thanks for your suggestions.
@tuxedo0801 I think that you can resolve your problem using a shell script similar with run-demo.sh. The script is also available for windows (I don't know if it's still OK because I use linux). From that script you can see that I created a demo-dist folder (demo distribution). You can zip that folder and this is all you need.

[tuxedo0801]

The complete stuff is build with maven. Why should I now add a shell-script (which is platform dependend) if I could do it with maven?

Have a look at the answer to my pull-request. The remaining thing I would require is the "protected" instead of "private" access of "pluginsDirectory" field.

If extending DefaultPluginManager is not supported, one should be able to create an own PluginManager. But then DependencyResolver should be public.

[lightglitch]

@tuxedo0801 @decebals About the assembly I implemented this in https://github.com/lightglitch/pf4j/tree/assembly, just run "mvn assembly:assembly" against the Demo Project and you will get a zip with the exact same structure provided by the "run-demo".

I can create a pull request if you guys think is good to have in the project.

Edit: the "mvn clean package" also works.
Edit: Added the #67 it's easier to review.

[tuxedo0801]

@lightglitch
The idea was to NOT have an extra assembly.xml but have all the required stuff in a relative short part in pom.xml...
For me that makes sense, because it eases the way plugins need to be created. Just copy&paste the required xml-part from my "how to create a plugin" documentation into the pom and you're done. AND it's easier to depend in the build artifact, because it's a well-known standard-jar (which maven can easily handle) instead of a custom zip.

[nordoff]

1. Documentation
2. API Documentation
3. Documentation

[MTyson]

This is a really cool project but it's kind of hard to tell how alive it still is?

[decebals]

The project is alive. See the date for last commits, last release, last posts on forum. Why you think that it's not alive?
I think that from the features point of view, the project is complete. For the moment I don't have new features in the pipe. It's a community driven project. I am happy to help you if you need new features or you have some issues.

[ghost]

Hello @decebals . Nice project :-)
I have one small feature request : simple addition of inheritance on @Extension. I'm creating a PR for this little change. Their is no need to update the AnnotationProcessor, and this way, for projects where extension API is defined as abstract class (for various reasons), it is easier to force-set the inherited classes as Extensions.
Thank you

[decebals]

@elecomte-pro I merged the PR submitted by you. Can you explain in few words your use case, why do you need inherited marker on Extension annotation? (I wish that other people to understand the value of this modification).
Thanks!

[ghost]

Thank you for the PR approbation and for your quick answer :-)

I see two use cases (identified from an actual project)

• Shared extensibility Inside a Plugin => Give the ability in a plugin to specify the shared features in some abstract classes defined as ExtensionPoint and annotated with @Extension, and then make all their child classes immediately available as extensions
• Shared extensibility at API level => Define the API extension definitions with abstract classes if required (for example when the API is defined for integration in an existing complex / inheritance based application), specifiy them as @Extension in the API directly, and this way be sure that the developers will not have to apply the annotation themselves to make their plugin extensions available.

[decebals]

Thank you for your detailed explanation. I updated the SNAPSHOT to include your modification.

[tuxedo0801]

Feature-Idea:

Auto-Detect plugin-updates....

If plugin-zip (or in my case plugin-jar) changes, unload/stop the already loaded/started plugin und load/start the new version.

Scenario:

With help of pf4j, I created a kind of small "application server". With help of plugins, I can easily extend the functionality.
The server already runs on my target test machine. When I now work on a plugin update, I would like to be able to deploy the plugin to the server, which then hot-deploys/start the updated plugin.
So the only thing I have to do: write the deployment (via scp to my target host) into my plugin's pom and run this goal. Plugin will be copied and server automatically hot-deploys the new plugin.

It's not onyl helpful wheile developing, but also for end-users who want to replace a plugin by just replacing the file on the disc without restarting the application.

I'm currently evaluating of this feature can be achived by creating an own pluginmanager (which I already use for ma .jar extension plugins).

[decebals]

The hot swap it's not so trivial to implement. I will try to take a look and start work at this concept. I have the same feature in pending for other project so maybe I will do something in this direction. Thanks for explanation.

[tuxedo0801]

Indeed. Had a look at the code... It's not that easy, but should be possible.

[tuxedo0801]

Another, maybe personal wish:

stop using zip als plugin-file-container. Use JAR instead (makes life easier). And when using JAR: Don't extract it. Just use the JAR in combination with an UrlClassLoader. Would maybe make things wrt. hotswap also easier.

I'm already using JAR (but still with zip extraction): https://github.com/tuxedo0801/KnxAutomationDaemon/tree/master/KnxAutomationDaemon/src/main/java/de/root1/kad/pf4j

[decebals]

stop using zip als plugin-file-container

I don't know what to say. Your variant with jar as plugin container is OK for small plugins without dependencies. I don't want to be forced to use the maven shade plugin to pack my plugin with external libraries in a big uber-jar file. Maybe an idea is to add support for a such thing in core, in a minimalist way, and to have the possibility at runtime to specify the behavior (zip with or without extraction, jar, ...)

I'm already using JAR (but still with zip extraction)

I am happy to see that you can customize the PF4J to satisfy you expectations..

[davidsusu]

My major wishes:

• set runtime mode (and other hidden settings) on-the-fly (e. g. new DefaultPluginManager(RuntimeMode.DEVELOPMENT))
• basic support/capability for alternative plugin types (mixed with zips and each other), e. g.:
  • JAR plugin
  • single script (e. g. my-plugin.js, in the simplest case it could contain a single function which implements a functional extension point)
  • folder of scripts
• utilities for sandboxing (for example runOnPluginThread(Runnable action))

[decebals]

@davidsusu

JAR plugin

Somebody used a JarPluginManager with success. To implement a such plugin manager you must override one or two methods in DefaultPluginManager.

set runtime mode (and other hidden settings) on-the-fly (e. g. new DefaultPluginManager(RuntimeMode.DEVELOPMENT))

I told in documentation that you can set pf4j.mode property (-Dpf4j.mode=development for exmaple or via System.setProperty() method in code).

• Single script (e. g. my-plugin.js, in the simplest case it could contain a single function which implements a functional extension point)
• folder of scripts
• utilities for sandboxing (for example runOnPluginThread(Runnable action))

I don't understand these features. What they try to solve?

[davidsusu]

@decebals

Somebody used a JarPluginManager with success. To implement a such plugin manager you must override one or two methods in DefaultPluginManager.

Okay. However it would be nice to support it out-of-the-box, without extracting archives.

I don't understand these features. What they try to solve?

Some users just prefer scripts. Because of this, scripting was my original concept for plugin management (before I choose PF4J as plugin manager). Of course, not for "system plugins".

[decebals]

Okay. However it would be nice to support it out-of-the-box, without extracting archives.

I created #118. The status is work in progress. This feature will be available in the next version.

Some users just prefer scripts. Because of this, scripting was my original concept for plugin management (before I choose PF4J as plugin manager). Of course, not for "system plugins".

I am afraid that I don't understand. What kind of scripts? Can you describe a workflow on an example?

[tuxedo0801]

I am afraid that I don't understand. What kind of scripts? Can you describe a workflow on an example?

RTFM? ;-)

single script (e. g. my-plugin.js, in the simplest case it could contain a single function which implements a functional extension point)

.js = JavaScript?!

And btw: I was the person who created the JarPluginManager.

Today I'm not using pf4j any more, as it was no longer matching my requirements.

I created a special "plugin framework", which was easier (most of the parts I copied from another project I made in the past) than adapting pf4j to my needs. It's (for me) much simpler, but totally addicted to my needs.

[decebals]

@tuxedo0801

Today I'm not using pf4j any more, as it was no longer matching my requirements.

Just a curiosity. What requirements you have that can not be implemented with pf4j?

[tuxedo0801]

1. I never said that is was not possible to implement... It is possible, but it was too much/complicated to change for me. It was easier/faster to got for an own solution.

2. I really prefer JAR instead of ZIP. I recommended long time ago to add this (because my implementation is more or less a "hack"... And now there are other people asking for .JAR instead of .ZIP and "no archive extraction"... You had enough time to react on this. Nothing happened.

3. I not just want to use JAR, I want to have it "hot-deployable" and "hot-replaceable" (just remove or replace file on disk). That seemed to be too complicated to do it with pf4j. Not impossible, but too complicated.

4. Years ago, I implemented a kind of "minimalistic osgi". I was able to re-use a lot of code and had a working solution within 1-2hrs, which uses JAR instead if ZIP, don't need to extract files onto disk to use them, had a slightly cleaner interface and was hot-deployable and hot-replaceable. So why investigating more time in pf4j to adapt it to my needs (and wasting time arguing that this might be a good idea to have it integrated in pf4j), when the ideal solution just takes me 1-2 hrs?!

just my 2 cents

[decebals]

@tuxedo0801 Thanks for explanations.

[davidsusu]

@tuxedo0801

RTFM? ;-)

Me? Of course, I have read the readme and looked through the code.

.js = JavaScript?!

In this example, yes (I think it's logical, especially since Java 8). However, the point is the concept of scripting.

@decebals

I am afraid that I don't understand. What kind of scripts? Can you describe a workflow on an example?

Okay, here is a dummy example.

There is a statistician who is a bit familiar with JavaScript but have no or not much experience (and need) with Java, Maven, etc. He uses my application and want to put short script files into a folder.

I, the developer define an extension point:

public interface Formatter extends ExtensionPoint {
    
    public String format(int number, DataContext context);
    
}

Formatters can be used in reports by choosing from a list.

My user put a file hash-decoration.js which contains something like this:

/**
 * Prepend a hash character before the numerical data
 *
 * @PluginName: Hash decoration
 * @PluginId: com.someuser.someplugin
 * @PluginVersion: 1.0.0
 * @Extension: com.example.software.Formatter
 */
function (number, context) {
    return "#" + number;
}

After this, he can use it in any report by choosing "Hash decoration" from the list.

This can be accomplished by using a custom class loader which loads the .js file on-the-fly. hash-decoration.js is just a simple plugin which does not require any compilation by the user.

[tuxedo0801]

Me? Of course, I have read the readme and looked through the code.

No.

[turiot]

@decebals @tuxedo0801
Could you please tell me why hotswap is difficult as there is a existant project named 'pf4-update'; would you say that the update functionality of this one doesn't work ?

[decebals]

@turiot
A class (MyClass) is defined by its full name (it contains the package name) and a ClassLoader that loaded that class.
A class that was loaded by a ClassLoader cannot be unloaded (ClassLoader keeps a cache with all loaded classes, cache that cannot be altered from outside). A class is loaded one time by a class loader. When a class is loaded, all classes it references are loaded too.
To reload that class you should use another ClassLoader but a class MyClass that was loaded by two different class loaders is different.

The problem is presented with details in this article.
With some discipline in code (for example use in your code only the interfaces of the classes that are marked for reloading) I think that you can obtain something reloadable but it's hard to create an universal solution.

[turiot]

I know that, but my itention is to get the extension each time I want to use the plugin, in order to be dynamic (with a timed retry).
I'll try a POC as soon as possible.

[decebals]

Using pf4j-update you can add new plugins or update the existing ones. It's very useful when you want to publish plugins that will be available for all your applications instances. A kind of Eclipse Marketplace or Google Play.

[janhoy]

Closing this long running issue. @turiot can you please create a separate issue for the "hot swap" feature.