Plugin API

[tresf]

QZ Tray could be the home of a plugin API which would allow 3rd party library support with the software to add proprietary libraries without violating the LGPL 2.1 license:

• USB hardware "black boxes" transaction library support (used in Brazil)
• Encrypted USB card reader Java library support
• Fiscally aware printers (Bematech, Tremol)
  • Tremol has an official Java API which is open source here: https://sourceforge.net/projects/zfplib/files/zfplib_java/, but it's from 2006.

http://stackoverflow.com/a/10913898/3196753

[dsanders11]

I've added basic support for UVC (USB Video Class) devices in dsanders11/tray@e0eb796 which might be useful to this enhancement. The code is not polished (it's a component of a component of a larger project, so Good Enough™ is in play) but it is functional. If there was a plugin API it would have been implemented as such, instead it was integrated directly into QZ Tray. For more color, it is used to control settings on a webcam (zoom, focus, contrast, etc) which aren't exposed in the HTML5 webcam API.

May be useful to peruse as a real-world example of what types of hooks would be useful for the plugin API. In particular I added a parent interface to DeviceIO to make it a bit more generic for supporting a class which doesn't have readData/sendData or setStreaming/isStreaming although the implementation is a bit iffy (requires some nasty casting and could collide if you opened a USB device and a UVC device with the same identifiers).

Another change which might be useful to pull back into QZ Tray is adding support for serial numbers on USB devices. In my use case I have two identical USB devices connected to the same host so I need to use serial numbers on the devices to differentiate them. Looks like there's support in the USB library that QZ Tray uses for getting the USB device's serial number so it could be implemented there. My implementation is a bit iffy, and uses Apache Commons Collections lib for MultiKey to extend the open devices mapping. Punted on how to address the optional nature of the serial number field and the complexities that can add.

BTW, if you want to play with the branch it should work with any Logitech webcam (they're UVC complaint) and probably a good number of other webcam brands.

[tresf]

@dsanders11 great feedback, thanks! I've opened up two bug reports to help nudge this along -- 1. Serial Numbers for USB support #147 -- and -- 2. UVC support #148. 👍

[mrloop]

@tresf has there been any work done on the Plugin API? Having a look at the https://github.com/qzind/tray/commits/2.1 can't see any obvious commits.

[tresf]

No, none, sorry. This won't make 2.1.

[dsanders11]

@tresf, if you're still considering this, Apache Felix seems like an interesting way to go (and a quick SO question about how to expose packages to the 'bundles'). There's also Eclipse Equinox, they're both based on OSGi.

[tresf]

@dsanders11 thanks. The only concern I have is that if the proprietary libraries aren't OSGi compliant, we'd be forced to write a wrapper for each one, which may come with proprietary API restrictions.

One example -- and a huge untapped market -- is Bematech. Bematech offers a Java API for talking to their fiscal printers but may introduce licensing issues.

Another example which is way out of left field is the Evolis Premium SDK... Take this code for example...

String ip = "11.1.24.210";
int port = 18000;
char[] data = new char[1024];
String request = "{\"jsonrpc\":\"2.0\",\"id\":\"1\",\"method\":\"CMD.SendCommand\",\"params\":{\"command\":\"Rfv\", \"device\":\"Evolis Primacy\", \"timeout\":\"5000\"}}"; 
String answer = "";

I suppose we could create these modules as our own, separate "proprietary" OSGi components and go from there. Thoughts and ideas welcome.

[dsanders11]

@tresf, yea, I think a wrapper is going to be required either way. I'm not sure how plugins would work without one. Needs to be some glue code for interacting with the QZ Tray code, and the logical place for that is in the 'plugin' so that QZ Tray doesn't need any knowledge of them.

I've currently got three different extensions to QZ Tray that it'd be preferable to turn into plugins: webcam control, pulling jobs from an HTTP queue, and renewing Let's Encrypt certs automatically. Looking at the Felix examples I can more or less imagine how they could be turned into bundles. The first would require a method for providing new commands to QZ Tray (probably a mapping of names to functions), the second needs access to some of the QZ Tray print classes. The third is the most standalone, it only needs to be able to reload QZ Tray so it's a very small hook. I could maybe try to do a quick and dirty proof of concept for this last one.

[dsanders11]

@tresf, I made a little time and created a quick and dirty proof of concept of using Apache Felix as a plugin system. Unfortunately I couldn't find any JavaDocs online for Felix (kind of annoying) so I had to poke around a bit in the dark.

Can take a look at it in commit 6ff8386, which was forked from a branch I have that has other changes, but the only changes in that commit are ones relevant to the plugin proof of concept.

There's a bunch of (hastily made) design choices I made there, but basically it loads OSGi bundles under the /plugins directory in QZ Tray's installed location. It also puts a /felix-cache directory in that location as well, that's a requirement of Felix (and OSGi), but it gets cleared on every startup, and can be easily blown away with no repercussions AFAIK. Could be changed to be in system temp instead, that's easily configurable.

Quick rundown to make the diff make more sense on initial look, I pulled out the code I'd written to automatically renew Let's Encrypt certificates (which contacts a custom API running on a server) for the proof of concept as it has very few dependencies on the rest of QZ's code. It just needs to get the tray properties so it can open the keystore. Then it runs once a day checking the cert to see if it needs to renew. For testing purposes I commented out and changed some stuff so it runs immediately and every 10 seconds so I can easily see that it's working. So the only real dependencies on QZ are access to the properties, and the ability to reload QZ (which depends on the changes I made in #275).

Here's a high-level overview of how I implemented the proof of concept:

• PluginService interface
  • This is an interface all plugins (which are just OSGi services, hence the term service used everywhere) implement. When loading the plugins it simply finds all bundles that implement that interface.
  • I didn't really flesh out the full interface, it only has an initialize which gives access to the tray properties and kicks things off
  • This is where most of the design work for a true plugin API would be. I'd imagine another very useful method on the interface would be something like Map<String, ?> getCommands() which would return a mapping of commands (like if serial functionality were a plugin, all the serial.* commands) to callables (which could be accomplished by ways mentioned here), and in PrintSocketClient the default case for the processMessage method's switch statement would check if the command is in the master mapping QZ compiled from all plugins, and if so pass it along to the plugin, and send any return value from the plugin via sendResult.
• HostService interface
  • I added this at the last minute to cover all bases on how to implement the plugin interface and prove it out.
  • There's two ways the plugin can interact with QZ's code, the first is how I started to implement it, which is to simply tell OSGi to expose full packages (such as qz.ws) to the plugin. Packages not exposed this way will lead to a NoClassDefFoundError. This is pretty coarse as it could require exposing a lot of internal workings to the plugins, making for a not well-defined API, potentially breaking plugins by making otherwise routine changes, etc. It does, however, work, and requires the least amount of designing or boiler plate. It's also often times painful as you can't wildcard packages, so you have to list every package, sub-package, etc. The second way is to create a HostService which is simply a built-in OSGi service (so no JAR required) which plugins can use to interact with QZ code.
  • For the proof of concept here I simply exposed the reload method so that the plugin could reload QZ when needed. The plugin implements both methods, since I threw it in at the last second it's quite dirty, I simply made it so if there's no qz-tray.properties found the plugin reloads QZ every 10 seconds (using the HostService method), otherwise it does what it's supposed to do (and would reload via the exposed qz.ws package).
• Activator.java and manifest.mf
  • Entry point into the plugin, registers the service (which implements PluginService)
  • Manifest can bundle dependency JARs via Bundle-ClassPath
  • Any packages not under java.* that the plugin needs to use must be listed in Import-Package. If they're not a standard system package (java.*, javax.*, etc) they need to be listed when initializing Felix.
  • HostActivator.java is the entry point for the 'host'. It's not required, but it's the logical place to register the HostService if going that route.
  • It gets a little tricky here when dealing with stuff like org.codehaus.jettison and org.bouncycastle. Exposing them package by package (since sub-packages have to be listed) is very painful and error-prone. Having the plugin simply include the JAR on its class path is better, but could create problems because then Java considers them separate classes, so if you tried to include one of those classes in the plugin API (say if the API included JSONObject), it needs to be exposed rather than packaged as a dependency. However, I think this is a good level of abstraction. I included jettison and pdfbox (an unfortunately very roundabout way to get access to Bouncycastle) because they made sense as dependencies of the plugin, which don't cross the API boundary. This has the nice benefit of they can run different versions without conflict, so a plugin can include its own version of jettison for internal use and not worry about breaking if QZ updates the jettison version. I did, however, chose to expose org.slf4j because I wanted to make sure the logging config was consistent across plugins and main QZ code.
• Building the plugin JAR
  • This is a simple JAR build command (done after building the code with ant) which I added under the file felix-build-command for easy viewing. Needs to reference manifest.mf and include any dependency JARs.

Here are my thoughts having played with this:

• Define a nice plugin API via the PluginService interface and include as many possible hooks as seem reasonable.
• Define a clear API for interacting with standard QZ code (reloading, submitting a print job, etc) via HostService interface (or rename it to something less obtuse).
• Come up with a way to expose only relevant properties from qz-tray.properties to a plugin, to prevent leaking credentials for other plugins, etc. Perhaps the plugin can provide a property key prefix, and QZ can check for conflicts before initializing plugins (and not continue if found) to prevent load order from letting you 'steal' a prefix.
• Look into OSGi security to potentially limit what plugins can do, sandbox them to only writing files in the QZ tray directory, etc.
• Move extended functionality already present in QZ (serial.*, usb.*, hid.*) to built-in services (so defined like HostService, not in external JARs) and dog food the plugin API. This also provides a trivial and clean way to turn on and off these functionalities (say via true/false flags in qz-tray.properties). While they're very useful when you need them, they also increase the attack surface and are potential security risks when you don't (which is why I suggested USB Whitelist/Blacklist #78). Being able to turn these off when all you need/want is the standard printer functionality would be a major improvement to QZ, IMHO. I'd definitely use it.
• There's potential with Felix/OSGi stuff to allow downloading plugins via HTTP rather than requiring users to place them directly. That's probably a bell and whistle not really needed, but it's something I think there's literature out there surrounding.
• I believe most of this stuff could be implemented the same way with Eclipse Equinox if preferred. They both seem to be active projects so I just threw a dart at the board and went with Apache Felix, but since they're both just layers on top of OSGi, the concepts are the same and should be applicable to both.

So there we are. Apologies if I forgot anything or started rambling, writing this rather late. It's definitely dirty and I'd do things slightly differently if I started from scratch (all mentioned in the above), but it's functional, so it definitely seems viable as a method for implementing plugins and moving extended functionality to built-in plugins.

[tresf]

Thanks! We'll take a look and address all points. Much appreciated! The proof-of-concept helps immensely because it gives us a baseline.

This may end up being a way to segment some of our existing code as well. Will have an update once the dust settles on pushing 2.1 out the door.

[dsanders11]

Is there an ETA on 2.1? End of the year or further?

[tresf]

End of the year for sure. Hopefully before then. We have a few bugs to squash first.