Merge pull request #69 from killbill/janitor-docs

payment: add details about Janitor and payment retries

userguide/payment/includes/payment-overview.adoc

@@ -76,6 +76,33 @@ In addition to the payment state transitions, each `PaymentTransaction` has a st
 Note that the first 3 cases are normal cases but the last 2 are errors that are unrelated to the user being able to make the payment operation:
 
 * in the case of a hard plugin failure (CANCELED result code), the gateway was probably down and the payment wasn't attempted: there is no attempt to fix those.
-* in the case of a plugin timeout (or UNDEFINED result code), the operation might actually have completed; Kill Bill will run a background task to detect those cases and will query the plugin to verify if the state is actually known and when it is, it will update the *transaction status* and move the payment to its appropriate state. If the plugin cannot tell if the payment went through, the transaction will stay in an UNKNOWN state. It is advisable to check that those are rare instances and fix the data appropriately (by checking manually the status in the gateway for example).
+* in the case of a plugin timeout (or UNDEFINED result code), the operation might actually have completed; Kill Bill will run a background task to detect those cases and will query the plugin to verify if the state is actually known and when it is, it will update the *transaction status* and move the payment to its appropriate state. If the plugin cannot tell if the payment went through, the transaction will stay in an UNKNOWN state. It is advisable to check that those are rare instances and fix the data appropriately (by checking manually the status in the gateway for example). See the section Janitor below for more details.
 
 If a payment is aborted by a control plugin, a payment attempt row is recorded with a state of ABORTED (no payment nor payment transaction is created). Kill Bill will return a 422 HTTP code.
+
+=== Janitor
+
+PENDING and UNKNOWN transaction states should be temporary: the payment plugin needs to eventually tell Kill Bill what the final state of these transactions should be. This can be done explicitly by the plugin by calling the `notifyPendingTransactionOfStateChanged` PaymentApi (for instance, when receiving a webhook event from the gateway) or implicitly by the Janitor.
+
+The Janitor refers to a set of internal Kill Bill features to help ensure transactions end up in a terminal state: these are the Janitor notification queue and the on-the-fly Janitor. They both work by calling the plugin (via `getPaymentInfo` PaymentPluginApi) and updating the state owned by Kill Bill if it has changed. It is imperative payment plugin developers take great care when implementing this API for this mechanism to work (see our Payment Plugin developer documentation for more details).
+
+==== Janitor notification queue
+
+A Janitor notification entry is inserted whenever a payment results in a non terminal state (whether the payment was initiated internally by Kill Bill or externally via our payment APIs). Kill Bill will then periodically poll the plugin to attempt to fix the state.
+
+The polling delay is configured by `org.killbill.payment.janitor.unknown.retries` for UNKNOWN transactions (default `5m,1h,1d,1d,1d,1d,1d`) and `org.killbill.payment.janitor.pending.retries` for PENDING transactions (default `1h, 1d`). Companies relying on PENDING for offline payment methods (ACH, Boleto, etc.) should increase the number of attempts, as such payments can be slow (they can take multiple days to execute).
+
+==== On-the-fly Janitor
+
+The on-the-fly Janitor is invoked:
+
+* Whenever a payment (or set of payments) is retrieved via API and the flag `withPluginInfo` is set. In this case, a GET operation might result in data being written to disk (but it's not really new state, it's just existing state being updated that was incomplete).
+* Whenever an operation is performed on an existing payment (either initiated internally by Kill Bill or externally via our payment APIs): the Janitor is invoked first to get the latest state to ensure the internal state machine prevents disallowed transitions in case the state couldn't be fixed (or if it's already in a final state). For example, if an authorization state is currently UNKNOWN and you attempt to capture it, the Janitor will ask the plugin first to fix the state and either let the capture go through (if the authorization is fixed as SUCCESS) or disallow it (if the authorization couldn't be fixed or was unsuccessful).
+
+Note however that the `notifyPendingPaymentOfStateChanged` operation will not invoke the Janitor (as this API is often called from payment plugins, we don't want the state to be fixed twice).
+
+==== Janitor and Control Plugins
+
+When the Janitor fixes a payment, it will re-run the Payment Control state machine (i.e. go through Payment Control plugins) if needed. So for instance, invoice payments that were created as part of a RECURRING invoicing may end up being fixed automatically by the Janitor, and the balance associated with the invoice would then be correctly reflected.
+
+Any invoice payment triggered by API and fixed by the Janitor would not be retried, in order to leave full control to the caller to decide what to do and whether to retry. Invoice payments triggered automatically by the system and fixed by the Janitor would however be retried as configured.

userguide/tutorials/payment_plugin.adoc

@@ -43,70 +43,19 @@ When building PaymentTransactionInfoPlugin objects, it is very important to corr
 * if the gateway wasn't contacted (DNS error, SSL handshake error, socket connect timeout, etc.), the plugin should return CANCELED
 * for all other cases (socket read timeout, 500 returned, etc.), the plugin should return UNDEFINED
 
-You should try to avoid UNDEFINED as much as possible, because it is the only case where Kill Bill cannot retry payments (since the payment may or may not have happened). Kill Bill will also attempt to fix these states by polling the plugin via getPaymentInfo: if the plugin subsequently returns PROCESSED for example (maybe by querying the gateway directly), the internal payment state (as well as invoices balance, etc.) will be reflected.
+You should try to avoid UNDEFINED as much as possible, because it is the only case where Kill Bill cannot retry payments (since the payment may or may not have happened).
 
+==== Janitor integration
 
-=== Plugin Activation
-
-When a payment operation occurs, Kil Bill will chose which payment plugin to go to based on the `paymentMethodId` specified. The `paymentMethodId` will map to an specific payment method, which in turns contains the name of the plugin to chose for the payment.
-
-== Payment plugins frameworks
-
-To help with writing payment plugins, we developed https://github.com/killbill/killbill-plugin-framework-java[Java] and https://github.com/killbill/killbill-plugin-framework-ruby[Ruby] frameworks which ensure that the plugin will respect the previous mentioned conditions. If you are developing your own plugin, we strongly encourage you to use these libraries. The easiest is to copy an existing plugin and modify it for your gateway. The https://github.com/killbill/killbill-adyen-plugin[Adyen] plugin (Java) and https://github.com/killbill/killbill-cybersource-plugin[CyberSource] (Ruby) plugins are good starting points.
-
-=== ActiveMerchant plugin generation
-
-https://github.com/activemerchant/active_merchant[ActiveMerchant] is a Ruby gem supporting dozens of gateways. For convenience, we provide a tool to generate a plugin based on an ActiveMerchant connector (if your gateway is not supported by ActiveMerchant yet, it is almost always easier and faster to first write the ActiveMerchant connector and generate the plugin code, rather than implementing your plugin from scratch).
-
-First, clone the https://github.com/killbill/killbill-plugin-framework-ruby[killbill-plugin-framework-ruby] repository and run:
-
-[source,ruby]
-----
-./script/generate active_merchant gateway_name /path/to/dir
-----
-
-Replace gateway_name with the snake case of your ActiveMerchant gateway (e.g. yandex, stripe, paypal_express, etc.).
+Kill Bill will attempt to fix PENDING and UNKNOWN states by polling the plugin via `getPaymentInfo`: if the plugin subsequently returns PROCESSED for example (maybe by querying the gateway directly), the internal payment state (as well as invoice balance, etc.) will be reflected.
 
-This will generate a tree of files ready to be plugged into Kill Bill. Most of the work will consist of filling in the blanks in api.rb (payment plugin API for ActiveMerchant gateways) and application.rb (sinatra application for ActiveMerchant integrations, i.e. Offsite Payments). For example, make sure the right constructor parameters are passed in the initialize method. Check the https://github.com/killbill/killbill-stripe-plugin/blob/master/lib/stripe/api.rb[Stripe plugin] for an example.
+The Janitor will match internal transactions against plugin transactions via the transaction id: make sure `getKbTransactionPaymentId` is correctly implemented.
 
-The generator comes with a series a basic unit and integration tests (the latter requires gateway credentials in your local YAML configuration file). To run them:
-
-[source,ruby]
-----
-rm -f Gemfile.lock Jarfile.lock .jbundler/classpath.rb
-bundle install
-# Run gem install jbundler if needed
-jbundle install
-bundle exec rake
-bundle exec rake test:remote:spec
-----
-
-Not all tests will work with your gateway (maybe some features are not supported): feel free to remove them. But make sure some basic calls are working noneless (e.g. purchase), as it will make debugging in Kill Bill much harder otherwise.
-
-=== Building Ruby plugins
-
-Unlike Java plugins, which are deployed as jars, Ruby plugins need a specific directory structure on the filesystem to run (containing all of the gems dependencies). Fortunately, we provide scripts to generate self-contained artifacts: all you need to do is untar (or unzip) these.
-
-==== Prerequisites
-
-Ruby 2.1+ or JRuby 1.7.20+ is recommended. If you don't have a Ruby installation yet, use https://rvm.io/rvm/install[RVM]:
-
-[source,bash]
-----
-gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
-\curl -sSL https://get.rvm.io | bash -s stable --ruby
-----
-
-After following the post-installation instructions, you should have access to the `ruby` and `gem` executables.
+=== Plugin Activation
 
-Install the following gems:
+When a payment operation occurs, Kil Bill will chose which payment plugin to go to based on the `paymentMethodId` specified. The `paymentMethodId` will map to an specific payment method, which in turns contains the name of the plugin to chose for the payment.
 
-[source,bash]
-----
-gem install bundler
-gem install jbundler
-----
+== Payment plugins framework
 
-==== Build
+To help with writing payment plugins, we developed a https://github.com/killbill/killbill-plugin-framework-java[Java] framework which ensures that the plugin will respect the previous mentioned conditions. If you are developing your own plugin, we strongly encourage you to use this library. The easiest is to copy an existing plugin and modify it for your gateway. The https://github.com/killbill/killbill-adyen-plugin[Adyen] plugin is a good starting point.
 
-Please refer to the http://docs.killbill.io/latest/plugin_development.html#_build_2[Plugin Development Documentation]