ARTEMIS-4205 Database Primitive Manager #4402
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Using locking capabilities of the database, implement the a primitive manager to allow quorum voting to occur. This implementation has support for Oracle, MSSQL and Postgres -- with H2 being used for unit tests.
The locking features used are not locks against a tables, but instead the idea of an arbitrarily named "application" lock. There is no SQL standard for this behavior and each database implementation is different. I have accounted for this by creating the concept of a database adapter -- i.e. adapting the primitive manager to work for the particular database. I have attempted to reduce duplication by declaring a BaseDatabaseAdapter which I think should work generally for all databases.
Additionally, I introduced the concept of a DatabaseConnectionProvider. This was done for any users that are utilizing Artemis as an embedded feature or that otherwise need to declare their own mechanism for getting credentials. For instance, they may very well want to utilize some encryption for storing a database password in the configuration. The DefaultDatabaseConnectionProvider just gets url, username and password from the properties. It is likely that this implementation is enough for most uses, but we may want to see if we can make it optional to store the password encrypted. I am wondering if there is stuff already in existence for this kind of thing?
I tried to follow the same locking approach in the zookeeper curator implementation to keep the threading/calling equivalently safe, but review of this is welcome.
In the end it was only necessary to create a database record when storing a mutable long value. These records are held in an ARTEMIS_LOCKS table. A lock that doesn't use and store a long value can come and go without any lasting signature in the database signature. The default value for a lock is 0 and doesn't require a record in the database until the long value is updated. The longs must be stored in the database for a different node to be able to acquire a lock and have the same long value present there, of course.
I would welcome logical review of my approach to maintenance and cleanup by someone more familiar with all aspects of using these locks:
I store a "LAST_ACCESS" value with each of those LONG_VALUE records. Periodically I enumerate the records that have LAST_ACCESS over two hours ago, then grab the lock. The assumption is this: if the maintenance system can grab the lock and it hasn't been accessed in over two hours, it is probably defunct and should be cleaned up.
I didn't know what exactly was culture regarding the database drivers. Oracle and MSSQL drivers are both proprietary and Postgres is open source. Ultimately in order for anyone to use this feature, they need to provide the right driver for the database they are using. But to make the code implementation testable via unit tests, I used the open source H2 database and wrote a little piece that extended the database to add locking functions similar to Oracle, MSSQL and Postgres. I include instructions in the comments at the top of the DatabaseDistributedLockTest on how someone can setup to run the unit tests in Oracle, MSSQL or Postgres modes. I am ultimately leaving this running as much of the code as possible via unit tests, but inherently to run all of it would require 3 database servers with logins.
My rationale for creating this feature is that I believe if someone is using the broker in an application environment where there is a database already present, it would be nice for them to be able to simply use that database and not have to have yet another piece of infrastructure to provide for the quorum based ha replication that Artemis supports. And I also think that it should be possible to do this kind of thing with just 2 broker nodes without requiring a shared file system -- if someone wanted this.