Implement upgrade from 3.1 to 4.0

[keith-turner]

The elasticity branch does not currently have functioning upgrade code. Need to implement upgrade from 3.1 to 4.0.

[dlmarion]

This is already started, in class Upgrader12to13, it's just that the upgrade is disabled in AccumuloDataVersion.

[keith-turner]

We have issues open for some upgrade task. Plan to do a survey of the code and see if any other issue need to be opened and find all of the issues that are currently open.

[keith-turner]

Some of the issues currently open re upgrade are #4692, #4652, #4637, #4315. For the survey of what may need to be done for upgrade, plan to look over what is currently done for upgrade, metadata schema changes, zookeeper schema changes, and prop changes.

[keith-turner]

Compared the files Constants.java, MetadataSchema.java, and Property.java across elasticity and main branches. Analyzing Constants.java resulted in opening #4712 and #4713. Looking at MetadataSchema.java did not see anything to do, did notice #4637.

Looking at Property.java the only table property that was removed was table.compaction.selector. Wondering if we want to delete this if it exists on upgrade. There are also some properties that could have been set at the system level that were deleted. This is related to #4315, but more general.

[keith-turner]

The upgrade code in elasticity currently has no tests. Will need to add test. Was pondering doing the following for testing upgrade.

• Create an IT in main/3.1 branch that :
  • runs mini accumulo
  • creates data in zookeeper and metadata table that upgrade would need to delete
  • stops all processes
  • then creates a zip file of the mini accumulo directory
• In the elasticity branch do the following :
  • copy the zip file created in the 3.1 branch to test resources
  • create an IT that initializes a mini accumulo instance using this zip file
  • verify upgrade happened
  • add a comment to the IT about running the test in the other branch to update the zip file

The IT in elasticity could use this MAC functionality

accumulo/minicluster/src/main/java/org/apache/accumulo/miniclusterImpl/MiniAccumuloConfigImpl.java

Line 737 in f0e3849

public MiniAccumuloConfigImpl useExistingInstance(File accumuloProps, File hadoopConfDir)