HDDS-3237. Recon should provide the list of datanodes that a missing …

[vivekratnavel]

…container was present in.

What changes were proposed in this pull request?

• Create a new table in SQLite to keep track of container replica history
• Include the last known datanodes of a missing container in API (/api/v1/containers/missing)
• Add a new API (/api/v1/containers/{id}/replicaHistory) to get complete history of container replicas.
• Make changes in the UI to display datanode information in missing containers page.
• Add unit and acceptance tests.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-3237

How was this patch tested?

• Unit tests, acceptance tests and manual test via docker-compose.

To test the change,
cd hadoop-ozone/dist/target/ozone-0.6.0-SNAPSHOT/compose/ozone
Add the following lines to docker-config for faster testing

OZONE-SITE.XML_ozone.scm.dead.node.interval=2m
OZONE-SITE.XML_ozone.scm.stale.node.interval=1m

docker-compose up -d
docker-compose exec om bash

ozone freon rk --replicationType=RATIS --numOfVolumes=10 --numOfBuckets=10 --numOfKeys=10 --factor=ONE --numOfThreads=20

Now, exit the shell and bring down datanode
docker ps
docker kill <containerid>

Navigate to Recon UI and wait for a few minutes to see missing container alert in the Overview page. Clicking on the alert box should take you to the missing containers page:

[vivekratnavel]

@avijayanhwx @elek Please review

[avijayanhwx]

Overall, this approach looks good. Some comments inline. With this JIRA, we are creating a DB operation for every replica in every report. Can we take a follow up task to try this out against a busy cluster with around 1000 containers at least?

[avijayanhwx]

Thank you for working on this @vivekratnavel.