DRILL-7547: Support credentials store for mongo connections

[dobesv]

DRILL-7547: Support credentials store for mongo connections

Description

This uses the hadoop Configuration.getPassword method to retrieve the
username and password for mongo connections. This allows the user to
supply credentials or credential store configuration in core-site.xml
instead of inlining the credentials in the storage plugin configuration
that is stored in ZooKeeper.

Refer to the CredentialProviderAPI document for more information about
how credential provider plugins work.

Documentation

Defining Credentials in the Drill core-site.xml File

To configure the mongo username and password in Drill's core-site.xml file, navigate to the $DRILL_HOME/conf or $DRILL_SITE directory, and rename the core-site-example.xml file to core-site.xml. Insert your mongo username and password as shown in the following example:

   <configuration>
       <property>
           <name>drill.exec.store.mongo.username</name>
           <value>drill</value>
       </property>
       <property>
           <name>drill.exec.store.mongo.password</name>
           <value>secretpassword</value>
       </property>
   </configuration>  

Remove any username and password from your mongo storage plugin configuration if you do this.

Notes

• as with the S3 plugin you can configure an external credentials provider. Refer to those docs for details
• if you have multiple mongo storage plugins setup or your mongo plugin is not named mongo, replace "mongo" in the property name with the name of the storage plugin you are configuring, e.g. for a storage plugin named "mongo2" you would set drill.exec.store.mongo2.username and drill.exec.store.mongo2.password

Testing

JUnit tests and manual test.

[cgivre]

@dobesv Thanks for this! Once this is approved, can you update the Mongo Storage Plugin info on the gh-pages branch for the main Drill website?

[cgivre]

A question and a comment:

1. What happens if the user has multiple mongo storage plugins? Are the creds carried over to all of them? If a user specifies creds in the storage plugin config does it overwrite the config file?

2. This isn't for this PR, but would it make sense for us to do this for other storage plugins that are not likely to have multiple instances? Kudu, HBase or Hive for instance?

[dobesv]

1. What happens if the user has multiple mongo storage plugins? Are the creds carried over to all of them? If a user specifies creds in the storage plugin config does it overwrite the config file?

The configuration key is based on the plugin name, actually. mongo is the default name for the storage plugin, but if you had another one named mongo-prod you would set drill.exec.store.mongo-prod.username.

If the connection string already has credentials, this will not replace them.

2. This isn't for this PR, but would it make sense for us to do this for other storage plugins that are not likely to have multiple instances? Kudu, HBase or Hive for instance?

I think it is an OK approach for any plugin that has credentials currently stored in ZooKeeper. Note that the key I use is based on the plugin's name so you can setup multiple.

[dobesv]

@arina-ielchiieva Great catch on the core-site.xml location, I totally thought I had put that in the test resources before, such a silly mistake. Hooray for code reviews!

I think I addressed your other comments as well, and I did a bit of research and added some more detail to the core-site-example file. The credentials provider system is actually pretty powerful once you get into it.

[arina-ielchiieva]

LGTM, +1