[CARBONDATA-3185] Fix alluxio file rename

[kevinjmh]

Problem
Exception thrown when create table on alluxio because rename schema file failed

Analyse
Re-run the command after adding some logs, I found that the file system object it uses is alluxio.hadoop.FileSystem, and it is not an instanceof DistributedFileSystem, such that the renameForce method return false.

Solution
By checking hierarchy of class alluxio.hadoop.FileSystem, it is ok to use method of hadoop FileSystem directly. We can fix it by remove instant type check and rewrite the overwrite file code by hand.

FileSystem (org.apache.hadoop.fs)
|__ AbstractFileSystem (alluxio.hadoop)
      |__FileSystem (alluxio.hadoop)

Log

18/12/20 09:31:50 ERROR thriftserver.SparkExecuteStatementOperation: Error executing query, currentState RUNNING,
org.apache.carbondata.spark.exception.ProcessMetaDataException: operation failed for default.alluxio: Create table'alluxio' in database 'default' failed, temporary file renaming failed, src=alluxio://localhost:19998/user/hive/warehouse/carbon.store/default/alluxio/Metadata/schema.write, dest=alluxio://localhost:19998/user/hive/warehouse/carbon.store/default/alluxio/Metadata/schema
        at org.apache.spark.sql.execution.command.MetadataProcessOpeation$class.throwMetadataException(package.scala:55)
        at org.apache.spark.sql.execution.command.MetadataCommand.throwMetadataException(package.scala:120)
        at org.apache.spark.sql.execution.command.table.CarbonCreateTableCommand.processMetadata(CarbonCreateTableCommand.scala:179)
        at org.apache.spark.sql.execution.command.MetadataCommand$$anonfun$run$1.apply(package.scala:122)
        at org.apache.spark.sql.execution.command.MetadataCommand$$anonfun$run$1.apply(package.scala:122)
        at org.apache.spark.sql.execution.command.Auditable$class.runWithAudit(package.scala:104)
        at org.apache.spark.sql.execution.command.MetadataCommand.runWithAudit(package.scala:120)
        at org.apache.spark.sql.execution.command.MetadataCommand.run(package.scala:122)
        at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
        at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
        at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:67)
        at org.apache.spark.sql.Dataset.<init>(Dataset.scala:183)
        at org.apache.spark.sql.CarbonSession$$anonfun$sql$1.apply(CarbonSession.scala:91)
        at org.apache.spark.sql.CarbonSession$$anonfun$sql$1.apply(CarbonSession.scala:90)
        at org.apache.spark.sql.CarbonSession.withProfiler(CarbonSession.scala:136)
        at org.apache.spark.sql.CarbonSession.sql(CarbonSession.scala:88)

Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:

• Any interfaces changed?

• Any backward compatibility impacted?

• Document update required?

• Testing done
  Please provide details on
  - Whether new unit test cases have been added or why no new tests are required?
  - How it is tested? Please attach test report.
  - Is it a performance related change? Please attach the performance test report.
  - Any additional information to help reviewers in testing this change.

• For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

[xubo245]

are there any risk for deleting path?

[kevinjmh]

This should be managed by developer. If developer wants a simple rename, he should call renameTo instead of renameForce. Codes in this PR implements the OVERWRITE(delete before rename), as S3CarbonFile , ViewFSCarbonFile, LocalCarbonFile do

[xubo245]

There is a PR to fix the same issue: #2161

[CarbonDataQA]

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1861/

[CarbonDataQA]

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2069/

[CarbonDataQA]

Build Failed with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10116/

[kevinjmh]

There is a PR to fix the same issue: #2161

I see. It fixed LockFile problem I faced now too.
Hope it got fix soon. And I am closing this PR now