[Hive Support] Implement mapred Mrv1 API for Iceberg

[rdsr]

We are doing some work on support Mrv2 API. This API is will eventually be used by Pig and standalone MR. There are other MR based frameworks which have not moved away from the older MRv1 api e.g Hive. Also AFAIK both apis are stable and neither has been marked deprecated.

We should provide support for Mrv1 API. There are libraries which provide an automatic conversion from Mrv2 to Mrv1, e.g see #843 (comment), something to take inspiration from.

[jerryshao]

Any progress on this @rdsr @massdosage , I saw this PR #933 has an implementation from scratch, should we refactor based on the current work?

[massdosage]

We are working on this via #933 but in turn have discovered that we are first going to need #935 so it's dependent on that. What are you proposing gets refactored?

[jerryshao]

What implemented in #933 are mostly addressed in #843 , like mentioned here, we could have a library to covert from MRv2 to MRv1, no need to implement the same logic again. That's what I mean about the refactoring. I don't think this work is blocked by guava thing.

[massdosage]

Yes, we plan to try re-use as much code from #843 in #933 as possible, it's being discussed on the related WIP PR. Unfortunately this is blocked by Guava because when you try actually use the InputFormat in Hive it fails due to Hive having a different version of Guava on its classpath which leads to runtime exceptions with method signatures that have changed etc. So one can't actually use the InputFormat until we resolve all the version clashes between classes that both Iceberg and Hive use that have incompatible version changes between them. We have an integration test using HiveRunner that we could add here to demonstrate this but it will require quite a few exclusions, some new dependencies on various Hive modules and all the shading and relocating before it will pass.

[pvary]

Checking Hive related issues I have found this. Can we close this now as the implementation is committed?