Skip to content

Commit

Permalink
Exclude org.apache.hadoop classes from tables-test-fixtures uber jar (#…
Browse files Browse the repository at this point in the history
…140)

## Summary

<!--- HINT: Replace #nnn with corresponding Issue number, if you are
fixing an existing issue -->

tables test fixtures fat jar ships with a version of OSS hadoop thats
incompatible with newer version of li-hadoop (specifically 2.10.0.780).
When li-openhouse was upgraded to this version of li-hadoop, unit tests
depending on tables test fixtures fat jar started to fail when starting
a test hdfs cluster.

- [ ] Client-facing API Changes
- [ ] Internal API Changes
- [x] Bug Fixes
- [ ] New Features
- [ ] Performance Improvements
- [ ] Code Style
- [ ] Refactoring
- [ ] Documentation
- [ ] Tests

For all the boxes checked, please include additional details of the
changes made in this pull request.

## Testing Done
<!--- Check any relevant boxes with "x" -->

- [ ] Manually Tested on local docker setup. Please include commands
ran, and their output.
- [ ] Added new tests for the changes made.
- [ ] Updated existing tests to reflect the changes made.
- [ ] No tests added or updated. Please explain why. If unsure, please
feel free to ask for help.
- [x] Some other form of testing like staging or soak time in
production. Please explain.

Unpacked the jar locally after changes and verified that all hadoop
classes are relocated
<img width="1431" alt="image"
src="https://github.com/user-attachments/assets/6c61ac46-0209-4630-99b7-4b6f14a66736">

For all the boxes checked, include a detailed description of the testing
done for the changes made in this pull request.

# Additional Information

- [ ] Breaking Changes
- [ ] Deprecations
- [ ] Large PR broken into smaller PRs, and PR plan linked in the
description.

For all the boxes checked, include additional details of the changes
made in this pull request.

Co-authored-by: Vikram Bohra <vbohra@vbohra-mn2298.linkedin.biz>
  • Loading branch information
vikrambohra and Vikram Bohra committed Jul 22, 2024
1 parent b04bb5c commit b65b434
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions tables-test-fixtures/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ shadowJar {
exclude(dependency('ch.qos.logback::'))
exclude(dependency('com.fasterxml.jackson.module::'))
exclude(dependency('com.fasterxml.jackson.core::'))
exclude(dependency('org.apache.hadoop::'))
relocate ('org.yaml.snakeyaml', 'openhouse.relocated.org.yaml.snakeyaml')
relocate ('com.jayway.jsonpath', 'openhouse.relocated.com.jayway.jsonpath')
relocate ('org.json', 'openhouse.relocated.org.json')
Expand Down

0 comments on commit b65b434

Please sign in to comment.