Skip to content

[#1414] feat(rust): introduce native hdfs client - #1415

Merged
zuston merged 5 commits into
apache:masterfrom
zuston:hdfs-native
Jan 4, 2024
Merged

[#1414] feat(rust): introduce native hdfs client#1415
zuston merged 5 commits into
apache:masterfrom
zuston:hdfs-native

Conversation

@zuston

@zuston zuston commented Jan 3, 2024

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

Introduce hdfs-native crate as the native integration.

Why are the changes needed?

For: #1414

Native rust hdfs client could simplify the hadoop dependency and avoid JNI invoking cost.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Internal tests on Hadoop 3.1.2

@zuston
zuston requested a review from roryqi January 3, 2024 06:45
@zuston zuston linked an issue Jan 3, 2024 that may be closed by this pull request
3 tasks
roryqi
roryqi previously approved these changes Jan 3, 2024

@roryqi roryqi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@roryqi

roryqi commented Jan 3, 2024

Copy link
Copy Markdown
Contributor

You should pay attention to the dependencies of the project. The license is important.

@codecov-commenter

codecov-commenter commented Jan 3, 2024

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (a98412e) 53.24% compared to head (c8129b7) 54.18%.
Report is 2 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #1415      +/-   ##
============================================
+ Coverage     53.24%   54.18%   +0.93%     
- Complexity     2720     2721       +1     
============================================
  Files           419      399      -20     
  Lines         23966    21605    -2361     
  Branches       2042     2042              
============================================
- Hits          12761    11706    -1055     
+ Misses        10418     9183    -1235     
+ Partials        787      716      -71     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@zuston

zuston commented Jan 3, 2024

Copy link
Copy Markdown
Member Author

You should pay attention to the dependencies of the project. The license is important.

It's under Apache 2.0

@zuston
zuston requested a review from roryqi January 3, 2024 07:09

@advancedxy advancedxy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, this is great.

@zuston
zuston merged commit 602611c into apache:master Jan 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] native hdfs client for rust

4 participants