Skip to content

[GOBBLIN-1487] Add gridfs datanode gaas#3326

Merged
autumnust merged 2 commits into
apache:masterfrom
Will-Lo:add-gridfs-datanode-gaas
Jul 1, 2021
Merged

[GOBBLIN-1487] Add gridfs datanode gaas#3326
autumnust merged 2 commits into
apache:masterfrom
Will-Lo:add-gridfs-datanode-gaas

Conversation

@Will-Lo
Copy link
Copy Markdown
Contributor

@Will-Lo Will-Lo commented Jun 30, 2021

Dear Gobblin maintainers,

Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!

JIRA

Description

  • Here are some details about my PR, including screenshots (if applicable):
    Adds GridFS as a recognized fs schema

Tests

  • My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

  • My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Jun 30, 2021

Codecov Report

❌ Patch coverage is 55.55556% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.40%. Comparing base (2407be6) to head (48232c5).
⚠️ Report is 774 commits behind head on master.

Files with missing lines Patch % Lines
...modules/flowgraph/datanodes/fs/GridFSDataNode.java 55.55% 2 Missing and 2 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3326      +/-   ##
============================================
- Coverage     46.41%   46.40%   -0.02%     
- Complexity    10004    10014      +10     
============================================
  Files          2031     2034       +3     
  Lines         79002    79069      +67     
  Branches       8834     8840       +6     
============================================
+ Hits          36668    36689      +21     
- Misses        38932    38973      +41     
- Partials       3402     3407       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Will-Lo
Copy link
Copy Markdown
Contributor Author

Will-Lo commented Jul 1, 2021

@aplex @jack-moseley @arjun4084346 @umustafi please take a look

job.name=DistcpToGridFS

#team.name and dataset.name to be supplied via flowConfig
from=/data/out/${dataset.name}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are the from and to the same because we are making a copy from hdfs to gridfs?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's just the local cluster path in this scenario, so it doesn't matter what the fs is as it gets accounted for later.

@autumnust autumnust merged commit 7bbd722 into apache:master Jul 1, 2021
@Will-Lo Will-Lo deleted the add-gridfs-datanode-gaas branch May 4, 2022 07:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants