[GOBBLIN-910] Added a unix timestamp recursive copyable dataset finder

[vikrambohra]

Dear Gobblin maintainers,

Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!

JIRA

• [ x] My PR addresses the following Gobblin-910 issues and references them in the PR title. For example, "[GOBBLIN-910] My Gobblin PR"
  • https://issues.apache.org/jira/browse/GOBBLIN-910

Description

• [ x] Here are some details about my PR, including screenshots (if applicable):
• Added a unix timestamp recursive copyable dataset finder
• a snapshot selection policy
• lookback time

Tests

• [x ] My PR adds the following unit tests OR does not need testing for this extremely good reason:
• UnixTimestampRecursiveCopyableDatasetTest
• Ran tests on sample datasets on hadoop

Commits

• [x ] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

[vikrambohra]

@sv2000

[codecov-io]

Codecov Report

Merging #2765 into master will increase coverage by 0.03%.
The diff coverage is 85.71%.

@@             Coverage Diff              @@
##             master    #2765      +/-   ##
============================================
+ Coverage      45.3%   45.33%   +0.03%     
- Complexity     8846     8858      +12     
============================================
  Files          1892     1894       +2     
  Lines         70752    70815      +63     
  Branches       7773     7784      +11     
============================================
+ Hits          32054    32106      +52     
- Misses        35740    35747       +7     
- Partials       2958     2962       +4

Impacted Files	Coverage Δ	Complexity Δ
...ement/copy/UnixTimestampCopyableDatasetFinder.java	100% <100%> (ø)	2 <2> (?)
...nt/copy/UnixTimestampRecursiveCopyableDataset.java	85% <85%> (ø)	8 <8> (?)
...in/java/org/apache/gobblin/cluster/HelixUtils.java	35.51% <0%> (-3.74%)	12% <0%> (-1%)
...lin/elasticsearch/writer/FutureCallbackHolder.java	61.42% <0%> (-1.43%)	4% <0%> (ø)
.../org/apache/gobblin/metrics/RootMetricContext.java	79.68% <0%> (+1.56%)	16% <0%> (+1%)	⬆️
...lin/util/filesystem/FileSystemInstrumentation.java	100% <0%> (+7.14%)	4% <0%> (+1%)	⬆️
...a/org/apache/gobblin/util/limiter/NoopLimiter.java	60% <0%> (+20%)	3% <0%> (+1%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b92f516...cf51242. Read the comment docs.

[sv2000]

The PR is missing a title. @vikrambohra

[vikrambohra]

@sv2000

[sv2000]

@vikrambohra there are some unrelated changes that have been pulled into this PR. Can you back out those unrelated changes?

[vikrambohra]

I just did a pull from upstream master before pushing.

On Thu, Oct 17, 2019 at 9:45 PM Sudarshan Vasudevan < ***@***.***> wrote: @vikrambohra <https://github.com/vikrambohra> there are some unrelated changes that have been pulled into this PR. Can you back out those unrelated changes? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2765>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF4PH7NMG2H4XPVRUTSAXMDQPE5P3ANCNFSM4JBEIT7Q> .

-- Vikram Bohra Tel: 352-284-1403

[sv2000]

+1. LGTM.