Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HDDS-3390. Rebase OFS branch - 2. Adapt OFS classes to HDDS-3101 #822

Closed
wants to merge 136 commits into from
Closed

HDDS-3390. Rebase OFS branch - 2. Adapt OFS classes to HDDS-3101 #822

wants to merge 136 commits into from

Conversation

smengcl
Copy link
Contributor

@smengcl smengcl commented Apr 15, 2020

What changes were proposed in this pull request?

  1. Rebase OFS dev branch
  2. Fix OFSPath: 9b47210
  • HDDS-3353 broke OFSPath since the latter uses org.apache.commons.codec.digest.DigestUtils but the dependency is removed.
  • HDDS-3359 broke OFSPath because the latter also uses org.apache.yetus.audience, but this is a simple fix - just replace it with org.apache.hadoop.hdds.annotation.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-3390

How was this patch tested?

This PR itself is only for triggering tests and commenting.
Will commit manually via command line.

hanishakoneru and others added 30 commits March 27, 2020 12:25
@hanishakoneru
HDDS-3281. Add timeouts to all robot tests (apache#723)
eece604
@rakeshadr
HDDS-3288: Update default RPC handler SCM/OM count to 100 (apache#729)
a661a75
@swagle
HDDS-3273. getConf does not return all OM addresses. (apache#727)
099ab62
@elek
Revert "HDDS-3064. Get Key is hung when READ delay is injected in chu…
d7dcb8b 
…nk file path. (apache#673)"

This reverts commit c8b6fff.
@elek
Revert "HDDS-3234. Fix retry interval default in Ozone client. (apach…
f5fa408 
…e#698)"

This reverts commit c64d86f.
@elek
HDDS-3294. Disable intermittent TestContainerStateMachineFailureOnRead
39e116a
@elek
HDDS-2011. Ignore TestRandomKeyGenerator.bigFileThan2GB
ad37f03
@elek
HDDS-3290. Ignore TestRandomKeyGenerator
b719173
@elek
HDDS-3297. Ignore TestOzoneClientKeyGenerator.
d2082b8
@elek
HDDS-3290. Disable all the freon integration tests
a66aae8
@elek
HDDS-3298. Ignore TestFailureHandlingByClient.testDatanodeExclusionWi…
f63c5af 
…thMajorityCommit
@elek
HDDS-3265. Ignore TestRatisPipelineLeader.java
a17551e
@elek
HDDS-3272. Disable hdfs 27 acceptance test
557c48e
@elek
HDDS-3251. Bump version to 0.6.0-SNAPSHOT
386baf1 
Closes apache#713
@hanishakoneru
HDDS-2980. Delete replayed entry from OpenKeyTable during commit (apa…
79ce00e 
…che#625)
@bharatviswa504
HDDS-3291. Write operation when both OM followers are shutdown. (apac…
5e23b25 
…he#733)
@adoroszlai
HDDS-2943. Parameterize unit tests for chunk manager implementation (a…
33ac261 
…pache#694)
@elek
HDDS-3160. Disable index and filter block cache for RocksDB
6b5298b 
Closes apache#665
@elek
HDDS-3023. Create Freon test to test isolated Ratis LEADER
94413cd 
Closes apache#555
@elek
HDDS-3236. Fix Dropwizard metrics mapping for latest Ratis metrics
18577a9 
Closes apache#707
@elek
HDDS-3142. Create isolated environment for OM to test it without SCM
2fefd46 
Closes apache#717
@isahkemat
HDDS-3249: remove unrelated assert (apache#737)
3d3b374
@isahkemat
HDDS-3248. shutdown defaultMetricsSystem before tests (apache#704)
97f59f8 
Co-authored-by: isa <isa.hekmatizadeh@snapp-box.com>
@hanishakoneru
HDDS-2621. Enable OM HA acceptance tests. (apache#265)
924b50c
@elek
HDDS-2184. Rename ozone scmcli to ozone admin (apache#687)
bd1995a
@timmylicheng
HDDS-3156 update allocateContainer to remove additional createPipelin…
f2343f1 
…e step. (apache#663)
@hanishakoneru
HDDS-2339. Add OzoneManager to MiniOzoneChaosCluster (apache#643)
410e4d3
@aryangupta1998
HDDS-3289. Add a freon generator to create nested directories. (apach…
7faa07f 
…e#730)
@elek
HDDS-3289. (Checkstyle addendum) Add a freon generator to create nest…
32f13f7 
…ed directories. (apache#730)
@elek
HDDS-3281. (Addendum: topology) Add timeouts to all robot tests
f43b33e 
Closes apache#743
@smengcl
Copy link
Contributor Author

smengcl commented Apr 20, 2020

Rebased as HDDS-3456 is committed.

umamaheswararao and others added 16 commits April 20, 2020 23:45
@umamaheswararao @bharatviswa504
HDDS-3380. MiniOzoneHAClusterImpl#initOMRatisConf will reset the conf… (
ce94889 
apache#817)

* HDDS-3380. MiniOzoneHAClusterImpl#initOMRatisConf will reset the configs and causes for test failures.

Co-authored-by: Bharat Viswanadham <bviswanadham@cloudera.com>
@elek
HDDS-3101. Depend on lightweight ConfigurationSource interface instea…
ffb340e 
…d of Hadoop Configuration

Closes apache#834
@bshashikant
HDDS-3468. Organize log4j dependency in pom.xml. (apache#849)
fc4d951
@elek
HDDS-3399. Update JaegerTracing
2acd2b1 
Closes apache#835
@bshashikant
HDDS-3471. Update to latest Ratis Snapshot 0.6.0-490b689-SNAPSHOT. (a…
6afe2a8 
…pache#851)
@timmylicheng
HDDS-3139. Pipeline placement should max out pipeline usage (apache#668)
fac26e1
@nandakumar131
HDDS-3451. Add timeout to XceiverServerRatis#submitRequest call. (apa…
a59ff25 
…che#845)
@smengcl
HDDS-3056. Allow users to list volumes they have access to, and optio…
091993b 
…nally allow all users to list all volumes (apache#696)
@xiaoyuyao
HDDS-3388. Add bucket encryption key info to bucket create audit log. (…
1b7f617 
…apache#820)
HDDS-2809. recipe/_index.md translation (apache#795)
ca6da46
@vivekratnavel
HDDS-3393. Recon throws NPE in clusterState endpoint (apache#854)
00799c2
@umamaheswararao
HDDS-3467. OM Client RPC failover retries happening more than configu…
36cdf5a 
…red. (apache#853)
@runzhiwang
HDDS-3168. Improve read efficiency by merging a lot of RPC call getCo…
706fe1b 
…ntainerWithPipeline into one. (apache#692)
@captainzmc
HDDS-3155. Improved ozone client flush implementation to make it fast…
2098516 
…er. (apache#716)
@elek
HDDS-3172. Use DBStore instead of MetadataStore in SCM
0466ade 
Closes apache#700
HDDS-2810. recipe/Prometheus.md translation (apache#796)
fa3aaf1
@smengcl
Copy link
Contributor Author

smengcl commented Apr 22, 2020

#847 (OFSPath refactoring) is merged. Will update this draft in a moment to trigger a new test run.

@smengcl
Merge remote-tracking branch 'asf/master' into HDDS-3390
e8198f2 
Conflicts:
hadoop-ozone/dist/src/main/smoketest/ozonefs/ozonefs.robot
@smengcl
Copy link
Contributor Author

smengcl commented Apr 23, 2020 •
edited

Run in this jira only failed acceptance, the rest passed: https://github.com/apache/hadoop-ozone/runs/611064662 - Actually, the acceptance test suites passed. I checked the log/report manually and all shows passing. It reports failure due to cp: cannot stat '/home/runner/work/hadoop-ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.6.0-SNAPSHOT/compose/ozonesecure/result/*.out*': No such file or directory during clean up.

Run in my branch only failed it-client, the rest passed: https://github.com/smengcl/hadoop-ozone/runs/611083862. TestBlockOutputStreamWithFailures is known flaky.

I can deem the test passed at this point.

@smengcl smengcl changed the title HDDS-3390. Rebase OFS branch - 2 HDDS-3390. Rebase OFS branch - 2. Adapt OFS classes to HDDS-3101 Apr 23, 2020
asfgit pushed a commit that referenced this pull request Apr 23, 2020
@smengcl
HDDS-3390. Rebase OFS branch - 2. Adapt OFS classes to HDDS-3101 (#822)
9ed2e15
@smengcl
Copy link
Contributor Author

smengcl commented Apr 23, 2020

Pushed merge (minor conflict in ozonefs.robot due to master branch change), and adapt commit.

Closing this PR.

@smengcl smengcl closed this Apr 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone