Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HDDS-4077. Incomplete OzoneFileSystem statistics #1329

Merged
merged 3 commits into from Aug 31, 2020
Merged

HDDS-4077. Incomplete OzoneFileSystem statistics #1329

merged 3 commits into from Aug 31, 2020

Conversation

adoroszlai
Copy link
Contributor

What changes were proposed in this pull request?

Override some methods only for statistics. S3AFileSystem also does so.

https://issues.apache.org/jira/browse/HDDS-4077

How was this patch tested?

Enabled DEBUG log for Ozone FS:

etc/hadoop/ozone-shell-log4j.properties
35:log4j.logger.org.apache.hadoop.fs.ozone=DEBUG

etc/hadoop/log4j.properties
138:log4j.logger.org.apache.hadoop.fs.ozone=DEBUG

Ran ozonefs.robot and verified some previously missing operations are logged (for both OFS and O3FS).

2020-08-12 12:42:38,057 [main] DEBUG ozone.OzoneFSStorageStatistics: op_glob_status += 1  ->  1
...
2020-08-12 12:42:51,163 [main] DEBUG ozone.OzoneFSStorageStatistics: op_mkdirs += 1  ->  1
...
2020-08-12 12:43:15,071 [main] DEBUG ozone.OzoneFSStorageStatistics: op_exists += 1  ->  1

@adoroszlai adoroszlai self-assigned this Aug 13, 2020
@bshashikant bshashikant merged commit 78ca8bf into apache:master Aug 31, 2020
@adoroszlai adoroszlai deleted the HDDS-4077 branch September 1, 2020 09:12
@adoroszlai
Copy link
Contributor Author

Thanks @bshashikant for reviewing and merging this.

rakeshadr pushed a commit to rakeshadr/hadoop-ozone that referenced this pull request Sep 3, 2020
@adoroszlai @rakeshadr
HDDS-4077. Incomplete OzoneFileSystem statistics (apache#1329)
3ca3278
errose28 added a commit to errose28/ozone that referenced this pull request Sep 11, 2020
@errose28
Merge branch 'master' into HDDS-4122
c2043a2 
* master: (26 commits)
  HDDS-4167. Acceptance test logs missing if fails during cluster startup (apache#1366)
  HDDS-4121. Implement OmMetadataMangerImpl#getExpiredOpenKeys. (apache#1351)
  HDDS-3867. Extend the chunkinfo tool to display information from all nodes in the pipeline. (apache#1154)
  HDDS-4077. Incomplete OzoneFileSystem statistics (apache#1329)
  HDDS-3903. OzoneRpcClient support batch rename keys. (apache#1150)
  HDDS-4151. Skip the inputstream while offset larger than zero in s3g (apache#1354)
  HDDS-4147. Add OFS to FileSystem META-INF (apache#1352)
  HDDS-4137. Turn on the verbose mode of safe mode check on testlib (apache#1343)
  HDDS-4146. Show the ScmId and ClusterId in the scm web ui. (apache#1350)
  HDDS-4145. Bump version to 1.1.0-SNAPSHOT on master (apache#1349)
  HDDS-4109. Tests in TestOzoneFileSystem should use the existing MiniOzoneCluster (apache#1316)
  HDDS-4149. Implement OzoneFileStatus#toString (apache#1356)
  HDDS-4153. Increase default timeout in kubernetes tests (apache#1357)
  HDDS-2411. add a datanode chunk validator fo datanode chunk generator (apache#1312)
  HDDS-4140. Auto-close /pending pull requests after 21 days of inactivity (apache#1344)
  HDDS-4152. Archive container logs for kubernetes check (apache#1355)
  HDDS-4056. Convert OzoneAdmin to pluggable model (apache#1285)
  HDDS-3972. Add option to limit number of items displaying through ldb tool. (apache#1206)
  HDDS-4068. Client should not retry same OM on network connection failure (apache#1324)
  HDDS-4062. Non rack aware pipelines should not be created if multiple racks are alive. (apache#1291)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bshashikant bshashikant bshashikant approved these changes

Assignees

@adoroszlai adoroszlai

Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@adoroszlai @bshashikant