Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HADOOP-16398. Exports Hadoop metrics to Prometheus #1170

Merged
merged 2 commits into from
Jul 31, 2019
Merged

HADOOP-16398. Exports Hadoop metrics to Prometheus #1170

merged 2 commits into from
Jul 31, 2019

Conversation

aajisaka
Copy link
Member

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
0 reexec 77 Docker mode activated.
_ Prechecks _
+1 dupname 1 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
0 mvndep 73 Maven dependency ordering for branch
+1 mvninstall 1871 trunk passed
+1 compile 1582 trunk passed
+1 checkstyle 202 trunk passed
+1 mvnsite 146 trunk passed
+1 shadedclient 1236 branch has no errors when building and testing our client artifacts.
+1 javadoc 139 trunk passed
0 spotbugs 60 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 207 trunk passed
_ Patch Compile Tests _
0 mvndep 35 Maven dependency ordering for patch
+1 mvninstall 105 the patch passed
+1 compile 1409 the patch passed
+1 javac 1409 the patch passed
+1 checkstyle 228 the patch passed
+1 mvnsite 157 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 xml 1 The patch has no ill-formed XML file.
+1 shadedclient 791 patch has no errors when building and testing our client artifacts.
+1 javadoc 109 the patch passed
+1 findbugs 207 the patch passed
_ Other Tests _
+1 unit 604 hadoop-common in the patch passed.
+1 unit 64 framework in the patch passed.
+1 asflicense 79 The patch does not generate ASF License warnings.
9341
Subsystem Report/Notes
Docker Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1170/1/artifact/out/Dockerfile
GITHUB PR #1170
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname Linux e0d6764ddbb4 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 3426777
Default Java 1.8.0_212
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1170/1/testReport/
Max. process+thread count 1486 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-hdds/framework U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1170/1/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
0 reexec 72 Docker mode activated.
_ Prechecks _
+1 dupname 1 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
0 mvndep 30 Maven dependency ordering for branch
+1 mvninstall 2004 trunk passed
+1 compile 1828 trunk passed
+1 checkstyle 203 trunk passed
+1 mvnsite 141 trunk passed
+1 shadedclient 1113 branch has no errors when building and testing our client artifacts.
+1 javadoc 108 trunk passed
0 spotbugs 62 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 199 trunk passed
_ Patch Compile Tests _
0 mvndep 35 Maven dependency ordering for patch
+1 mvninstall 102 the patch passed
+1 compile 1782 the patch passed
+1 javac 1782 the patch passed
+1 checkstyle 194 the patch passed
+1 mvnsite 131 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 xml 2 The patch has no ill-formed XML file.
+1 shadedclient 749 patch has no errors when building and testing our client artifacts.
+1 javadoc 102 the patch passed
+1 findbugs 213 the patch passed
_ Other Tests _
+1 unit 621 hadoop-common in the patch passed.
+1 unit 46 framework in the patch passed.
+1 asflicense 53 The patch does not generate ASF License warnings.
9737
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1170/2/artifact/out/Dockerfile
GITHUB PR #1170
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname Linux 6f8f40534f6b 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / aebac6d
Default Java 1.8.0_212
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1170/2/testReport/
Max. process+thread count 1374 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-hdds/framework U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1170/2/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

anuengineer
anuengineer approved these changes Jul 26, 2019
View reviewed changes
Copy link
Contributor

@anuengineer anuengineer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM

@anuengineer
Copy link
Contributor

Setting the Ozone label to get an Ozone test run too. But we are good with committing and I will go ahead and commit this soon.

anuengineer
anuengineer reviewed Jul 26, 2019
View reviewed changes
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/BaseHttpServer.java
// TODO: Replace "hadoop.prometheus.endpoint.enabled" with
// CommonConfigurationKeysPublic.HADOOP_PROMETHEUS_ENABLED when possible.
conf.setBoolean("hadoop.prometheus.endpoint.enabled", false);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aajisaka Just trying to understand all the options here. I see we have 3 options of running this code.

  1. Ozone and HDFS as independent processes - No issues.
  2. HDFS starts first, and Ozone is enabled in the running process -- Works because the Prometheus servelet is loaded and running already.
  3. Ozone is running first -- and then HDFS starts up -- since they are independent processes it will work. That is, the port/prom , the port will be different so we are ok,
  4. There is no case, where both (Ozone and HDFS) are in the same process but Ozone starts first and then HDFS starts. Hence this setting is safe.
    @elek , @adoroszlai Thoughts? I am going to commit this tomorrow since we are all in different time zones. Just making sure that my reasoning is correct.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I think your understanding is correct.

adamantal
adamantal approved these changes Jul 29, 2019
View reviewed changes
Copy link
Contributor

@adamantal adamantal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 (non-binding) from me

@aajisaka aajisaka merged commit 8bda91d into apache:trunk Jul 31, 2019
@aajisaka
Copy link
Member Author

Thanks @anuengineer and @adamantal for the reviews. Merged.

@aajisaka aajisaka deleted the HADOOP-16398 branch July 31, 2019 17:12
shanthoosh pushed a commit to shanthoosh/hadoop that referenced this pull request Oct 15, 2019
@Sanil15 @rmatharu-zz
Samza-2330: Handle expired resource request for Container allocator w…
2ae3450 
…hen host affinity is disabled (apache#1170)

* Adding expiry check for unresponsive Cluster Manager when host affinity is off

* Fixing after rebase

* Addressing Ray's feedback

* Updating javadocs

* Nitpick improvements
amahussein pushed a commit to amahussein/hadoop that referenced this pull request Oct 29, 2019
@aajisaka
HADOOP-16398. Exports Hadoop metrics to Prometheus (apache#1170)
542f65a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@anuengineer anuengineer anuengineer approved these changes

@adamantal adamantal adamantal approved these changes

Assignees
No one assigned
Labels
ozone
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@aajisaka @hadoop-yetus @anuengineer @adamantal