Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JENKINS-50294] Prototype health checking #44

Conversation

@batmat
Copy link
Collaborator

commented Apr 5, 2018

Context: JENKINS-50294

The goal here is to show (see the tests.sh addition) how health-checking can be done:

  • simple check on /login that it returns HTTP-200
  • simple health-check on the pre-configured metrics-plugin exposed URL
@jenkinsadmin

This comment has been minimized.

Copy link

commented Apr 5, 2018

Build failed; the context from the latest run is:

Expand to view
a630c29327b2: Pull complete
85cc490838b0: Pull complete
Digest: sha256:31a1a26eb18bfc4a36466cc7dfdb0e3cc96b4c6ddc1aa6c2512357fecc9d9d5e
Status: Downloaded newer image for sahsu/docker-jsonlint:latest
test_login_http_200
test_metrics_health_check
ASSERT:expected:<200> but was:<403>
+ exec docker run --rm -w /home/jenkins/workspace/Infra_evergreen_PR-44-ZIKSHT5HJI6X2WTRNCL37EPUQ6JLCZZ7FKWALNCL236SRCBKNFJQ -v /home/jenkins/workspace/Infra_evergreen_PR-44-ZIKSHT5HJI6X2WTRNCL37EPUQ6JLCZZ7FKWALNCL236SRCBKNFJQ:/home/jenkins/workspace/Infra_evergreen_PR-44-ZIKSHT5HJI6X2WTRNCL37EPUQ6JLCZZ7FKWALNCL236SRCBKNFJQ -i sahsu/docker-jsonlint jsonlint
Error: Parse error on line 1:
<html><head><meta ht
^
Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '[', got 'undefined'
    at Object.parseError (/usr/lib/node_modules/jsonlint/lib/jsonlint.js:55:11)
    at Object.parse (/usr/lib/node_modules/jsonlint/lib/jsonlint.js:132:22)
    at parse (/usr/lib/node_modules/jsonlint/lib/cli.js:82:14)
    at Socket.<anonymous> (/usr/lib/node_modules/jsonlint/lib/cli.js:149:41)
    at emitNone (events.js:91:20)
    at Socket.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:80:11)
    at process._tickCallback (internal/process/next_tick.js:104:9)
ASSERT:expected:<0> but was:<1>

Cleaning up... G'day!

Ran 11 tests.

FAILED (failures=2)
Makefile:25: recipe for target 'container-check' failed
make: *** [container-check] Error 1
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Publish jenkins/evergreen)
Stage 'Publish jenkins/evergreen' skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline

GitHub has been notified of this commit’s build result

ERROR: script returned exit code 2
Finished: FAILURE

Powered by the Comment Logger

@batmat batmat removed the work-in-progress label Apr 5, 2018

@batmat batmat requested a review from rtyler Apr 5, 2018

@jenkinsadmin

This comment has been minimized.

Copy link

commented Apr 5, 2018

Build failed; the context from the latest run is:

Expand to view
test_metrics_health_check
+ exec docker run --rm -w /home/jenkins/workspace/Infra_evergreen_PR-44-ZIKSHT5HJI6X2WTRNCL37EPUQ6JLCZZ7FKWALNCL236SRCBKNFJQ -v /home/jenkins/workspace/Infra_evergreen_PR-44-ZIKSHT5HJI6X2WTRNCL37EPUQ6JLCZZ7FKWALNCL236SRCBKNFJQ:/home/jenkins/workspace/Infra_evergreen_PR-44-ZIKSHT5HJI6X2WTRNCL37EPUQ6JLCZZ7FKWALNCL236SRCBKNFJQ -i sahsu/docker-jsonlint jsonlint
Unable to find image 'realguess/jq:1.4' locally
1.4: Pulling from realguess/jq
a3ed95caeb02: Pulling fs layer
4317f356dab4: Pulling fs layer
5ba1f428cec5: Pulling fs layer
6e6f0848c9c7: Pulling fs layer
6e6f0848c9c7: Waiting
a3ed95caeb02: Download complete
5ba1f428cec5: Verifying Checksum
5ba1f428cec5: Download complete
6e6f0848c9c7: Verifying Checksum
6e6f0848c9c7: Download complete
a3ed95caeb02: Pull complete
4317f356dab4: Verifying Checksum
4317f356dab4: Download complete
4317f356dab4: Pull complete
5ba1f428cec5: Pull complete
6e6f0848c9c7: Pull complete
Digest: sha256:e0a4ef30634c982ef395a1a4bdf221df06b3def015f79536fa2d658d5bce9c88
Status: Downloaded newer image for realguess/jq:1.4

Cleaning up... G'day!

Ran 11 tests.

FAILED (failures=2)
Makefile:25: recipe for target 'container-check' failed
make: *** [container-check] Error 1
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Publish jenkins/evergreen)
Stage 'Publish jenkins/evergreen' skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline

GitHub has been notified of this commit’s build result

ERROR: script returned exit code 2
Finished: FAILURE

Powered by the Comment Logger

@batmat

This comment has been minimized.

Copy link
Collaborator Author

commented Apr 5, 2018

Interesting, the last commit seems to have revealed a race condition because I don't think I really changed the logic.
I think the test just ran a wee bit faster previously and ran before the healther-checker.log log file was created (incorrectly) under $JENKINS_HOME/logs.

Need to fix/adapt metrics-plugin to https://issues.jenkins-ci.org/browse/JENKINS-50291

@@ -0,0 +1,15 @@
<?xml version='1.1' encoding='UTF-8'?>

This comment has been minimized.

Copy link
@rtyler

rtyler Apr 5, 2018

Member

Perhaps it would be a good time to add a new directory for Jenkins configuration files such as this, the log properties, yaml files, etc

This comment has been minimized.

Copy link
@batmat

batmat Apr 5, 2018

Author Collaborator

Yes. Will give it some thought.

This comment has been minimized.

Copy link
@batmat

batmat Apr 9, 2018

Author Collaborator

I will file it as a separate PR, I have it ready locally.

FIXME: seems like the "origins" field cannot be saved using the config UI, bug?
-->
<accessKeys>
<jenkins.metrics.api.MetricsAccessKey>

This comment has been minimized.

Copy link
@rtyler

rtyler Apr 5, 2018

Member

Are these settings not configurable via the Configuration as Code plugin?

If not, I believe a ticket should be filed for the Metrics plugin, or for the Configuration as Code plugin, to add support.

This comment has been minimized.

Copy link
@batmat

batmat Apr 5, 2018

Author Collaborator

Agreed.

This comment has been minimized.

Copy link
@batmat

This comment has been minimized.

Copy link
@batmat

batmat Apr 9, 2018

Author Collaborator

@rtyler how do you feel we should move forward here: should we either:

I for one would prefer the latter to be able to move forward the whole health-checking JIRA and fix it underneath when we have the way to use CasC-based config.

WDYT?

This comment has been minimized.

Copy link
@rtyler

rtyler Apr 9, 2018

Member

Yeah, the latter, please add a reference to the tickets as a comment in the XML when you move it to configuration/

This comment has been minimized.

Copy link
@batmat

batmat Apr 9, 2018

Author Collaborator

Done @rtyler, in d513f91

@batmat batmat force-pushed the batmat:JENKINS-50294-prototype-metrics-health-checking branch from af75e8e to fe866ff Apr 6, 2018

@batmat batmat force-pushed the batmat:JENKINS-50294-prototype-metrics-health-checking branch from fe866ff to 63bffbe Apr 9, 2018

@batmat batmat removed the work-in-progress label Apr 9, 2018

@batmat batmat requested a review from rtyler Apr 9, 2018

@rtyler
rtyler approved these changes Apr 9, 2018
@batmat

This comment has been minimized.

Copy link
Collaborator Author

commented Apr 9, 2018

Will merge when the PR build succeeds. Or the first who sees this :P

@batmat batmat merged commit 50205f6 into jenkins-infra:master Apr 9, 2018

1 check passed

continuous-integration/jenkins/pr-merge This commit looks good
Details

@batmat batmat deleted the batmat:JENKINS-50294-prototype-metrics-health-checking branch Apr 9, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
You can’t perform that action at this time.