Adding CA cert functionality to elastic #982

bmacnee · 2021-07-02T18:53:18Z

Attempting to add functionality similar to other elastic-apm agents where a ca cert can be provided instead of the literal server cert.

I am a bit confused by the comment TestHTTPTransportServerCert :

// Reconfigure the transport so that it knows about the
// CA certificate. We avoid using server.Client here, as
// it is not available in older versions of Go.

This refers to a CA certificate, but my understanding is this functionality would only work for the servers cert.

cla-checker-service · 2021-07-02T18:53:22Z

💚 CLA has been signed

apmmachine · 2021-07-02T18:56:44Z

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2021-07-07T01:26:32.979+0000
Duration: 10 min 5 sec
Commit: 146b3b8

Test stats 🧪

Test	Results
Failed	0
Passed	7286
Skipped	213
Total	7499

Trends 🧪

Steps errors

Expand to view the steps failures

`Build`

Took 2 min 47 sec . View more details on here
Description: ./scripts/jenkins/build.sh

`Build`

Took 0 min 2 sec . View more details on here
Description: ./scripts/jenkins/build.sh

`Build`

Took 0 min 2 sec . View more details on here
Description: ./scripts/jenkins/build.sh

`Build`

Took 2 min 12 sec . View more details on here
Description: ./scripts/jenkins/build.sh

`Build`

Took 0 min 2 sec . View more details on here
Description: ./scripts/jenkins/build.sh

`Build`

Took 0 min 2 sec . View more details on here
Description: ./scripts/jenkins/build.sh

`Error signal`

Took 0 min 0 sec . View more details on here
Description: hudson.AbortException: script returned exit code 2

Log output

Expand to view the last 100 lines of log output

[2021-07-07T01:36:34.712Z] --- PASS: TestHandlerHTTPSuite (0.01s)
[2021-07-07T01:36:34.712Z]     --- PASS: TestHandlerHTTPSuite/TestExplicitErrorCapture (0.00s)
[2021-07-07T01:36:34.712Z]     --- PASS: TestHandlerHTTPSuite/TestImplicitWrite (0.00s)
[2021-07-07T01:36:34.712Z]     --- PASS: TestHandlerHTTPSuite/TestPanicAfterWrite (0.00s)
[2021-07-07T01:36:34.712Z]     --- PASS: TestHandlerHTTPSuite/TestPanicBeforeWrite (0.00s)
[2021-07-07T01:36:34.712Z] === RUN   TestHandler
[2021-07-07T01:36:34.712Z] --- PASS: TestHandler (0.00s)
[2021-07-07T01:36:34.712Z] === RUN   TestHandlerOutcome
[2021-07-07T01:36:34.712Z] --- PASS: TestHandlerOutcome (0.01s)
[2021-07-07T01:36:34.712Z] === RUN   TestHandlerCaptureBodyRaw
[2021-07-07T01:36:34.712Z] --- PASS: TestHandlerCaptureBodyRaw (0.00s)
[2021-07-07T01:36:34.712Z] === RUN   TestHandlerCaptureBodyConcurrency
[2021-07-07T01:36:34.712Z] --- PASS: TestHandlerCaptureBodyConcurrency (0.12s)
[2021-07-07T01:36:34.712Z] === RUN   TestHandlerCaptureBodyRawTruncated
[2021-07-07T01:36:34.712Z] --- PASS: TestHandlerCaptureBodyRawTruncated (0.01s)
[2021-07-07T01:36:34.712Z] === RUN   TestHandlerCaptureBodyForm
[2021-07-07T01:36:34.712Z] --- PASS: TestHandlerCaptureBodyForm (0.00s)
[2021-07-07T01:36:34.712Z] === RUN   TestHandlerCaptureBodyError
[2021-07-07T01:36:34.712Z] --- PASS: TestHandlerCaptureBodyError (0.01s)
[2021-07-07T01:36:34.712Z] === RUN   TestHandlerCaptureBodyErrorIgnored
[2021-07-07T01:36:34.712Z] --- PASS: TestHandlerCaptureBodyErrorIgnored (0.01s)
[2021-07-07T01:36:34.712Z] === RUN   TestHandlerRecovery
[2021-07-07T01:36:34.712Z] --- PASS: TestHandlerRecovery (0.00s)
[2021-07-07T01:36:34.712Z] === RUN   TestHandlerRecoveryNoHeaders
[2021-07-07T01:36:34.712Z] --- PASS: TestHandlerRecoveryNoHeaders (0.00s)
[2021-07-07T01:36:34.712Z] === RUN   TestHandlerWithPanicPropagation
[2021-07-07T01:36:34.712Z] --- PASS: TestHandlerWithPanicPropagation (0.01s)
[2021-07-07T01:36:34.712Z] === RUN   TestHandlerWithPanicPropagationResponseCodeForwarding
[2021-07-07T01:36:34.712Z] --- PASS: TestHandlerWithPanicPropagationResponseCodeForwarding (0.00s)
[2021-07-07T01:36:34.712Z] === RUN   TestHandlerRequestIgnorer
[2021-07-07T01:36:34.712Z] --- PASS: TestHandlerRequestIgnorer (0.01s)
[2021-07-07T01:36:34.712Z] === RUN   TestHandlerTraceparentHeader
[2021-07-07T01:36:34.712Z] --- PASS: TestHandlerTraceparentHeader (0.00s)
[2021-07-07T01:36:34.712Z] === RUN   TestHandlerTracestateHeader
[2021-07-07T01:36:34.712Z] --- PASS: TestHandlerTracestateHeader (0.00s)
[2021-07-07T01:36:34.712Z] === RUN   TestHandlerReaderFrom
[2021-07-07T01:36:34.712Z] --- PASS: TestHandlerReaderFrom (0.01s)
[2021-07-07T01:36:34.712Z] === RUN   TestServerRequestIgnorer
[2021-07-07T01:36:34.712Z] === RUN   TestServerRequestIgnorer/_/foo
[2021-07-07T01:36:34.712Z] === RUN   TestServerRequestIgnorer/_/foo?bar=baz
[2021-07-07T01:36:34.712Z] === RUN   TestServerRequestIgnorer/_http://testing.invalid/foo?bar=baz
[2021-07-07T01:36:34.973Z] === RUN   TestServerRequestIgnorer/,_/foo
[2021-07-07T01:36:34.973Z] === RUN   TestServerRequestIgnorer/*/foo*_/foo
[2021-07-07T01:36:34.973Z] === RUN   TestServerRequestIgnorer/*/foo*_/foo?bar=baz
[2021-07-07T01:36:34.973Z] === RUN   TestServerRequestIgnorer/*/foo*_http://testing.invalid/foo?bar=baz
[2021-07-07T01:36:34.973Z] === RUN   TestServerRequestIgnorer/*/FOO*_http://testing.invalid/foo?bar=baz
[2021-07-07T01:36:34.973Z] === RUN   TestServerRequestIgnorer/*/foo?bar=baz_/foo
[2021-07-07T01:36:34.973Z] === RUN   TestServerRequestIgnorer/*/foo?bar=baz_/foo?bar=baz
[2021-07-07T01:36:34.973Z] === RUN   TestServerRequestIgnorer/*/foo?bar=baz_http://testing.invalid/foo?bar=baz
[2021-07-07T01:36:34.973Z] === RUN   TestServerRequestIgnorer/http://*_/foo
[2021-07-07T01:36:34.973Z] === RUN   TestServerRequestIgnorer/http://*_/foo?bar=baz
[2021-07-07T01:36:34.973Z] === RUN   TestServerRequestIgnorer/http://*_http://testing.invalid/foo?bar=baz
[2021-07-07T01:36:34.973Z] --- PASS: TestServerRequestIgnorer (0.25s)
[2021-07-07T01:36:34.973Z]     --- PASS: TestServerRequestIgnorer/_/foo (0.02s)
[2021-07-07T01:36:34.973Z]     --- PASS: TestServerRequestIgnorer/_/foo?bar=baz (0.02s)
[2021-07-07T01:36:34.973Z]     --- PASS: TestServerRequestIgnorer/_http://testing.invalid/foo?bar=baz (0.02s)
[2021-07-07T01:36:34.973Z]     --- PASS: TestServerRequestIgnorer/,_/foo (0.02s)
[2021-07-07T01:36:34.973Z]     --- PASS: TestServerRequestIgnorer/*/foo*_/foo (0.02s)
[2021-07-07T01:36:34.973Z]     --- PASS: TestServerRequestIgnorer/*/foo*_/foo?bar=baz (0.02s)
[2021-07-07T01:36:34.973Z]     --- PASS: TestServerRequestIgnorer/*/foo*_http://testing.invalid/foo?bar=baz (0.02s)
[2021-07-07T01:36:34.973Z]     --- PASS: TestServerRequestIgnorer/*/FOO*_http://testing.invalid/foo?bar=baz (0.02s)
[2021-07-07T01:36:34.973Z]     --- PASS: TestServerRequestIgnorer/*/foo?bar=baz_/foo (0.02s)
[2021-07-07T01:36:34.973Z]     --- PASS: TestServerRequestIgnorer/*/foo?bar=baz_/foo?bar=baz (0.02s)
[2021-07-07T01:36:34.973Z]     --- PASS: TestServerRequestIgnorer/*/foo?bar=baz_http://testing.invalid/foo?bar=baz (0.02s)
[2021-07-07T01:36:34.973Z]     --- PASS: TestServerRequestIgnorer/http://*_/foo (0.02s)
[2021-07-07T01:36:34.973Z]     --- PASS: TestServerRequestIgnorer/http://*_/foo?bar=baz (0.02s)
[2021-07-07T01:36:34.973Z]     --- PASS: TestServerRequestIgnorer/http://*_http://testing.invalid/foo?bar=baz (0.02s)
[2021-07-07T01:36:34.973Z] === RUN   TestFallbackDeprecatedRequestIgnorer
[2021-07-07T01:36:34.973Z] --- PASS: TestFallbackDeprecatedRequestIgnorer (0.01s)
[2021-07-07T01:36:34.973Z] === RUN   TestParseTraceparentHeader
[2021-07-07T01:36:34.973Z] --- PASS: TestParseTraceparentHeader (0.00s)
[2021-07-07T01:36:34.973Z] === RUN   TestParseTracestateHeader
[2021-07-07T01:36:34.973Z] --- PASS: TestParseTracestateHeader (0.00s)
[2021-07-07T01:36:34.973Z] === RUN   ExampleWrapClient
[2021-07-07T01:36:35.233Z] --- PASS: ExampleWrapClient (0.26s)
[2021-07-07T01:36:35.233Z] PASS
[2021-07-07T01:36:35.233Z] ok  	go.elastic.co/apm/module/apmhttp	1.323s
[2021-07-07T01:36:35.233Z] + status=0
[2021-07-07T01:36:35.233Z] + go-junit-report
[2021-07-07T01:36:35.233Z] + exit 0
[2021-07-07T01:36:35.292Z] Recording test results
[2021-07-07T01:36:35.899Z] [Checks API] No suitable checks publisher found.
[2021-07-07T01:36:36.014Z] Terminated
[2021-07-07T01:36:37.025Z] Stage "Coverage" skipped due to earlier failure(s)
[2021-07-07T01:36:37.054Z] Stage "Benchmark" skipped due to earlier failure(s)
[2021-07-07T01:36:37.122Z] Stage "More OS" skipped due to earlier failure(s)
[2021-07-07T01:36:37.154Z] Stage "Windows" skipped due to earlier failure(s)
[2021-07-07T01:36:37.155Z] Stage "OSX" skipped due to earlier failure(s)
[2021-07-07T01:36:37.180Z] Failed in branch Windows
[2021-07-07T01:36:37.181Z] Failed in branch OSX
[2021-07-07T01:36:37.227Z] Stage "Integration Tests" skipped due to earlier failure(s)
[2021-07-07T01:36:37.255Z] Stage "Release" skipped due to earlier failure(s)
[2021-07-07T01:36:37.270Z] Stage "Release" skipped due to earlier failure(s)
[2021-07-07T01:36:37.299Z] Stage "Release" skipped due to earlier failure(s)
[2021-07-07T01:36:37.557Z] Running on Jenkins in /var/lib/jenkins/workspace/agent-go_apm-agent-go-mbp_PR-982
[2021-07-07T01:36:37.645Z] [INFO] getVaultSecret: Getting secrets
[2021-07-07T01:36:37.695Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-07-07T01:36:38.345Z] + chmod 755 generate-build-data.sh
[2021-07-07T01:36:38.345Z] + ./generate-build-data.sh https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-go/apm-agent-go-mbp/PR-982/ https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-go/apm-agent-go-mbp/PR-982/runs/8 FAILURE 605106
[2021-07-07T01:36:38.345Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-go/apm-agent-go-mbp/PR-982/runs/8/steps/?limit=10000 -o steps-info.json

bmacnee · 2021-07-02T19:23:36Z

signed the Agreement after the fact, dunno if there's a way to kick off the githooks again...

axw

This is great, thanks for your contribution! Just a few small comments, otherwise LGTM.

This refers to a CA certificate, but my understanding is this functionality would only work for the servers cert.

Yes indeed -- sorry, the comment is wrong 😅
While you're in the vicinity, would you mind replacing "CA" with "server" in that comment?

transport/http.go

transport/http_test.go

bmacnee · 2021-07-06T15:58:11Z

@axw curious on a ballpark estimate for how long it will take you folks to release the agent with these changes, as the work I'm doing is dependent on them. Thank you!! Worst case I can use my branch if it will take some time.

axw

@axw curious on a ballpark estimate for how long it will take you folks to release the agent with these changes, as the work I'm doing is dependent on them. Thank you!! Worst case I can use my branch if it will take some time.

@bmacnee there's nothing specifically scheduled, but I anticipate we'll cut a new release in 3-4 weeks.

axw · 2021-07-07T01:16:07Z

jenkins run the tests please

axw · 2021-07-07T02:36:32Z

Go 1.8/1.9 failures are unrelated, will be fixed by #984 -- merging.

Thanks again!

need to add unit tests

1e541e9

github-actions bot added the agent-go label Jul 2, 2021

adding unit test

717c652

bmacnee marked this pull request as ready for review July 2, 2021 19:15

axw requested changes Jul 6, 2021

View reviewed changes

transport/http.go Outdated Show resolved Hide resolved

transport/http.go Outdated Show resolved Hide resolved

transport/http_test.go Outdated Show resolved Hide resolved

addressing PR review and adding tests

ab73c06

bmacnee requested a review from axw July 6, 2021 15:50

adding another assert in tests

4555e20

simplifying tests

088913e

axw approved these changes Jul 7, 2021

View reviewed changes

Merge branch 'master' into ca-certs

a0dc690

make fmt

146b3b8

axw merged commit 5963902 into elastic:master Jul 7, 2021

stuartnelson3 pushed a commit that referenced this pull request Jul 30, 2021

Adding CA cert functionality to elastic (#982)

365b51f

axw mentioned this pull request Aug 9, 2021

docs: document ELASTIC_APM_SERVER_CA_CERT_FILE #1001

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding CA cert functionality to elastic #982

Adding CA cert functionality to elastic #982

bmacnee commented Jul 2, 2021 •

edited

Loading

cla-checker-service bot commented Jul 2, 2021 •

edited

Loading

apmmachine commented Jul 2, 2021 •

edited

Loading

Build stats

Test stats 🧪

Trends 🧪

`Build`

`Build`

`Build`

`Build`

`Build`

`Build`

`Error signal`

bmacnee commented Jul 2, 2021

axw left a comment

bmacnee commented Jul 6, 2021

axw left a comment

axw commented Jul 7, 2021

axw commented Jul 7, 2021

Adding CA cert functionality to elastic #982

Adding CA cert functionality to elastic #982

Conversation

bmacnee commented Jul 2, 2021 • edited Loading

cla-checker-service bot commented Jul 2, 2021 • edited Loading

apmmachine commented Jul 2, 2021 • edited Loading

💔 Build Failed

Build stats

Test stats 🧪

Trends 🧪

Steps errors

Build

Build

Build

Build

Build

Build

Error signal

Log output

bmacnee commented Jul 2, 2021

axw left a comment

Choose a reason for hiding this comment

bmacnee commented Jul 6, 2021

axw left a comment

Choose a reason for hiding this comment

axw commented Jul 7, 2021

axw commented Jul 7, 2021

bmacnee commented Jul 2, 2021 •

edited

Loading

cla-checker-service bot commented Jul 2, 2021 •

edited

Loading

apmmachine commented Jul 2, 2021 •

edited

Loading

`Build`

`Build`

`Build`

`Build`

`Build`

`Build`

`Error signal`