Fix dns value encoding to be compliant with acme v01 spec #1295

r0ro · 2015-12-18T14:32:29Z

"The record provisioned to the DNS is the base64url encoding of this digest"

jsha · 2015-12-18T17:18:34Z

Nice catch, thanks!

jsha · 2015-12-18T18:47:37Z

Actually, I realized this problem should be caught by more than just the integration test. Can you check if there's an appropriate unittest for this code path? If so, please fix it to make sure it catches this bug. If not, could you write one?

Thanks,
Jacob

r0ro · 2015-12-18T19:18:08Z

I think I saw some unit test for dns01 but it was marked as 'coverage' only test since it relies on network data.
I won't be able to work on this before the end of December. When do you plan on updating the staging server ?
I think it's important to merge this before the next update.

Le 18 déc. 2015 à 19:52, Jacob Hoffman-Andrews notifications@github.com a écrit :

Actually, I realized this problem should be caught by more than just the integration test. Can you check if there's an appropriate unittest for this code path? If so, please fix it to make sure it catches this bug. If not, could you write one?

Thanks,
Jacob

—
Reply to this email directly or view it on GitHub.

jmhodges · 2015-12-18T21:39:01Z

Thank for this change! Please write a unit test. It's important for the long health of this project and the DNS challenge itself. Integration tests won't cover all the possibilities now or in the future as the project grows.

On top of that, deploys to production are going to be, at most, very rare and narrow in their purview over the holidays. The DNS challenge is very unlikely to be turned in prod on anytime before January.

But even if that were not the case, it's important for us to take a deliberate, calm path of forward momentum. We're helping encrypt the entire web! A big deal!

Anyway, thanks again for handling this!

jmhodges · 2015-12-18T21:40:35Z

Your schedule, of course, is not ours. We may beat you to having time to fix this with a test in place.

r0ro · 2015-12-28T17:13:56Z

now with some test

You should update the _acme-challenge.good.bin.coffee TXT record to the value "LPsIwTo7o8BoG0-vjCyGQGBWSVIPxI-i_X336eUOQZo" to fix the TestDNSValidationLive. It's the expected value for the expectedToken (https://github.com/letsencrypt/boulder/blob/master/va/validation-authority_test.go#L70)

jmhodges · 2015-12-29T01:50:24Z

LGTM

Thank you!

r0ro · 2015-12-29T01:57:17Z

merged with master

janeczku · 2016-01-01T14:59:44Z

va/validation-authority.go

@@ -425,7 +426,7 @@ func (va *ValidationAuthorityImpl) validateDNS01(identifier core.AcmeIdentifier,
 	// Compute the digest of the key authorization file
 	h := sha256.New()
 	h.Write([]byte(challenge.KeyAuthorization.String()))
-	authorizedKeysDigest := hex.EncodeToString(h.Sum(nil))
+	authorizedKeysDigest := base64.RawURLEncoding.EncodeToString(h.Sum(nil))


base64.RawURLEncoding omits the pad character(s). This should use base64.URLEncoding to be really compliant with the acme spec.

janeczku · 2016-01-01T15:00:50Z

According to acme v01 spec the "(t)he record provisioned to the DNS is the base64url encoding of [the SHA-256 digest]."

"base64url" encoding is specified in RFC 4648 section 5 and is subject to the section 3.2 provisions concerning padding of encoded data:

"Implementations MUST include appropriate pad characters at the end of encoded data unless the specification referring to this document explicitly states otherwise."

The acme 01 specification does not state to omit pad characters in the dns value, accordingly a compliant implementation of the dns-01 challenge has to use padded base64url encoded data when provisioning/validating the TXT record.

Since boulder does a byte-to-byte comparison of the encoded data this is critical!

/cc @r0ro @jmhodges

r0ro · 2016-01-01T21:20:33Z

The spec says:

"For Base64 encoding, we use the variant defined in [RFC7515]. The important features of this encoding are (1) that it uses the URL-safe character set, and (2) that "=" padding characters are stripped."

I think it also applies for dns value

But perhaps the best solution would be to accept both variants ?

Le 1 janv. 2016 à 16:00, Jan B notifications@github.com a écrit :

According to acme v01 spec the "(t)he record provisioned to the DNS is the base64url encoding of [the SHA-256 digest]."

"base64url" encoding is defined in RFC 4648 section 5 and is subject to the section 3.2 provisions concerning padding of encoded data:

"Implementations MUST include appropriate pad characters at the end of encoded data unless the specification referring to this document explicitly states otherwise."

Since the acme 01 specification does not state to omit pad characters in the dns value, a compliant implementation of the dns-01 challenge has to use padded base64url encoded SHA256 digest when provisioning/validating the TXT record.

/cc @r0ro @jmhodges

—
Reply to this email directly or view it on GitHub.

jhass · 2016-01-01T23:11:04Z

+1 for being tolerant to both.

janeczku · 2016-01-01T23:12:14Z

@r0ro I don't see how that provision applies to encoding of key authorization resources. When read in context it specifies the encoding of JSON message data:

"Since JSON is a text-based format, binary fields are Base64-encoded. For Base64 encoding, we use the variant defined in {{RFC7515}}. The important features of this encoding are (1) that it uses the URL-safe character set, and (2) that "=" padding characters are stripped."

But i see how the spec leaves room for interpretation in that regard. And i agree that the safest solution is to accept both forms.

jmhodges · 2016-01-02T00:52:02Z

No, we will only allow the one that lines up with all of the other challenges as implemented.

The spec is in draft and can be clarified and Postel's Law is a quick way to build insecure software.

r0ro · 2016-01-04T15:48:58Z

so the current pull request is ok ?

jsha · 2016-01-04T21:49:14Z

mocks/mocks.go

@@ -37,6 +37,12 @@ func (mock *DNSResolver) LookupTXT(hostname string) ([]string, error) {
 	if hostname == "_acme-challenge.servfail.com" {
 		return nil, fmt.Errorf("SERVFAIL")
 	}
+	if hostname == "_acme-challenge.good-dns01.com" {
+		// base64(sha254("LoqXcYV8q5ONbJQxbmR7SCTNo3tiAXDfowyjxAjEuX0"


jsha · 2016-01-04T21:51:58Z

Basically LGTM. There's a small comment typo, and this needs update to latest master anyhow.

Fix dns value encoding to be compliant with acme v01 spec

r0ro · 2016-01-04T23:22:22Z

Don't forget to update the TXT entry for _acme-challenge.good.bin.coffee to LPsIwTo7o8BoG0-vjCyGQGBWSVIPxI-i_X336eUOQZo

r0ro · 2016-01-21T13:18:51Z

_acme-challenge.good.bin.coffee TXT record is still outdated, I don't know who is able to update its value to match the expected value from validation-authority_test.go

jcjones · 2016-01-21T19:30:57Z

Update is rolling out; sorry I missed this.

Fix dns value encoding according to acme v01 spec

6c3c074

"The record provisioned to the DNS is the base64url encoding of this digest"

r0ro force-pushed the fix-dns-value-enc branch 2 times, most recently from f10d7af to ab31d1c Compare December 18, 2015 15:01

r0ro referenced this pull request Dec 18, 2015

Val -> AuthzKeys in VA

325190e

fix dns integration test to use proper dns value encoding

8fa7b01

r0ro force-pushed the fix-dns-value-enc branch from ab31d1c to 8fa7b01 Compare December 18, 2015 15:10

jsha added the r=jsha label Dec 18, 2015

jsha removed the r=jsha label Dec 18, 2015

jsha mentioned this pull request Dec 19, 2015

dns-01 validation: wrong encoding? #1299

Closed

shred mentioned this pull request Dec 24, 2015

Test DNS challenge shred/acme4j#2

Closed

r0ro and others added 2 commits December 28, 2015 11:40

Merge branch 'master' into fix-dns-value-enc

df718fa

add dns validation test

79a7db4

r0ro force-pushed the fix-dns-value-enc branch from 1ce7298 to 79a7db4 Compare December 28, 2015 17:03

jmhodges added the r=jmhodges label Dec 29, 2015

Merge branch 'master' into fix-dns-value-enc

c8c9846

jmhodges added r=jmhodges and removed r=jmhodges labels Dec 30, 2015

janeczku mentioned this pull request Jan 1, 2016

Base64 encode dns-01 record go-acme/lego#68

Merged

janeczku reviewed Jan 1, 2016
View reviewed changes

alex mentioned this pull request Jan 1, 2016

Update DNS challenge code for the latest version of the ACME spec. certbot/certbot#2052

Closed

janeczku mentioned this pull request Jan 2, 2016

Clarify whether base64 encoding without padding applies to key authorization strings ietf-wg-acme/acme#64

Closed

alex mentioned this pull request Jan 2, 2016

Add dns-01 challenge support to the ACME client certbot/certbot#2061

Merged

Merge branch 'master' into fix-dns-value-enc

9bb3f45

jsha reviewed Jan 4, 2016
View reviewed changes

Merge branch 'master' into fix-dns-value-enc

b21240c

jsha added r=jsha and removed r=jmhodges labels Jan 4, 2016

jmhodges added the r=jmhodges label Jan 4, 2016

jmhodges added a commit that referenced this pull request Jan 4, 2016

Merge pull request #1295 from r0ro/fix-dns-value-enc

b4ebd95

Fix dns value encoding to be compliant with acme v01 spec

jmhodges merged commit b4ebd95 into letsencrypt:master Jan 4, 2016

jhass mentioned this pull request Jan 5, 2016

Update dns-01 record content to comply with ACME spec unixcharles/acme-client#46

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix dns value encoding to be compliant with acme v01 spec #1295

Fix dns value encoding to be compliant with acme v01 spec #1295

r0ro commented Dec 18, 2015

jsha commented Dec 18, 2015

jsha commented Dec 18, 2015

r0ro commented Dec 18, 2015

jmhodges commented Dec 18, 2015

jmhodges commented Dec 18, 2015

r0ro commented Dec 28, 2015

jmhodges commented Dec 29, 2015

r0ro commented Dec 29, 2015

janeczku Jan 1, 2016

janeczku commented Jan 1, 2016

r0ro commented Jan 1, 2016

jhass commented Jan 1, 2016

janeczku commented Jan 1, 2016

jmhodges commented Jan 2, 2016

r0ro commented Jan 4, 2016

jsha Jan 4, 2016

jsha commented Jan 4, 2016

r0ro commented Jan 4, 2016

r0ro commented Jan 21, 2016

jcjones commented Jan 21, 2016

Fix dns value encoding to be compliant with acme v01 spec #1295

Fix dns value encoding to be compliant with acme v01 spec #1295

Conversation

r0ro commented Dec 18, 2015

jsha commented Dec 18, 2015

jsha commented Dec 18, 2015

r0ro commented Dec 18, 2015

jmhodges commented Dec 18, 2015

jmhodges commented Dec 18, 2015

r0ro commented Dec 28, 2015

jmhodges commented Dec 29, 2015

r0ro commented Dec 29, 2015

janeczku Jan 1, 2016

Choose a reason for hiding this comment

janeczku commented Jan 1, 2016

r0ro commented Jan 1, 2016

jhass commented Jan 1, 2016

janeczku commented Jan 1, 2016

jmhodges commented Jan 2, 2016

r0ro commented Jan 4, 2016

jsha Jan 4, 2016

Choose a reason for hiding this comment

jsha commented Jan 4, 2016

r0ro commented Jan 4, 2016

r0ro commented Jan 21, 2016

jcjones commented Jan 21, 2016