Add --reuse-key feature #5901

schoen · 2018-04-26T00:18:17Z

Intended to supersede #4610 (with a different syntax per discussion with @ohemorange ).

Fixes #3788; fixes #3709.

jschlyter · 2018-04-26T06:48:46Z

Not tested, but feature-wise I agree this is a better solution than #4610.

schoen · 2018-05-03T00:36:37Z

This appears to be working well now and I tested it quite a bit.

I had a red herring because when I renewed with certbot certonly --force-renewal (without explicit --reuse-key) I lost the --reuse-key setting from the renewal configuration. By contrast, certbot renew worked as expected. This is a behavior that @bmw has discussed a number of times and that I've even sometimes helped people out on the forum with.

Anyway, issuing with --reuse-key and then renewing with certbot renew currently works as expected in terms of using the same private key for the renewal.

The only remaining problem that I know about is that I don't have test coverage for all four combinations of --dry-run --reuse-key, --dry-run --no-reuse-key, --reuse-key, and --no-reuse-key (I think effectively three of them are covered by the current unit tests here). I tried all of them manually and they did the right thing, but maybe they should all be covered explicitly in unit tests.

Fixes #3788 and #3709.

schoen · 2018-05-03T00:41:20Z

I tried all of them manually and they did the right thing, but maybe they should all be covered explicitly in unit tests.

Per discussion with @ohemorange, I'm going to add one more unit test related to this.

schoen · 2018-05-04T21:42:29Z

This is ready for review!

bmw · 2018-05-04T23:25:57Z

@ohemorange, can you add this to your review queue?

ohemorange · 2018-05-04T23:31:30Z

already on it!

innovia · 2018-05-07T07:06:29Z

awesome work thank you - i'm using this branch

ohemorange

Added some suggestions to increase pythonicness and such

ohemorange · 2018-05-11T00:18:05Z

certbot/client.py

@@ -266,7 +266,7 @@ def obtain_certificate_from_csr(self, csr, orderr=None):
        cert, chain = crypto_util.cert_and_chain_from_fullchain(orderr.fullchain_pem)
        return cert.encode(), chain.encode()

-    def obtain_certificate(self, domains):
+    def obtain_certificate(self, domains, oldkeypath=None):


old_key_path or old_keypath for pythonicness?

ohemorange · 2018-05-11T00:19:06Z

certbot/client.py

        # Create CSR from names
        if self.config.dry_run:
-            key = util.Key(file=None,
-                           pem=crypto_util.make_key(self.config.rsa_key_size))
+            if oldkeypath is None:


key = key or ...

ohemorange · 2018-05-11T00:19:27Z

certbot/client.py

            csr = util.CSR(file=None, form="pem",
                           data=acme_crypto_util.make_csr(
                               key.pem, domains, self.config.must_staple))
        else:
-            key = crypto_util.init_save_key(
-                self.config.rsa_key_size, self.config.key_dir)
+            if oldkeypath is None:


key = key or ...

ohemorange · 2018-05-11T00:20:22Z

certbot/client.py

+                keypath = oldkeypath
+                keypem = f.read()
+            key = util.Key(file=keypath, pem=keypem)
+            logger.info("New key obtained by reusing existing private key "


Perhaps clearer to drop the "new key obtained by"

I didn't think so because it does create a new file, but maybe users will be able to figure it out!

ohemorange · 2018-05-11T00:21:17Z

certbot/client.py

+            key = util.Key(file=keypath, pem=keypem)
+            logger.info("New key obtained by reusing existing private key "
+                        "from %s.", oldkeypath)
+


if you stick an else: key = None here, the rest gets a little cleaner

ohemorange · 2018-05-11T00:22:04Z

certbot/cli.py

+             "certificate.")
+    helpful.add(
+        "automation", "--no-reuse-key", dest="reuse_key",
+        action="store_false", default=flag_default("reuse_key"),


did we ever figure out why this works? as in where the magic code lives. just out of curiosity

Nope, I'm not really sure!

I'm not sure if this exactly answers your question, but we have a number of flags like --foo and --no-foo. For example, take a look at --redirect.

--redirect sets redirect on the config object to True while --no-redirect sets it to False. Both --redirect and --no-redirect share the same default which isn't True or False, but None. This allows us to differentiate between the user saying yes or no to a redirect and not answering the question.

Do we want to do anything special in the case where the user doesn't provide either --reuse-key or --no-reuse-key? If not, I think we can probably get rid of --no-reuse-key and keep the default reuse_key value to False. If we do want this for some reason, we should probably change the default value from False to None.

faaascinating.

the only reason I could think of to have --no-reuse-key would be to override the saved setting in the renewal config file, but users can just delete that line.

otherwise, I'm for just having --reuse-key and having the default be False.

After talking to @bmw, I more clearly understand the point that for options that don't relate to prompts, we don't need a separate --no version because simply leaving the option out when reissuing with certonly is already sufficient to set it to False.

ohemorange · 2018-05-11T00:24:28Z

certbot/renewal.py

@@ -295,7 +295,13 @@ def renew_cert(config, domains, le_client, lineage):
    _avoid_invalidating_lineage(config, lineage, original_server)
    if not domains:
        domains = lineage.names()
-    new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains)
+    if config.reuse_key:


old_key = lineage.privkey if config.reuse_key else None new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, old_key)

ohemorange · 2018-05-11T00:28:37Z

certbot/tests/main_test.py

-        # pylint: disable=too-many-locals,too-many-arguments
+                             quiet_mode=False, expiry_date=datetime.datetime.now(),
+                             reuse_key=False):
+        # pylint: disable=too-many-locals,too-many-arguments,too-many-branches


Is there a way to add the tests that doesn't stick the checks into _test_renewal_common? We already overload this with too much stuff.

I agree, but I think it might look even more complex by defining a whole new set of ways to test renewals!

ohemorange · 2018-05-11T00:34:18Z

tests/boulder-integration.sh

+ls -l "${root}/conf/archive/reusekey.le.wtf/privkey"*
+# The final awk command here exits successfully if its input consists of
+# exactly two lines with identical first fields, and unsuccessfully otherwise.
+sha256sum "${root}/conf/archive/reusekey.le.wtf/privkey"* | awk '{a[$1] = 1}; END {exit(NR !=2 || length(a)!=1)}'


let's stick this awk command into a named parametrized function

ArchangeGabriel · 2018-07-04T09:47:45Z

Maybe I’m missing something, but I don’t understand when/how old_keypath is set.

ohemorange · 2018-07-05T19:42:26Z

It's passed in as an argument to obtain_certificate when called from renewal.py. Are you working on a PR that you're looking for help with?

ArchangeGabriel · 2018-07-05T21:00:21Z

No, I’m trying to understand how to get certbot to reuse the same key for each request on my server.

ArchangeGabriel · 2018-07-05T21:00:44Z

(For existing lineage as well as new ones)

pgporada · 2020-04-11T23:42:37Z

Seth,

This just came in extremely handy to test a system we've been working on. Thank you.

Initial work on new version of --reuse-key

773602e

schoen mentioned this pull request Apr 26, 2018

Allow reuse of private keys #4610

Closed

Seth Schoen added 3 commits May 2, 2018 16:00

Merge remote-tracking branch 'origin/master' into reuse-key

6bfdfa8

Test for reuse_key

40f2891

Make lint happier

87e6fa2

schoen changed the title ~~WIP for --reuse-key feature~~ Add --reuse-key feature May 3, 2018

Seth Schoen added 2 commits May 3, 2018 15:06

Merge remote-tracking branch 'origin/master' into reuse-key

d635a44

Also test a non-dry-run reuse_key renewal

8596f75

Seth Schoen added 5 commits May 8, 2018 12:55

Test --reuse-key in boulder integration test

f06c883

Merge remote-tracking branch 'origin/master' into reuse-key

06c2671

Better reuse-key integration testing

27aa293

Log fact that key was reused

a98def9

Test that the certificates themselves are different

9407edf

schoen mentioned this pull request May 9, 2018

Use or remove old reuse-key-related code in storage.py #5942

Closed

ohemorange requested changes May 11, 2018

View reviewed changes

Seth Schoen added 6 commits May 11, 2018 22:15

Merge remote-tracking branch 'origin/master' into reuse-key

bc21655

Change "oldkeypath" to "old_keypath"

e0ed77d

Simply appearance of new-key generation logic

05bad34

Reorganize new-key logic

951961f

Move awk logic into TotalAndDistinctLines function

67446a2

After refactor, there's now explicit None rather than missing param

03f3317

ohemorange added the current sprint label May 23, 2018

bmw assigned ohemorange May 23, 2018

ohemorange assigned schoen and unassigned ohemorange May 23, 2018

Seth Schoen added 7 commits May 25, 2018 14:18

Merge remote-tracking branch 'origin/master' into reuse-key

74fc34b

Merge remote-tracking branch 'origin/master' into reuse-key

c21f76e

Indicate for MyPy that key can be None

3017c8f

Actually import the Optional type

ad13662

magic_typing is too magical for pylint

8ab775d

Remove --no-reuse-key option

db855c9

Correct pylint test disable

6beb3df

ohemorange approved these changes Jun 1, 2018

View reviewed changes

ohemorange merged commit e2d6faa into master Jun 1, 2018

ohemorange deleted the reuse-key branch June 1, 2018 22:21

bmw added this to the 0.25.0 milestone Jun 6, 2018

bmw removed the current sprint label Jun 27, 2018

kroeckx mentioned this pull request Nov 2, 2018

create (or output) DANE records #230

Open

Add --reuse-key feature #5901

Add --reuse-key feature #5901

schoen commented Apr 26, 2018 •

edited

jschlyter commented Apr 26, 2018

schoen commented May 3, 2018

schoen commented May 3, 2018

schoen commented May 4, 2018

bmw commented May 4, 2018

ohemorange commented May 4, 2018

innovia commented May 7, 2018

ohemorange left a comment

ohemorange May 11, 2018

ohemorange May 11, 2018

ohemorange May 11, 2018

ohemorange May 11, 2018

schoen May 12, 2018

ohemorange May 11, 2018

ohemorange May 11, 2018

schoen May 12, 2018

bmw May 22, 2018

ohemorange May 23, 2018

schoen May 30, 2018

ohemorange May 11, 2018

ohemorange May 11, 2018

schoen May 12, 2018

ohemorange May 11, 2018

ArchangeGabriel commented Jul 4, 2018

ohemorange commented Jul 5, 2018

ArchangeGabriel commented Jul 5, 2018

ArchangeGabriel commented Jul 5, 2018

pgporada commented Apr 11, 2020

Add --reuse-key feature #5901

Add --reuse-key feature #5901

Conversation

schoen commented Apr 26, 2018 • edited

jschlyter commented Apr 26, 2018

schoen commented May 3, 2018

schoen commented May 3, 2018

schoen commented May 4, 2018

bmw commented May 4, 2018

ohemorange commented May 4, 2018

innovia commented May 7, 2018

ohemorange left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ArchangeGabriel commented Jul 4, 2018

ohemorange commented Jul 5, 2018

ArchangeGabriel commented Jul 5, 2018

ArchangeGabriel commented Jul 5, 2018

pgporada commented Apr 11, 2020

schoen commented Apr 26, 2018 •

edited