Manual delay hook #6607
Manual delay hook #6607
Conversation
|
Note that this is just a POC. If there’s interest in adding such a hook, I could fix up a proper implementation. |
|
I think your proposition is very interesting. As a contributor in Lexicon, that backs a lot of Certbot DNS plug-in, and as the maintainer of a Docker that is handling DNS challenges from Certbot with Lexicon, it is not the first time I am confronted to this problematic. Since wildcard certificates are out there, DNS challenges became a biggest part of the challenges that need to be handled. But on the contrary to HTTP challenges, that can be treated as synchronous operations as it is basically put locally a file in a specific path, DNS challenges require most of the time asynchronous calls to Rest APIs, making this process highly asynchronous, and heavily slow if processed in a synchronous way. Your idea is focused on the main need, that is to wait DNS records modifications to be correctly propagated by the DNS provider, before asking the ACME CA server to trigger the challenge verification. I think your asynchronous approach can be generalized. Conceptually, the point is to have four logical blocks that are synchronously executed, but where every operation in a given block can be executed in the order you want, including as massively in parallel. This is first, challenges committing, then challenges propagation validation, then challenges check by ACME CA, and finally challenges cleanup. In theory, what you say about DNS challenges could be also applied to HTTP challenges, even if it is less critical: after the files have been written in the HTTP server, you could want to check first that theses files are effectively served before asking the ACME CA to launch its logic. |
|
So if I recall correctly, currently the provided authenticator hook (and same for clean up hook), is executed in a blocking loop for each domain described in the SANs of the current certificate. To take fully advantage of your delay hook (that I would call more check propagation hook), theses authenticator/cleanup hooks should be executed in a non blocking loop, certainly as processes, and in the middle, the check propagation hook would run, it also as non blocking, to trigger the ACME CA call when all processes returned green. |
|
However in this case, an independent check propagation hook may not be necessary. If we consider that an authenticator hook async process returning green means "challenge is written AND propagation has been checked", then everything can be done in the authenticator hook. This does not allow a specific and global logic to be applied for checking the propagation though. An example I have in my mind, is that for DNS challenges for domains all in the same DNS zone, you do not need to check each individual domain. You can just poll the entire TXT records of the zone, and wait for all expected records to appear in it. |
|
And of course, retro-compatibility needs to be taken care of. What about existing hooks that effectively relies on synchronous operations? Here it would be another argument for a separated class of hooks. Without any check propagation hook specified, the synchronous process is triggered. With it specified, then the asynchronous process is triggered. OK, finished to put my thinking here ^^ |
|
So currently, all auth hooks are executed sequentially, then the DNS is checked by the servers and finally the cleanup hooks can be used to clean up whatever the auth hooks did, if desired. The problem is, of course, that for this to be reliable, each auth hook needs to wait for propagation to complete, which is very very slow (at least a couple of minutes per hook invocation. Changing What this delay hook proposal allows you to do is to skip the propagation check in the auth hook. Just update your zone records and trigger a refresh/reload. This can be done fairly quick, like in a couple of seconds rather than minutes. Note that the auth hook will still execute at least twice (for Then, once all domains have been processed, the delay hook kicks in and waits until all domain propagation is complete. It would be a minor addition to the code and fully backward compatible too. Use your current auth hoops as-is, to split them into a separate auth and delay hook if desired. |
|
Instead of this design, what do people think of an environment variable like The reason I personally prefer this design is it avoids adding a 7th type of hook to Certbot to potentially confuse people on what they should use and while I admit I do not know what a user would want to do with this functionality, it's a little more flexible to know where you are in the list of challenges you're performing than getting called only on the last challenge. Inspiration for this proposal came from #5805. |
|
So if I understand correctly, with that design, every auth hook (and similarly clean up hook) would make non blocking requests to give hand back to certbot immediately, except the last one that could make a blocking loop to check correct DNS propagation? It could work, if a given hook has access to the list of all domains that are challenged, to be able to check DNS propagation for all tokens. |
|
To make sure we're talking about the same kind of blocking, I'm not proposing letting Certbot run hooks in parallel as was discussed earlier in this thread. The change I'm proposing here is quite small. We simply would add a decrementing counter in an environment variable. Each hook would be executed sequentially and Certbot will block on the hook to finish execution before continuing like it does now. As for what the hook itself does though, yes, it wouldn't have to wait separately for each DNS change to propagate and it could only do after making the last one. I suspect most people would just have the script sleep for a long time. That's what all of Certbot's DNS plugins do. Alternatively, they could in theory check each record. By the time |
|
Yes, with your design I had not a parallel execution in mind. Just a sequential execution that is fast, because the hook does not wait for anything except the last one. It can work. I am OK with it. However, even if it not blocking with that design, I think it is beneficial in general to get the list of all domains in an env variable. |
|
I think there is value to a variable like that, but I think most people won't use it, it's another variable to learn about and understand when writing hook scripts, and you can get the same functionality by writing a little code yourself. You OK leaving that out until a request for that feature arises from multiple users? |
|
Adrien and I talked about this out of band and he's OK with my design which I wrote up in #5484 (comment). @LeviticusMB, if you're interested in implementing this, I'll review the PR. |
|
I'm closing this PR because we'd like a slightly different design for this and this PR hasn't seen any response in a little over a month. If you're still interested in implementing this, feel free to open a PR with the new design at #5484 (comment). If you have any questions/concerns about the design, feel free to bring them up in that issue. |
|
@bmw @adferrand I would live A implementation like this will greatly help us. Can you please consider opening this issue? |
|
moving the comment to #5484. |
DNS verification with the manual plugin can be painfully slow if multiple domains or hostnames are to be included in the certificate.
I see that various suggestions have been made to improve this, like #4628, #5484 and #5805.
This PR proposes a new hook that can be used to wait for the DNS propagation. The auth hooks can now just trigger the DNS updates in the background and when all updates have been performed, the delay hooks wait for the updates to propagate before the actual verification continues.