implemented event "renewing" (#233)

[root360-AndreasUlm]

Signed-off-by: Andreas Ulm andreas.ulm@root360.de

[morph027]

Just noticed that the staging area does not get cleaned afterwards.

cat <mod_md_dir>/staging/example.com/job.json
{
  "name": "example.com",
  "finished": false,
  "notified": false,
  "errors": 0,
  "last": {
    "status": 0
  },
  "log": {
    "entries": [
      {
        "when": "Tue, 19 Jan 2021 09:50:41 GMT",
        "type": "message-renewing"
      }
    ]
  }
}

[root360-AndreasUlm]

@icing should I implement cleanup of staging area when renewing event-command fails?

[icing]

(Planning to merge this today.)

@root360-AndreasUlm I guess cleaning staging would be counter-productive as another cluster node is most likely using it?

But it touches a point: if a cluster node restarts, it might reset an existing staging. This needs then also be prevented somehow...

[daum3ns]

hello
first of all: thanks for all that work on mod_md!

im not sure if i understand the whole discussion correct, but would this new event interface (especially the "renewing" event) allow me to distribute the temporary certificate used for tls-alpn/1 challenge among several nodes?

the problem is, we have several instances behind a load balancer. because we cannot ensure that the load balancer forwards the request to the one host which initially asked for cert_renewal, the idea is to distribute the challenge, so that each host can answer this challenge.. could this be achieved by this latest commits, i.e does the script get this info on the "renewal" event?

after that, the certificate can be distributed to all nodes in the cluster when one triggers "installed"...

thanks a lot for clarification!

[root360-AndreasUlm]

@daum3ns
The initial idea behind this event is to create a script that checks several parameters to decide whether the current node should run the renewal process or not.

We created a setup with mod_md that stores the whole MDStoreDir on a NFS shared between all nodes behind the loadbalancer.
As Apache copies the certificates into memory on start only the starting process is slower than normal but we haven't found any further impact with this setup.
By using this event our renewal process looks like this:

• node1 starts renewal process & creates folder structure in staging including md.json
• on all other nodes the script executed by "renewing"-event exits with non-zero as the md.json file exists
• let's encrypt requests the challenge
• as all nodes have access to the same MDStoreDir everyone can respond correctly to the challenge
• after successful creation node1 moves the new certificate files to the target destination
• then the apache is reloaded on all nodes via our configuration management system

Maybe you can use this setup description for your systems too.

[icing]

The shared directory approach makes things easier, but that may not be common on clusters.

We can add another "event" (call to MDMessageCmd) when challenges have been created. The only caveat here is that all invocations during the renewal process are made as "www-data" (or whatever user your server process is running on). So, if this is not an obstacle, we can certainly add more invocations.

[daum3ns]

thanks for your answers!
no, shared directory via NFS is not an option for our case.
the main question is:
after your step 1: "node1 starts renewal process & creates folder structure in staging including md.json"
would the challenges be ready, so that the script triggered with "renewing" can distribute it with rsync to the other nodes. and if so, is there enough time between the "renewing" and the challenge request from lets_encrypt to do so...

or even more extrem: could the challenge for the next renewal already be prepared immediately after a cert renewal.. like this we would have loads of time to distribute it among the nodes...

sorry if this question is entirely stupid, but i dont understand the module 100% atm...
thanks so much!

[root360-AndreasUlm]

No the "renewing" event is triggered before the challenge is created thus it would require an additional new event.
As the events are blocking the scripts started can control how long to wait.

@daum3ns can create a new issue describing your intention and use-case the we can discuss the implementation there.

[icing]

We follow up the discussion on the new ticket. @root360-AndreasUlm are we fine with closing this issue, now that the feature has been implemented?

[root360-AndreasUlm]

Yes, we can close this issue now. Thanks.