Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Difficulty changing LetsEncrypt certificate domains #406

Open
lindhe opened this issue Dec 29, 2017 · 36 comments
Open

Difficulty changing LetsEncrypt certificate domains #406

lindhe opened this issue Dec 29, 2017 · 36 comments
Labels
bug Something isn't working

Comments

@lindhe
Copy link

lindhe commented Dec 29, 2017

Three months ago, I ran sudo nextcloud.enable-https lets-encrypt and went through the prompts and was very conveniently served a signed cert for my domain. But I thought it would renew automatically, so I just let it be. Two days ago it expired.

Yesterday, tried to run sudo nextcloud.enable-https lets-encrypt again, and it seemed to succeed (please see output below). But I was not served with a new cert. I still got the old one. I thought that it maybe was cached, but I've waited more than 12 hours now so I thought I'd get the valid one by now.

First of all I would like to understand how I update the certificate, and secondly I would like to know how I get it to automatically renew. I really thought that lets-encrypt did so by default.

I'm running nextcloud snap on Ubuntu 16.10 server.

May or may not be related to #401.

> sudo nextcloud.enable-https lets-encrypt 
In order for Let's Encrypt to verify that you actually own the
domain(s) for which you're requesting a certificate, there are a
number of requirements of which you need to be aware:

1. In order to register with the Let's Encrypt ACME server, you must
   agree to the currently-in-effect Subscriber Agreement located
   here:

       https://letsencrypt.org/repository/

   By continuing to use this tool you agree to these terms. Please
   cancel now if otherwise.

2. You must have the domain name(s) for which you want certificates
   pointing at the external IP address of this machine.

3. Both ports 80 and 443 on the external IP address of this machine
   must point to this machine (e.g. port forwarding might need to be
   setup on your router).

Have you met these requirements? (y/n) y
Please enter an email address (for urgent notices or key recovery): me@example.com
Please enter your domain name(s) (space-separated): me.example.com example.com
Attempting to obtain certificates... done
Restarting apache... done
@lindhe
Copy link
Author

lindhe commented Dec 29, 2017

Actually, it really should have updated automatically. Running sudo nextcloud.enable-https -h yields

lets-encrypt [OPTIONS]
    Obtain a certificate from Let's Encrypt and automatically keep it
    up-to-date.

@lindhe
Copy link
Author

lindhe commented Dec 29, 2017

Possibly related

I found that the log file /var/snap/nextcloud/current/apache/logs/error_log contained thousands of lines like this:

[Fri Dec 29 09:56:35.233007 2017] [unixd:alert] [pid 4771:tid 139884604061568] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive

@lindhe
Copy link
Author

lindhe commented Dec 29, 2017

renewal config

This is the (censored) content of /var/snap/nextcloud/current/certs/certbot/config/renewal/example.com.conf

# renew_before_expiry = 30 days
version = 0.14.1
archive_dir = /var/snap/nextcloud/current/certs/certbot/config/archive/example.com
cert = /var/snap/nextcloud/current/certs/certbot/config/live/example.com/cert.pem
privkey = /var/snap/nextcloud/current/certs/certbot/config/live/example.com/privkey.pem
chain = /var/snap/nextcloud/current/certs/certbot/config/live/example.com/chain.pem
fullchain = /var/snap/nextcloud/current/certs/certbot/config/live/example.com/fullchain.pem

# Options used in the renewal process
[renewalparams]
account = asdfasdfasdfasdfasdfasdfasdfasdf
work_dir = /var/snap/nextcloud/current/certs/certbot/work
authenticator = nextcloud:webroot
logs_dir = /var/snap/nextcloud/current/certs/certbot/logs
rsa_key_size = 4096
installer = None
config_dir = /var/snap/nextcloud/current/certs/certbot/config
nextcloud:webroot_path = /var/snap/nextcloud/current/certs/certbot

@lindhe
Copy link
Author

lindhe commented Dec 29, 2017

I found the culprit. The logfile /var/snap/nextcloud/current/certs/certbot/logs/letsencrypt.log informed me that certbot exited abnormally because it could not reach two of the domains I had signed.

Previously I had two domains that I used (let's call them {a,b}.example.com). But nowadays I use {c,d}.example.com instead. When I installed Nextcloud last time around, I was in a transition phase so I signed for all four of {a,b,c,d}.example.com. Now when I had removed the DNS records for {a,b}.example.com, certbot would fail since it could not reach my nextcloud server via those domains.

Solution

Make sure that all domains for the cert points to your server. Also, check the logs before asking questions.

I am more or less certain that this is why my cert was not renewed automatically. But I'll have to leave this issue open until DNS has propagated the new records and certbot has had the decency to run again.

I still want to get rid of those domains though, so I'll leave this issue open until I find or someone informs me about how to update the config/cert to not include the deprecated domains.

@kyrofa
Copy link
Member

kyrofa commented Dec 29, 2017

Wow, quick investigation! Thank you for sharing the solution here. Indeed, this sounds exactly correct.

The snap actually makes some pretty bad assumptions if you want to change the domain and get a new cert, so I'm going to leave this issue open to track the fix for that. Until I fix it, I suggest you toast the cert that's currently there and get a new one. Specifically:

First of all, disable HTTPS (this just removes a symlink, it doesn’t remove any certs):

$ sudo nextcloud.disable-https

Then blow away any certs that are there (this includes self-signed certs, Let’s Encrypt certs, everything). Make sure you get this command right, you don’t want to delete anything else in the current/ dir:

$ sudo rm -rf /var/snap/nextcloud/current/certs

Then pretend you’re enabling HTTPS for the first time, using only the domains you want:

$ sudo nextcloud.enable-https lets-encrypt

@kyrofa
Copy link
Member

kyrofa commented Dec 29, 2017

For future reference, you can see helpful logs from the renew-certs service:

$ sudo journalctl -u snap.nextcloud.renew-certs

@lindhe
Copy link
Author

lindhe commented Dec 30, 2017

Ah, great input! Thank you, I will try this.

The snap actually makes some pretty bad assumptions if you want to change the domain and get a new cert, so I'm going to leave this issue open to track the fix for that.

I'll leave the issue open for you to close when you see fit.

@kyrofa kyrofa changed the title Can not renew LetsEncrypt certificate Difficulty changing LetsEncrypt certificate domains Jan 2, 2018
@imatasic
Copy link

Just a bump, how does this correlate with the recent revoke of TLS-SNI validation?
Will there be any problems with it?
I have 60 days to go on my certificate for nextcloud, but am in a process of replacing all of them using http-01 challenge.

@ghost
Copy link

ghost commented Jan 29, 2018

Also curious about the revocation of that validation -- I've been having similar timeouts trying to get a Let's Encrypt cert as the couple other recent reports, been beating my head against a wall for a couple days trying to find something wrong with my router/forwarding/DNS as that seems to be the fix for most people with similar issues, but everything seems fine. Disabling https and connection on port 80 is good, self-signed cert works fine otherwise. I've cleared out the certs and started fresh several times, even removed the snap and reinstalled. Wondering if maybe certbot is configured for the wrong challenge at this point? Does this recent change affect the Let's Encrypt function for the snap?

@imatasic
Copy link

@grantwinship You can find the solution here

@kyrofa
Copy link
Member

kyrofa commented Jan 29, 2018

Wondering if maybe certbot is configured for the wrong challenge at this point? Does this recent change affect the Let's Encrypt function for the snap?

I just tried it on a fresh snap install, and it works for me. This change does not seem to affect the snap.

@ghost
Copy link

ghost commented Jan 29, 2018

yeah, tried testing the version @imatasic linked as well as installing the current version outside the snap and running the webroot style auth and got some similar errors, so back to thinking it must be something with my DNS/router/forwarding situation 🤔 -- thought I had it there for a minute!

I appreciate the confirmation on that @kyrofa , at least I can narrow it down to my setup now!

@ghost
Copy link

ghost commented Jan 30, 2018

ISP blocking the whole time 😭 --- fixed that and worked like a charm. At least I learned a lot of new stuff about networking trying to troubleshoot 😸

@fdemassis
Copy link

Hi I have problem with nextcloud 12 snap 5132 on Ubuntu Server 16.04.3.
I enabled https self signed with hook: "sudo nextcloud.enable-https self signed" but when I try to access https I get an internal error and in /var/snap/nextcloud/current/apache/logs I find this line:

AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive

Solution should be to set User and/or Group directives in apache configuration, probably needed by unixd apache module (chrooting apache), but I do not know how to do this. I tried to create configuration file:
/var/snap/nextcloud/current/apache/custom.conf (from another post), but it did not work....
Never possible via snapctl set because no hook exist for that parameter in apache configuration.
Can someone help me?

Thank you in advance,
Fabrizio

@kyrofa
Copy link
Member

kyrofa commented Feb 14, 2018

@fdemassis that isn't related to this issue, please log a new one. Note that AH02155 is an alert, not an error, and is expected. Something else is going on. Please log a new bug and include the output from this script.

@lindhe
Copy link
Author

lindhe commented Mar 20, 2018

@kyrofa It's complaining on me again. Would you kindly assist in figuring out why?

I get emails from LetsEncrypt saying that my cert will expire in 9 days. The logs at my server says:

Mar 18 20:27:35 server nextcloud.renew-certs[4203]: -------------------------------------------------------------------------------
Mar 18 20:27:35 server nextcloud.renew-certs[4203]: The following certs are not due for renewal yet:
Mar 18 20:27:35 server nextcloud.renew-certs[4203]:   /var/snap/nextcloud/current/certs/certbot/config/live/lindhe.io/fullchain.pem (skipped)
Mar 18 20:27:35 server nextcloud.renew-certs[4203]: No renewals were attempted.
Mar 18 20:27:35 server nextcloud.renew-certs[4203]: No hooks were run.
Mar 19 20:27:38 server nextcloud.renew-certs[4203]: Saving debug log to /var/snap/nextcloud/current/certs/certbot/logs/letsencrypt.log
Mar 19 20:27:38 server nextcloud.renew-certs[4203]: Cert not yet due for renewal
Mar 19 20:27:38 server nextcloud.renew-certs[4203]: -------------------------------------------------------------------------------

over and over again.

Where do I begin?

@Nubzori
Copy link

Nubzori commented May 11, 2018

Hello,

I have similar problems, my cert did not renew automatically. My ip has changed to different one during this 90 day certificate. I've changed my domain to point into my new ip.
I tried:

sudo nextcloud.enable-https lets-encrypt

but it gives me this error:

An unexpected error occurred: IOError: [Errno 13] Permission denied: '/etc/letsencrypt/cli.ini' Please see the logfile '/tmp/tmpdvL2dk' for more details.

Logfile contains same error message:

2018-05-10 23:38:30,567:DEBUG:certbot.main:certbot version: 0.14.1 2018-05-10 23:38:30,567:DEBUG:certbot.main:Arguments: ['--text', '--config-dir', '/var/snap/nextcloud/current/certs/certbot/config', '--work-dir', '/var/snap/nextcloud/current/certs/certbot/work', '--logs-dir', '/var/snap/nextcloud/current/certs/certbot/logs', '--post-hook', 'restart-apache'] 2018-05-10 23:38:30,567:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#nextcloud:webroot,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot) 2018-05-10 23:38:30,572:DEBUG:certbot.log:Exiting abnormally: Traceback (most recent call last): File "/snap/nextcloud/6916/bin/certbot", line 11, in <module> sys.exit(main()) File "/snap/nextcloud/6916/lib/python2.7/site-packages/certbot/main.py", line 729, in main args = cli.prepare_and_parse_args(plugins, cli_args) File "/snap/nextcloud/6916/lib/python2.7/site-packages/certbot/cli.py", line 1101, in prepare_and_parse_args return helpful.parse_args() File "/snap/nextcloud/6916/lib/python2.7/site-packages/certbot/cli.py", line 534, in parse_args parsed_args = self.parser.parse_args(self.args) File "/snap/nextcloud/6916/lib/python2.7/site-packages/configargparse.py", line 373, in parse_args env_vars = env_vars) File "/snap/nextcloud/6916/lib/python2.7/site-packages/configargparse.py", line 472, in parse_known_args config_streams = self._open_config_files(args) File "/snap/nextcloud/6916/lib/python2.7/site-packages/configargparse.py", line 715, in _open_config_files for f in files] IOError: [Errno 13] Permission denied: '/etc/letsencrypt/cli.ini' ~

Running sudo journalctl -u snap.nextcloud.renew-certs
contains this:

-- Logs begin at Tue 2018-05-08 23:37:50 EEST, end at Fri 2018-05-11 15:16:45 EEST. -- May 08 23:38:21 asuka systemd[1]: Started Service for snap application nextcloud.renew-certs. May 08 23:38:23 asuka nextcloud.renew-certs[1831]: Self-signed certificates aren't due for renewal May 08 23:38:28 asuka nextcloud.renew-certs[1831]: An unexpected error occurred: May 08 23:38:28 asuka nextcloud.renew-certs[1831]: IOError: [Errno 13] Permission denied: '/etc/letsencrypt/cli.ini' May 08 23:38:28 asuka nextcloud.renew-certs[1831]: Please see the logfile '/tmp/tmpIe8fch' for more details. May 09 23:38:28 asuka nextcloud.renew-certs[1831]: Self-signed certificates aren't due for renewal May 09 23:38:29 asuka nextcloud.renew-certs[1831]: An unexpected error occurred: May 09 23:38:29 asuka nextcloud.renew-certs[1831]: IOError: [Errno 13] Permission denied: '/etc/letsencrypt/cli.ini' May 09 23:38:29 asuka nextcloud.renew-certs[1831]: Please see the logfile '/tmp/tmpnJWLrf' for more details. May 10 23:38:29 asuka nextcloud.renew-certs[1831]: Self-signed certificates aren't due for renewal May 10 23:38:30 asuka nextcloud.renew-certs[1831]: An unexpected error occurred: May 10 23:38:30 asuka nextcloud.renew-certs[1831]: IOError: [Errno 13] Permission denied: '/etc/letsencrypt/cli.ini' May 10 23:38:30 asuka nextcloud.renew-certs[1831]: Please see the logfile '/tmp/tmp6YFSR3' for more details.

Any ideas what to do?

@kyrofa
Copy link
Member

kyrofa commented May 11, 2018

@Nubzori please log a new issue, that seems completely unrelated. Please include the output of snap version as well.

@kyrofa kyrofa added the bug Something isn't working label Aug 27, 2018
@mdsharpe
Copy link

Thank you @kyrofa your solution worked for me.

@lindhe
Copy link
Author

lindhe commented Aug 2, 2020

Hi. I had this issue again, and tried to sudo rm -rf /var/snap/nextcloud/current/certs like previously. But now when I run sudo nextcloud.enable-https lets-encrypt again I get an error. Have something changed since last time, perhaps the certbot binary is now stored in the certs directory?

...
Attempting to obtain certificates... error running certbot:

/var/snap/nextcloud/current/certs/certbot does not exist or is not a directory

@jotoho
Copy link

jotoho commented Oct 6, 2020

After deleting the /var/snap/nextcloud/current/certs directory I got the same certbot error as @lindhe when attempting to get a new certificate.

Perhaps I should have read to the bottom of the thread before copy-pasting commands into terminal...

@kyrofa
Copy link
Member

kyrofa commented Oct 6, 2020

Try creating an empty directory there, does it change anything?

@ElijahHW
Copy link

Try creating an empty directory there, does it change anything?

for me this now gives me:

Some challenges have failed.
IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: MyDomainName
   Type:   None
   Detail: No valid IP addresses found for MyDomainName

@kyrofa
Copy link
Member

kyrofa commented Jan 16, 2021

@ElijahHW that sounds legitimate, you're sure you typed it right/registered it properly? Can you ping that domain name? Any chance you feel like shooting me an email containing the domain so I can make sure it looks okay from here?

@theoneandonly-vector
Copy link

theoneandonly-vector commented Mar 11, 2021

Attempting to obtain certificates... error running certbot:

Saving debug log to /var/snap/nextcloud/current/certs/certbot/logs/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Unable to register an account with ACME server
root@ubuntu:/var/snap/nextcloud/current/certs# sudo nextcloud.enable-https lets-encrypt

this happens after creating an empty
/certs/certbot

the log:

2021-03-11 19:34:25,129:DEBUG:certbot.main:certbot version: 0.33.1
2021-03-11 19:34:25,130:DEBUG:certbot.main:Arguments: ['--text', '--config-dir', '/var/snap/nextcloud/current/certs/certbot/config', '--work-dir', '/>2021-03-11 19:34:25,130:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#nextcloud:webroot,PluginEntry>2021-03-11 19:34:25,190:DEBUG:certbot.log:Root logging level set at 20
2021-03-11 19:34:25,192:INFO:certbot.log:Saving debug log to /var/snap/nextcloud/current/certs/certbot/logs/letsencrypt.log
2021-03-11 19:34:25,194:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2021-03-11 19:34:25,213:DEBUG:certbot.plugins.selection:Single candidate plugin: * webroot
Description: Place files in webroot directory
Interfaces: IAuthenticator, IPlugin
Entry point: webroot = certbot.plugins.webroot:Authenticator                                                                                          Initialized: <certbot.plugins.webroot.Authenticator object at 0xffff8c587590>
Prep: True
2021-03-11 19:34:25,215:DEBUG:certbot.plugins.selection:Selected authenticator <certbot.plugins.webroot.Authenticator object at 0xffff8c587590> and i>2021-03-11 19:34:25,215:INFO:certbot.plugins.selection:Plugins selected: Authenticator webroot, Installer None
2021-03-11 19:34:28,371:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/directory.                                      2021-03-11 19:34:28,385:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org:443
2021-03-11 19:34:29,035:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "GET /directory HTTP/1.1" 200 658                       2021-03-11 19:34:29,039:DEBUG:acme.client:Received response:                                                                                          HTTP 200
Server: nginx                                                                                                                                         Date: Thu, 11 Mar 2021 19:34:28 GMT                                                                                                                   Content-Type: application/json
Content-Length: 658                                                                                                                                   Connection: keep-alive
Cache-Control: public, max-age=0, no-cache

removing the whole dir was the wrong idea..
now is there a way without reinstalling?

@theoneandonly-vector
Copy link

Finally @home I just setup a fresh install (pi3 with SD is so much slower than a pi4 with an SSD.. -> why did I even bother to update it..) finally I just copied over the removed dir -> worked.

@072neonusr
Copy link

Ran into the same issue after trying to renew my server url after the old one got purged from afraid.org. (mistake: I created a new cert before disabling https)
I followed the steps shown here and removing
/var/snap/nextcloud/current/certs/certbot/
also killed my abillity to create new certs because of a certbot error

As @theoneandonly-vector, I stole the folder from a fresh vm install and copied it into my server which fixed it again. New url is running without throwing cert-errors in browsers

@codygamer666
Copy link

codygamer666 commented Dec 22, 2021

Ran into the same issue after trying to renew my server url after the old one got purged from afraid.org. (mistake: I created a new cert before disabling https) I followed the steps shown here and removing /var/snap/nextcloud/current/certs/certbot/ also killed my abillity to create new certs because of a certbot error

As @theoneandonly-vector, I stole the folder from a fresh vm install and copied it into my server which fixed it again. New url is running without throwing cert-errors in browsers

Deleting the folder was ideed usefull, but restoring the certbot files is just 3 lines of cli:

#install certbot#
sudo snap install certbot

#create the new certbot folder#
cd /var/snap/nextcloud/current
sudo mkdir certs

#copy certbot files to nextcloud snap#
sudo cp -a /snap/certbot/current/. /var/snap/nextcloud/current/certs/certbot/

#unistall certbot snap | optional #
sudo snap remove certbot

i hope you will find this reply usefull

@kyrofa
Copy link
Member

kyrofa commented Dec 22, 2021

The certbot snap is not required and indeed not useful in this context.

@codygamer666
Copy link

The certbot snap is not required and indeed not useful in this context.

You are right, but there is a valid reason: for instance if someone unluky deletes the folder and needs a quick way to recover it this is a valid solution and solved my problem. I started my nextcloud vm with 2 A records (example1.ex.ex; example2.ex.ex) and 2 public ips, everything was behind a dedicated firewall. Then i decided only to use one A record so i disabled https and enabled again with letsencrypt with only one name and disabled the rule for that ip in the firewall. The problem was that the acme client still tried to get a certificate for the second domain. I followed several guides but with no results. I came across this post and deleted the certbot folder. After copying a fresh copy from the snap version and it worked flawlessly again. I don't know if this is an universal fix, i hope this can help someone.

@H4NN351
Copy link

H4NN351 commented Jan 20, 2022

@codygamer666 Thank You!
Your fix actually worked for me, one thing I have to add is that you have to create the certs directory, before you copy. I did that like this:
cd /var/snap/nextcloud/current
sudo mkdir certs

I must say I don't understand the problem fully, if only the directories are missing or certbots actual files are needed but this fixed it for me.

@kyrofa You might want to edit you inital response, to include a warning that deleting may cause problems. I am very thankful though, I scoured the web for 3 hours until I found your fix :)

EDIT:
I believe you should only delete some parts of: (If someone has a clean install you could check maybe)
/var/snap/nextcloud/28714/certs/certbot/config/

@codygamer666
Copy link

@codygamer666 Thank You! Your fix actually worked for me, one thing I have to add is that you have to create the certs directory, before you copy. I did that like this: cd /var/snap/nextcloud/current sudo mkdir certs

I must say I don't understand the problem fully, if only the directories are missing or certbots actual files are needed but this fixed it for me.

@kyrofa You might want to edit you inital response, to include a warning that deleting may cause problems. I am very thankful though, I scoured the web for 3 hours until I found your fix :)

EDIT: I believe you should only delete some parts of: (If someone has a clean install you could check maybe) /var/snap/nextcloud/28714/certs/certbot/config/

I'll edit my comment right now to include the folder creation step, thanks for your feedback

@kasnder
Copy link

kasnder commented Feb 5, 2022

Make sure that all domains for the cert points to your server. Also, check the logs before asking questions.

This wasn't intuitive to me.

Was surprised that my certificates didn't update automatically, because one of my two domains with certificates wasn't in use anymore.

@kyrofa
Copy link
Member

kyrofa commented Feb 5, 2022

Was surprised that my certificates didn't update automatically, because one of my two domains with certificates wasn't in use anymore.

Yes, this is simply how Let's Encrypt works. If you have a single cert valid for multiple domains, it has to re-validate all of those domains in order to renew that certificate. The only thing different they could have done was to say "oh, domain A didn't validate. Well, I guess we'll just issue a new cert and strip that domain off." I think they made the right call, there-- an error is definitely better.

@kasnder
Copy link

kasnder commented Feb 5, 2022

Was surprised that my certificates didn't update automatically, because one of my two domains with certificates wasn't in use anymore.

Yes, this is simply how Let's Encrypt works. If you have a single cert valid for multiple domains, it has to re-validate all of those domains in order to renew that certificate. The only thing different they could have done was to say "oh, domain A didn't validate. Well, I guess we'll just issue a new cert and strip that domain off." I think they made the right call, there-- an error is definitely better.

I see the logic behind the implementation of cerbot. What would've helped me a lot is see a notification in the NextCloud UI with a link to more information to fix the problem. To make the error super salient, and help address the problem easily.

@kyrofa
Copy link
Member

kyrofa commented Feb 5, 2022

Yeah I think ideally the whole process of setting up HTTPS certs would be a Nextcloud app. That requires more time than I have, but we might be able to simply surface the error through the existing notification functionality.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests