Augeas fails to parse RewriteRule and FilesMatch regexps (incl. Drupal .htaccess files) #1531

Closed
kristofferwiklund opened this Issue Nov 17, 2015 · 23 comments

Projects

None yet

6 participants

@kristofferwiklund

I am trying running the following command:
./letsencrypt-auto --server https://acme-v01.api.letsencrypt.org/directory --help

And I get:
Updating letsencrypt and virtual environment dependencies.......
Running with virtualenv: sudo /home/koffe/.local/share/letsencrypt/bin/letsencrypt --apache --server https://acme-v01.api.letsencrypt.org/directory --agree-dev-preview
The apache plugin is not working; there may be problems with your existing configuration.
The error was: PluginError(('There has been an error in parsing the file (%s): %s', u'/var/aegir/platforms/20151024/.htaccess', u'Syntax error'),)

With a verbose output:
Requested authenticator apache and installer apache
Other error:(PluginEntryPoint#apache): ('There has been an error in parsing the file (%s): %s', u'/var/aegir/platforms/20151024/.htaccess', u'Syntax error')
Traceback (most recent call last):
File "/home/koffe/.local/share/letsencrypt/local/lib/python2.7/site-packages/letsencrypt/plugins/disco.py", line 103, in prepare self._initialized.prepare()
File "/home/koffe/.local/share/letsencrypt/local/lib/python2.7/site-packages/letsencrypt_apache/configurator.py", line 152, in prepare self.check_parsing_errors("httpd.aug")
File "/home/koffe/.local/share/letsencrypt/local/lib/python2.7/site-packages/letsencrypt_apache/augeas_configurator.py", line 68, in check_parsing_errors raise errors.PluginError(msg)
PluginError: ('There has been an error in parsing the file (%s): %s', u'/var/aegir/platforms/20151024/.htaccess', u'Syntax error')

Here is the .htaccess file
htaccess.txt

@bmw bmw added the apache label Nov 18, 2015
@anarcat
anarcat commented Nov 18, 2015

this is likely #1294 - i had the same issue and in there they say it was fixed in master. yet i have tried to run the dev version as explained in:

https://letsencrypt.readthedocs.org/en/latest/contributing.html#hacking

.. but i got the same error. either the bug is not fixed or the install instructions are too complicated or it's time to make a release (or all of the above).

@anarcat
anarcat commented Nov 18, 2015

yep - i tried flushing the venv and re-running the above instructions, i still get the bug:

http://paste.debian.net/333539/

$ git show
commit fc07238804314ae3b29d915e40756e5d3e4d0908
Merge: f265179 ec58ad2
Author: Peter Eckersley <pde@users.noreply.github.com>
Date:   Mon Nov 16 12:19:55 2015 -0800

    Merge pull request #1516 from henrychen95/master

    Fix Amazon Linux bootstrapping error.

@kristofferwiklund

I am running on master already. Doing a "git pull" in ~/letsencrypt.
I am on this commit: 457be44

Still same problem.

@anarcat
anarcat commented Nov 19, 2015

i wonder if this is not because of RewriteRules, which are both in Drupal's .htaccess and my specific config. This is my config:

# Apache config file for anarcat.wiki.orangeseeds.org
# Automatically generated by ikisite; do not modify directly.

<VirtualHost *:80>
        ServerAdmin root@localhost
        ServerName anarcat.wiki.orangeseeds.org:80


        SuexecUserGroup w-anarcat w-anarcat


        UserDir disabled

        RewriteEngine On
        RewriteRule ^/(.*) http\:\/\/anarc\.at\/$1 [L,R,NE]

        ErrorLog /var/log/ikiwiki-hosting/w-anarcat/error.log
        LogLevel warn
        CustomLog /var/log/ikiwiki-hosting/w-anarcat/access.log combined
</VirtualHost>

i am also on 457be44 now.

@pde pde added this to the Nice for 1.0 milestone Nov 21, 2015
@pde
Member
pde commented Nov 21, 2015

@domcleal Is this an augeas lens bug?

@pde
Member
pde commented Nov 21, 2015

Probably an instance of hercules-team/augeas#307

@pde pde changed the title from Syntax error on Drupal .htaccess to Augeas fails to parse RewriteRule and FilesMatch regexps (incl. Drupal .htaccess files) Nov 22, 2015
@domcleal
Collaborator

Yep, it's that.

@pde pde modified the milestone: 1.0 for launch, Nice for 1.0 Nov 24, 2015
@pde
Member
pde commented Nov 27, 2015

The Drupal .htaccess case is hercules-team/augeas#324.

@pde
Member
pde commented Dec 2, 2015

We're waiting for the augeas team to review their PR for this; will ship it when it's reviewed. Meanwhile, kicking the remainder of this issue out of 1.0 for launch.

@pde pde modified the milestone: Nice for 1.0, 1.0 for launch Dec 2, 2015
@domcleal
Collaborator
domcleal commented Dec 2, 2015

@pde sorry about that, merged as hercules-team/augeas@cb85b59.

@pde pde closed this in #1692 Dec 8, 2015
@kristofferwiklund

Running ./letsencrypt-auto --apache with the latest version of letsencrypt (commit 3838ea4)

thecomputer ~/letsencrypt (master): ./letsencrypt-auto --apache
Updating letsencrypt and virtual environment dependencies.......
Running with virtualenv: sudo /home/theuser/.local/share/letsencrypt/bin/letsencrypt --apache
The apache plugin is not working; there may be problems with your existing configuration.
The error was: PluginError(('There has been an error in parsing the file (%s): %s', u'/var/aegir/platforms/20151024/.htaccess', u'Syntax error'),)

So it is not working. This is still a issue.

@domcleal
Collaborator
domcleal commented Dec 8, 2015

While I've not tried the LE client, the lens currently in 3838ea4 seems to be parsing the htaccess file linked in the description just fine, and correctly fails on the previous commit.

@acrollet
acrollet commented Dec 8, 2015

@kristofferwiklund - I had trouble with this too - you need to be sure that letsencrypt-apache/letsencrypt_apache/augeas_lens/httpd.aug has been updated.

@pde pde modified the milestone: 0.1.1, Nice for 1.0 Dec 8, 2015
@kristofferwiklund

httpd.aug had been update yesterday. But still problems running commit ce14851

@domcleal
Collaborator
domcleal commented Dec 9, 2015

Can you pastebin the exact file so I can double check it's parsing with the lens itself? We may be able to narrow down then whether it's another issue in the lens or if it's a problem in how it's being run.

If you can follow these steps too, we can check it's parsing on your computer by doing:

  1. download augcheck from https://raw.githubusercontent.com/raphink/augeas-sandbox/master/augcheck
  2. ensure you have augparse installed (augeas-tools package on Debian/Ubuntu, augeas on RPM distros)
  3. from the LE directory, run: AUGEAS_LENS_LIB=letsencrypt-apache/letsencrypt_apache/augeas_lens/ augcheck /var/aegir/platforms/20151024/.htaccess Httpd
@kristofferwiklund

The .htaccess file is the same as the reported (https://github.com/letsencrypt/letsencrypt/files/36394/htaccess.txt)

I have runned the commands provide:
augcheck.result.txt

Seems to be working standalone.

@kristofferwiklund

Log files says:

2015-12-09 21:57:32,099:DEBUG:letsencrypt.cli:Root logging level set at 30
2015-12-09 21:57:32,100:INFO:letsencrypt.cli:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2015-12-09 21:57:32,100:DEBUG:letsencrypt.cli:letsencrypt version: 0.1.0
2015-12-09 21:57:32,100:DEBUG:letsencrypt.cli:Arguments: ['--apache']
2015-12-09 21:57:32,100:DEBUG:letsencrypt.cli:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#webroot,PluginEntryPoint#null,Plug
2015-12-09 21:57:32,103:DEBUG:letsencrypt.cli:Requested authenticator apache and installer apache
2015-12-09 21:57:32,341:DEBUG:letsencrypt.plugins.disco:Other error:(PluginEntryPoint#apache): ('There has been an error in parsing the file (%s): %s', u'/va
Traceback (most recent call last):
File "/home/koffe/.local/share/letsencrypt/local/lib/python2.7/site-packages/letsencrypt/plugins/disco.py", line 103, in prepare
self._initialized.prepare()
File "/home/koffe/.local/share/letsencrypt/local/lib/python2.7/site-packages/letsencrypt_apache/configurator.py", line 150, in prepare
self.check_parsing_errors("httpd.aug")
File "/home/koffe/.local/share/letsencrypt/local/lib/python2.7/site-packages/letsencrypt_apache/augeas_configurator.py", line 68, in check_parsing_errors
raise errors.PluginError(msg)
PluginError: ('There has been an error in parsing the file (%s): %s', u'/var/aegir/platforms/20151024/.htaccess', u'Syntax error')
2015-12-09 21:57:32,341:DEBUG:letsencrypt.display.ops:No candidate plugin
2015-12-09 21:57:32,341:DEBUG:letsencrypt.cli:Selected authenticator None and installer None

@kristofferwiklund

I am testing to copy the .htaccess file to another server, that is not running Drupal sites (and the Aegir setup). And there the parsing works, just get some other error
But that is not related to this. Its about the verification and I get the error: The client lacks sufficient authorization :: Correct zName not found for TLS SNI challenge.

So there is something in the setup of Apache that triggers a Syntax error for the htaccess file.

@pde pde added a commit that referenced this issue Dec 10, 2015
@pde pde Another #1531 8f553b6
@domcleal
Collaborator

Are /home/koffe/.local/share/letsencrypt/local/lib/python2.7/site-packages/letsencrypt_apache/augeas_lens/httpd.aug and letsencrypt-apache/letsencrypt_apache/augeas_lens/httpd.aug identical (compare the md5sum)?

A quick test here shows that it might not get updated in ~/.local when you run ./letsencrypt-auto from the git checkout.

@kristofferwiklund

They aren't.. :)

Should I just delete the .local folder and hope letsencrypt recreates it. Or is the some update command that can be run for ./letsencrypt-auto

@domcleal
Collaborator

It should be updating already, so it's a bug somewhere (it looks like it's running pip install to update it, but perhaps not registering that the file changed). I'd probably just copy the file for now as .local may contain things other than LE.

Edit: correction, you could probably delete ~/.local/share/letsencrypt safely and it'll reinstall everything.

It appears to me that letsencrypt-auto is only designed to install the latest released version, not the latest contents of a git checkout.

@kristofferwiklund

Now that is working. But it found some more problems with a Drupal 6 site. But for this I can understand it.

Here is the Drupal 6.37 .htaccess file: http://cgit.drupalcode.org/drupal/tree/.htaccess?h=6.37
On line 22 the ending " is missing. Some type of syntax hack for Apache 1.3

Drupal 6 is EOL in February 2016 so people still using it have to patch their on .htaccess file.

Adding in the " and letsencrypt is proceeding. Now I just get "Correct zName not found for TLS SNI challenge.". But that is for an other issue, if Google is not helping me.

@domcleal
Collaborator

Thanks, the missing double quote is bug #1724.

@pde pde added a commit that referenced this issue Dec 11, 2015
@pde pde Another #1531 bdfca70
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment