Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Getting sed error using dns_linode_v4 on macOS/Ubuntu: RE error: invalid repetition count(s) #3645

Closed
ddribin opened this issue Aug 5, 2021 · 11 comments

Comments

@ddribin
Copy link

ddribin commented Aug 5, 2021

Steps to reproduce

I'm on macOS Catalina 10.15.7. Following the instructions from:

https://github.com/acmesh-official/acme.sh/wiki/dnsapi#14-use-linode-domain-api

export LINODE_V4_API_KEY="<redacted>"
acme.sh --staging --issue --dns dns_linode_v4 --dnssleep 300 -d 'yurt.dribin.net'  --debug 2

I see two sed errors:

...
[Wed Aug  4 23:54:00 CDT 2021] Adding txt value: <redacted> for domain:  _acme-challenge.yurt.dribin.net
[Wed Aug  4 23:54:00 CDT 2021] Using Linode
sed: 1: "s/.*\(\{.*"domain": *"y ...": RE error: invalid repetition count(s)
[Wed Aug  4 23:54:01 CDT 2021] Domain resource successfully added.
[Wed Aug  4 23:54:01 CDT 2021] The txt record is added: Success.
[Wed Aug  4 23:54:01 CDT 2021] Sleep 300 seconds for the txt records to take effect
[Wed Aug  4 23:59:04 CDT 2021] Verifying: yurt.dribin.net
[Wed Aug  4 23:59:04 CDT 2021] Pending, The CA is processing your order, please just wait. (1/30)
[Wed Aug  4 23:59:06 CDT 2021] Success
[Wed Aug  4 23:59:07 CDT 2021] Removing DNS records.
[Wed Aug  4 23:59:07 CDT 2021] Removing txt: <redacted> for domain: _acme-challenge.yurt.dribin.net
[Wed Aug  4 23:59:07 CDT 2021] Using Linode
sed: 1: "s/.*\(\{.*"domain": *"y ...": RE error: invalid repetition count(s)
[Wed Aug  4 23:59:08 CDT 2021] Domain resource successfully deleted.
[Wed Aug  4 23:59:08 CDT 2021] Removed: Success
...

Debug log

acme.sh --staging --issue --dns dns_linode_v4 --dnssleep 300 -d 'yurt.dribin.net'  --debug 2
[Wed Aug  4 23:46:45 CDT 2021] Lets find script dir.
[Wed Aug  4 23:46:45 CDT 2021] _SCRIPT_='/Users/dave/Scratch/acme.sh/acme.sh'
[Wed Aug  4 23:46:45 CDT 2021] _script='/Users/dave/Scratch/acme.sh/acme.sh'
[Wed Aug  4 23:46:45 CDT 2021] _script_home='/Users/dave/Scratch/acme.sh'
[Wed Aug  4 23:46:45 CDT 2021] Using config home:/Users/dave/Scratch/acme.sh
[Wed Aug  4 23:46:45 CDT 2021] LE_WORKING_DIR='/Users/dave/Scratch/acme.sh'
https://github.com/acmesh-official/acme.sh
v3.0.1
...
[Wed Aug  4 23:46:49 CDT 2021] Using Linode
...
[Wed Aug  4 23:46:49 CDT 2021] _CURL='curl --silent --dump-header /Users/dave/Scratch/acme.sh/http.header  -L  --trace-ascii /var/folders/74/r34bgh6d35xgnv_4r1pnb75h0000gn/T/tmp.kM3Z2rAw  -g '
* Closing connection 0
[Wed Aug  4 23:46:50 CDT 2021] ret='0'
[Wed Aug  4 23:46:50 CDT 2021] _hcode='0'
[Wed Aug  4 23:46:50 CDT 2021] response='{"data": [{"id": 348530, "type": "master", "domain": "dribin.net", "tags": [], "group": "", "status": "active", "description": "", "soa_email": "<redacted>", "retry_sec": 0, "master_ips": [], "axfr_ips": [], "expire_sec": 0, "refresh_sec": 300, "ttl_sec": 300, "created": "2012-08-05T15:41:40", "updated": "2021-08-05T04:45:53"}], "page": 1, "pages": 1, "results": 1}'
[Wed Aug  4 23:46:50 CDT 2021] h='yurt.dribin.net'
sed: 1: "s/.*\(\{.*"domain": *"y ...": RE error: invalid repetition count(s)
[Wed Aug  4 23:46:50 CDT 2021] h='dribin.net'
[Wed Aug  4 23:46:50 CDT 2021] _domain_id='348530'
[Wed Aug  4 23:46:50 CDT 2021] _sub_domain='_acme-challenge.yurt'
[Wed Aug  4 23:46:50 CDT 2021] _domain='dribin.net'
...
@ddribin
Copy link
Author

ddribin commented Aug 5, 2021

I have tracked this down to these two lines in dnsapi/dns_linode_v4.sh. Line 77:

    resource="$(echo "$response" | _egrep_o "\{.*\"name\": *\"$_sub_domain\".*}")"

And line 143:

      hostedzone="$(echo "$response" | _egrep_o "\{.*\"domain\": *\"$h\".*}")"

_egrep_o uses sed:

_egrep_o() {
  if ! egrep -o "$1" 2>/dev/null; then
    sed -n 's/.*\('"$1"'\).*/\1/p'
  fi
}

And sed is not liking the backslash before the brace: \{. I can reproduce this in a simple shell script:

#!/bin/bash

set -o xtrace

_egrep_o() {
  if ! egrep -o "$1" 2>/dev/null; then
    sed -n 's/.*\('"$1"'\).*/\1/p'
  fi
}

response='foo'
_sub_domain=yurt.dribin.net

echo "foo" | _egrep_o "\{.*\"name\": *\"$_sub_domain\".*}"
% /tmp/invalid_repetition
+ response=foo
+ _sub_domain=yurt.dribin.net
+ echo foo
+ _egrep_o '\{.*"name": *"yurt.dribin.net".*}'
+ egrep -o '\{.*"name": *"yurt.dribin.net".*}'
+ sed -n 's/.*\(\{.*"name": *"yurt.dribin.net".*}\).*/\1/p'
sed: 1: "s/.*\(\{.*"name": *"yur ...": RE error: invalid repetition count(s)

@ddribin ddribin changed the title Getting sed error on macOS: RE error: invalid repetition count(s) with dns_linode_v4 Getting sed error using dns_linode_v4 on macOS: RE error: invalid repetition count(s) Aug 5, 2021
@ddribin
Copy link
Author

ddribin commented Aug 5, 2021

I can fix it by removing the backslash, but I don't know how that affects other platforms:

diff --git a/dnsapi/dns_linode_v4.sh b/dnsapi/dns_linode_v4.sh
index 9504afb..2933c87 100755
--- a/dnsapi/dns_linode_v4.sh
+++ b/dnsapi/dns_linode_v4.sh
@@ -74,7 +74,7 @@ dns_linode_v4_rm() {
   if _rest GET "/$_domain_id/records" && [ -n "$response" ]; then
     response="$(echo "$response" | tr -d "\n" | tr '{' "|" | sed 's/|/&{/g' | tr "|" "\n")"
 
-    resource="$(echo "$response" | _egrep_o "\{.*\"name\": *\"$_sub_domain\".*}")"
+    resource="$(echo "$response" | _egrep_o "{.*\"name\": *\"$_sub_domain\".*}")"
     if [ "$resource" ]; then
       _resource_id=$(printf "%s\n" "$resource" | _egrep_o "\"id\": *[0-9]+" | _head_n 1 | cut -d : -f 2 | tr -d \ )
       if [ "$_resource_id" ]; then
@@ -140,7 +140,7 @@ _get_root() {
         return 1
       fi
 
-      hostedzone="$(echo "$response" | _egrep_o "\{.*\"domain\": *\"$h\".*}")"
+      hostedzone="$(echo "$response" | _egrep_o "{.*\"domain\": *\"$h\".*}")"
       if [ "$hostedzone" ]; then
         _domain_id=$(printf "%s\n" "$hostedzone" | _egrep_o "\"id\": *[0-9]+" | _head_n 1 | cut -d : -f 2 | tr -d \ )
         if [ "$_domain_id" ]; then

@ddribin
Copy link
Author

ddribin commented Aug 5, 2021

Looks like the backslashes were explicitly added with commit 38a8721.

@Neilpang
Copy link
Member

Neilpang commented Aug 5, 2021

you don't have egrep in your Mac?

@ddribin
Copy link
Author

ddribin commented Aug 5, 2021

I do:

% where egrep
/usr/bin/egrep

@ddribin
Copy link
Author

ddribin commented Aug 5, 2021

I ran dns_linode_v4.sh with set -o xtrace. Here's the output from this snippet of code in _get_root():

    response="$(echo "$response" | tr -d "\n" | tr '{' "|" | sed 's/|/&{/g' | tr "|" "\n")"
    while true; do
      h=$(printf "%s" "$domain" | cut -d . -f $i-100)
      _debug h "$h"
      if [ -z "$h" ]; then
        #not valid
        return 1
      fi

      hostedzone="$(echo "$response" | _egrep_o "\{.*\"domain\": *\"$h\".*}")"
+ response='{"data": [{"id": 348530, "type": "master", "domain": "dribin.net", "tags": [], "group": "", "status": "active", "description": "", "soa_email": "redacted@example.com", "retry_sec": 0, "master_ips": [], "axfr_ips": [], "expire_sec": 0, "refresh_sec": 300, "ttl_sec": 300, "created": "2012-08-05T15:41:40", "updated": "2021-08-05T14:50:07"}], "page": 1, "pages": 1, "results": 1}'
+ '[' 0 '!=' 0 ']'
+ _debug2 response '{"data": [{"id": 348530, "type": "master", "domain": "dribin.net", "tags": [], "group": "", "status": "active", "description": "", "soa_email": "redacted@example.com", "retry_sec": 0, "master_ips": [], "axfr_ips": [], "expire_sec": 0, "refresh_sec": 300, "ttl_sec": 300, "created": "2012-08-05T15:41:40", "updated": "2021-08-05T14:50:07"}], "page": 1, "pages": 1, "results": 1}'
+ '[' 3 -ge 2 ']'
+ _log response '{"data": [{"id": 348530, "type": "master", "domain": "dribin.net", "tags": [], "group": "", "status": "active", "description": "", "soa_email": "redacted@example.com", "retry_sec": 0, "master_ips": [], "axfr_ips": [], "expire_sec": 0, "refresh_sec": 300, "ttl_sec": 300, "created": "2012-08-05T15:41:40", "updated": "2021-08-05T14:50:07"}], "page": 1, "pages": 1, "results": 1}'
+ '[' -z /Users/dave/Scratch/acme.sh/acme.sh.log ']'
+ _printargs response '{"data": [{"id": 348530, "type": "master", "domain": "dribin.net", "tags": [], "group": "", "status": "active", "description": "", "soa_email": "redacted@example.com", "retry_sec": 0, "master_ips": [], "axfr_ips": [], "expire_sec": 0, "refresh_sec": 300, "ttl_sec": 300, "created": "2012-08-05T15:41:40", "updated": "2021-08-05T14:50:07"}], "page": 1, "pages": 1, "results": 1}'
+ _exitstatus=1
+ '[' -z 1 ']'
+ '[' 1 = 0 ']'
+ '[' -z '{"data": [{"id": 348530, "type": "master", "domain": "dribin.net", "tags": [], "group": "", "status": "active", "description": "", "soa_email": "redacted@example.com", "retry_sec": 0, "master_ips": [], "axfr_ips": [], "expire_sec": 0, "refresh_sec": 300, "ttl_sec": 300, "created": "2012-08-05T15:41:40", "updated": "2021-08-05T14:50:07"}], "page": 1, "pages": 1, "results": 1}' ']'
+ printf -- %s 'response='\''{"data": [{"id": 348530, "type": "master", "domain": "dribin.net", "tags": [], "group": "", "status": "active", "description": "", "soa_email": "redacted@example.com", "retry_sec": 0, "master_ips": [], "axfr_ips": [], "expire_sec": 0, "refresh_sec": 300, "ttl_sec": 300, "created": "2012-08-05T15:41:40", "updated": "2021-08-05T14:50:07"}], "page": 1, "pages": 1, "results": 1}'\'''
+ printf '\n'
+ return 1
+ '[' 0 -ge 8 ']'
+ '[' 0 -ge 2 ']'
+ return 0
++ echo '{"data": [{"id": 348530, "type": "master", "domain": "dribin.net", "tags": [], "group": "", "status": "active", "description": "", "soa_email": "redacted@example.com", "retry_sec": 0, "master_ips": [], "axfr_ips": [], "expire_sec": 0, "refresh_sec": 300, "ttl_sec": 300, "created": "2012-08-05T15:41:40", "updated": "2021-08-05T14:50:07"}], "page": 1, "pages": 1, "results": 1}'
++ tr -d '\n'
++ tr '{' '|'
++ sed 's/|/&{/g'
++ tr '|' '\n'
+ response='
{"data": [
{"id": 348530, "type": "master", "domain": "dribin.net", "tags": [], "group": "", "status": "active", "description": "", "soa_email": "redacted@example.com", "retry_sec": 0, "master_ips": [], "axfr_ips": [], "expire_sec": 0, "refresh_sec": 300, "ttl_sec": 300, "created": "2012-08-05T15:41:40", "updated": "2021-08-05T14:50:07"}], "page": 1, "pages": 1, "results": 1}'
+ true
++ printf %s _acme-challenge.yurt.dribin.net
++ cut -d . -f 2-100
+ h=yurt.dribin.net
+ _debug h yurt.dribin.net
+ '[' 3 -ge 1 ']'
+ _log h yurt.dribin.net
+ '[' -z /Users/dave/Scratch/acme.sh/acme.sh.log ']'
+ _printargs h yurt.dribin.net
+ _exitstatus=1
+ '[' -z 1 ']'
+ '[' 1 = 0 ']'
+ '[' -z yurt.dribin.net ']'
+ printf -- %s 'h='\''yurt.dribin.net'\'''
+ printf '\n'
+ return 1
+ '[' 0 -ge 7 ']'
+ '[' 0 -ge 1 ']'
+ '[' -z yurt.dribin.net ']'
++ echo '
{"data": [
{"id": 348530, "type": "master", "domain": "dribin.net", "tags": [], "group": "", "status": "active", "description": "", "soa_email": "redacted@example.com", "retry_sec": 0, "master_ips": [], "axfr_ips": [], "expire_sec": 0, "refresh_sec": 300, "ttl_sec": 300, "created": "2012-08-05T15:41:40", "updated": "2021-08-05T14:50:07"}], "page": 1, "pages": 1, "results": 1}'
++ _egrep_o '\{.*"domain": *"yurt.dribin.net".*}'
++ egrep -o '\{.*"domain": *"yurt.dribin.net".*}'
++ sed -n 's/.*\(\{.*"domain": *"yurt.dribin.net".*}\).*/\1/p'
sed: 1: "s/.*\(\{.*"domain": *"y ...": RE error: invalid repetition count(s)
+ hostedzone=
+ '[' '' ']'

I don't know exactly what this line is trying to do:

      hostedzone="$(echo "$response" | _egrep_o "\{.*\"domain\": *\"$h\".*}")"

But there is no "domain": field matching $h in that JSON response. Which is why grep -o returns a failure. Which is why it falls back to sed.

@ddribin
Copy link
Author

ddribin commented Aug 5, 2021

FWIW, this also fails on Ubuntu 20.04, but with a different error message:

[Thu 05 Aug 2021 10:15:10 AM PDT] Getting webroot for domain='yurt.dribin.net'
[Thu 05 Aug 2021 10:15:10 AM PDT] Adding txt value: <redacted> for domain:  _acme-challenge.yurt.dribin.net
[Thu 05 Aug 2021 10:15:10 AM PDT] Using Linode
sed: -e expression #1, char 50: Invalid preceding regular expression

@ddribin ddribin changed the title Getting sed error using dns_linode_v4 on macOS: RE error: invalid repetition count(s) Getting sed error using dns_linode_v4 on macOS/Ubuntu: RE error: invalid repetition count(s) Aug 5, 2021
@ddribin
Copy link
Author

ddribin commented Aug 5, 2021

_egrep_o() {
  if ! egrep -o "$1" 2>/dev/null; then
    sed -n 's/.*\('"$1"'\).*/\1/p'
  fi
}

I don't think this _egrep_o function is doing what it intends to do. I think it's supposed to use egrep -o if it exists and fall back to sed if does not. But what it's really doing is falling back to sed any time egrep -o fails to find a match.

@nocsav
Copy link

nocsav commented Jul 15, 2023

Reviving this old thread, as I'm struggling with same sed error on an Ubuntu 20.04 server.

just to follow up on this comment:

I don't think this _egrep_o function is doing what it intends to do. I think it's supposed to use egrep -o if it exists and fall back to sed if does not. But what it's really doing is falling back to sed any time egrep -o fails to find a match.

I think the purpose is to extract a matching substring, like so:

$ echo "a b c" | _egrep_o c
c

or return empty string if no match found:

$ echo "a b d" | _egrep_o c

@Neilpang
Copy link
Member

I made a fix, please try again:

acme.sh  --upgrade -b dev

@nocsav
Copy link

nocsav commented Jul 29, 2023

I made a fix, please try again:

That worked and resolved my issue. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants