error with variable openssl11

[cdbattags]

Paging @cybrq-as! Looks to be an issue with the way openssl11 is defined. Any insight into this error?

2018/05/31 16:54:43 [error] 8727#8727: *1 lua entry thread aborted: runtime error: /usr/local/openresty/site/lualib/resty/evp.lua:149: /usr/local/openresty/luajit/lib/libluajit-5.1.so.2: undefined symbol: EVP_MD_CTX_create
stack traceback:
coroutine 0:
        [C]: in function '__index'
        /usr/local/openresty/site/lualib/resty/evp.lua:149: in function 'ctx_new'
        /usr/local/openresty/site/lualib/resty/evp.lua:255: in function 'verify'
        /usr/local/openresty/site/lualib/resty/jwt.lua:812: in function 'verify_jwt_obj'
        /usr/local/openresty/site/lualib/resty/openidc.lua:819: in function 'openidc_load_jwt_and_verify_crypto'
        /usr/local/openresty/site/lualib/resty/openidc.lua:1418: in function 'jwt_verify'
        /usr/local/openresty/site/lualib/resty/openidc.lua:1455: in function 'bearer_jwt_verify'
        access_by_lua(http_overrides.conf:461):24: in function <access_by_lua(http_overrides.conf:461):1>, client: 127.0.0.1, server: <redacted>, request: "GET <redacted> HTTP/1.1", host: "<redacted>"

[cdbattags]

The block starting at https://github.com/cdbattags/lua-resty-jwt/blob/master/lib/resty/evp.lua#L135 seems to be the issue.

[cdbattags]

foo@bar:~$ /usr/local/openresty/openssl/bin/openssl version
OpenSSL 1.1.0h  27 Mar 2018

[ghost]

The check for OpenSSL 1.1. via the variable openssl11 seems to not work out in your environment. We know that because the symbol EVP_MD_CTX_create is not found.

Thus the call to either _C.HMAC_CTX_new() or _C.HMAC_CTX_free(ctx) fails.

Over here I have the following:

• openresty openssl

$ /usr/local/openresty/openssl/bin/openssl version
OpenSSL 1.1.0h  27 Mar 2018

• openresty

$ /usr/local/openresty/bin/openresty -v
nginx version: openresty/1.13.6.2

The symbol for OpenSSL 1.1 is present, the one for OpenSSL 1.0 not:

$ grep EVP_MD_CTX_create /usr/local/openresty/* -r -i | wc -l
0
$ grep HMAC_CTX_new /usr/local/openresty/* -r -i | wc -l
3
$ grep HMAC_CTX_new /usr/local/openresty/* -r -i
Binary file /usr/local/openresty/openssl/bin/openssl matches
Binary file /usr/local/openresty/openssl/lib/libcrypto.so.1.1 matches
Binary file /usr/local/openresty/openssl/lib/libssl.so.1.1 matches

[cdbattags]

I'm extremely confused. I have the same as the above on my box currently. Hmm...

[ghost]

Is there another version installed as well? Does your openresty need a restart after an upgrade maybe?

[cdbattags]

Looks like a conflict, hmmm.

libgnutls-openssl27/trusty-updates,trusty-security,now 2.12.23-12ubuntu2.8 amd64 [installed]
libssl1.0.0/now 1.0.1f-1ubuntu2.23 amd64 [installed,upgradable to: 1.0.1f-1ubuntu2.25]
openresty-openssl/trusty,now 1.1.0h-2~trusty1 amd64 [installed,automatic]
openssl/trusty-updates,trusty-security,now 1.0.1f-1ubuntu2.25 amd64 [installed,automatic]

[cdbattags]

A little more context: I finally added ngx.DEBUG logging to see if this might help my situation and the error I'm getting out is missing declaration for symbol 'HMAC_CTX_new' which doesn't make any sense at all... Or rather I guess this points me back out to why isn't my module seeing the packaged version of OpenSSL installed using:

wget -qO - https://openresty.org/package/pubkey.gpg | sudo apt-key add -
add-apt-repository -y "deb http://openresty.org/package/ubuntu $(lsb_release -sc) main"
apt-get update
apt -y install openresty

[cdbattags]

@cybrq-as, also isn't it kind of odd that the first line of

$ grep EVP_MD_CTX_create /usr/local/openresty/* -r -i | wc -l
0
$ grep HMAC_CTX_new /usr/local/openresty/* -r -i | wc -l
3
$ grep HMAC_CTX_new /usr/local/openresty/* -r -i
Binary file /usr/local/openresty/openssl/bin/openssl matches
Binary file /usr/local/openresty/openssl/lib/libcrypto.so.1.1 matches
Binary file /usr/local/openresty/openssl/lib/libssl.so.1.1 matches

grep EVP_MD_CTX_create /usr/local/openresty/* -r -i | wc -l returns 0?

Shouldn't it still be there in the else block of evp.lua?

Edit:

Ohh, you meant just vanilla OpenResty. Got it.

[euank]

@cdbattags I don't think the files in /usr/local/openresty/... matter so much for luajit ffi.

My understanding is ffi.C gets its symbols from the usual library path (e.g. /usr/lib/libssl.so), so it might be better to double check what package provides that and what version it currently is.

I could, of course, be wrong on all this since I'm not super familiar with this mess.

[cdbattags]

@euank, so then where would those shared object (.so) files come from if per se I downloaded OpenSSL and built from source?

Edit:

They are already packaged up with the latest release of OpenResty it's just a matter of properly linking to /usr/lib?

[euank]

Oops, on ubuntu libraries are in a different location than on my distro (/usr/lib/x86_64-linux-gnu as defined in /etc/ld.so.conf.d/*.conf)

For something you installed yourself, you can usually override that by setting LD_LIBRARY_PATH to include your librarie's directory, or alternately putting it in /usr/local/lib/... appropriately since I think that usually overrides.

You could use strace -e open command | grep libssl to try and find which ssl library it does end up using as well if that seems like a promising debug avenue.

Edit: I may be barking up the wrong tree entirely; I'm not totally sure how libssl is ending up in luajit's ffi.C without an explicit call to 'ffi.load'; depending how that's happening, it might be pulling it in from the usual linker paths, or it might be getting it from the openresty specific paths

[cdbattags]

So per jkeys089/lua-resty-hmac#11 and @jkeys0089's notes it looks like NGINX has this lib properly linked... Why on Earth would I still be getting this?