kong migrations bootstrap Error

[liudonghua123]

I tried to run kong followed the docs on https://docs.konghq.com/gateway/latest/install/docker/.

And when I execute the following command, it failed with the following errors.

[root@localhost ~]# docker network create kong-net
e7423602c16e0fd07c27326f177a2e02196370d2a83353dccff3d5f1a82904f4
[root@localhost ~]#
[root@localhost ~]#  docker run -d --name kong-database \
>   --network=kong-net \
>   -p 5432:5432 \
>   -e "POSTGRES_USER=kong" \
>   -e "POSTGRES_DB=kong" \
>   -e "POSTGRES_PASSWORD=kongpass" \
>   postgres:13
3f4be3f294944dee2df65d58181eb82389c9a73b66274095efe007302a06a149
[root@localhost ~]# docker run --rm --network=kong-net \
>   -e "KONG_DATABASE=postgres" \
>   -e "KONG_PG_HOST=kong-database" \
>   -e "KONG_PG_USER=kong" \
>   -e "KONG_PG_PASSWORD=kongpass" \
>   -e "KONG_PASSWORD=test" \
>  kong/kong-gateway:3.2.1.0 kong migrations bootstrap --v
2023/03/09 02:32:20 [verbose] Kong: 3.2.1.0-enterprise-edition
2023/03/09 02:32:20 [verbose] no config file found at /etc/kong/kong.conf
2023/03/09 02:32:20 [verbose] no config file found at /etc/kong.conf
2023/03/09 02:32:20 [verbose] no config file, skip loading
2023/03/09 02:32:20 [verbose] prefix in use: /usr/local/kong
2023/03/09 02:32:20 [verbose] preparing nginx prefix directory at /usr/local/kong
2023/03/09 02:32:20 [verbose] SSL enabled on 
2023/03/09 02:32:20 [verbose] generating /usr/local/kong/ssl/kong-default.crt SSL certificate (
2023/03/09 02:32:20 [verbose] generating /usr/local/kong/ssl/kong-default-ecdsa.crt SSL certificate (
2023/03/09 02:32:20 [verbose] SSL enabled on 
2023/03/09 02:32:20 [verbose] generating /usr/local/kong/ssl/admin-kong-default.crt SSL certificate (
2023/03/09 02:32:21 [verbose] generating /usr/local/kong/ssl/admin-kong-default-ecdsa.crt SSL certificate (
2023/03/09 02:32:21 [verbose] SSL enabled on 
2023/03/09 02:32:21 [verbose] generating /usr/local/kong/ssl/admin-gui-kong-default.crt SSL certificate (
2023/03/09 02:32:21 [verbose] generating /usr/local/kong/ssl/admin-gui-kong-default-ecdsa.crt SSL certificate (
2023/03/09 02:32:21 [verbose] generating trusted certs combined file in 
Error: 
/usr/local/share/lua/5.1/kong/cmd/migrations.lua:222: 
stack traceback:
	[C]: in function 'assert'
	/usr/local/share/lua/5.1/kong/cmd/migrations.lua:222: in function 'cmd_exec'
	/usr/local/share/lua/5.1/kong/cmd/init.lua:104: in function </usr/local/share/lua/5.1/kong/cmd/init.lua:104>
	[C]: in function 'xpcall'
	/usr/local/share/lua/5.1/kong/cmd/init.lua:104: in function </usr/local/share/lua/5.1/kong/cmd/init.lua:61>
	/usr/local/bin/kong:9: in function 'file_gen'
	init_worker_by_lua:48: in function <init_worker_by_lua:46>
	[C]: in function 'xpcall'
	init_worker_by_lua:55: in function <init_worker_by_lua:53>
[root@localhost ~]#
[root@localhost ~]# docker ps|grep kong
3f4be3f29494        postgres:13                  "docker-entrypoint.s…"   4 minutes ago       Up 4 minutes        0.0.0.0:5432->5432/tcp                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   kong-database
[root@localhost ~]# docker ps -a|grep kong
3f4be3f29494        postgres:13                          "docker-entrypoint.s…"   4 minutes ago       Up 4 minutes                0.0.0.0:5432->5432/tcp                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   kong-database
6698c1a3f490        kong/kong-gateway:3.2.1.0            "/entrypoint.sh kong…"   32 minutes ago      Exited (1) 32 minutes ago                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            kong-gateway
[root@localhost ~]#
[root@localhost ~]# docker --version
Docker version 19.03.13, build 4484c46d9d
[root@localhost ~]# cat /etc/os-release 
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

[root@localhost ~]#

[gszr]

I have ran the exact same commands locally and the migrations finish successfully.

However, this line looks odd 2023/03/09 02:32:21 [verbose] generating trusted certs combined file in . It comes from the function below:

local function gen_trusted_certs_combined_file(combined_filepath, paths)
  log.verbose("generating trusted certs combined file in ",
              combined_filepath)

  local fd = assert(io.open(combined_filepath, "w"))

  for _, path in ipairs(paths) do
    fd:write(assert(pl_file.read(path)))
    fd:write("\n")
  end

  io.close(fd)
end

If combined_filepath is nil (or otherwise empty), it could explain why the subsequent assert fails.

I will keep poking at this.

[marc-enseo]

Having the same issue after upgrading from 3.0.1 to 3.2.2. Back reving to 3.1.1 makes the issue go away.

[marc-enseo]

Updating docker from 18.x to 20.10.17 fixed the issue for me

[liudonghua123]

I updated the docker to the latest version. And it seems works now.

[root@localhost ~]# sudo yum install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
[root@localhost ~]#
[root@localhost ~]# docker -v
Docker version 23.0.1, build a5ee5b1
[root@localhost ~]#
[root@localhost ~]# docker rm kong-database
kong-database
[root@localhost ~]#  docker run -d --name kong-database \
>   --network=kong-net \
>   -p 5432:5432 \
>   -e "POSTGRES_USER=kong" \
>   -e "POSTGRES_DB=kong" \
>   -e "POSTGRES_PASSWORD=kongpass" \
>   postgres:13
1263adcd4673df3d58c1a4e861708b5d77dd5af052363a2a032c355bb6f06a44

[root@localhost ~]# 
[root@localhost ~]# docker run --rm --network=kong-net \
>   -e "KONG_DATABASE=postgres" \
>   -e "KONG_PG_HOST=kong-database" \
>   -e "KONG_PG_PASSWORD=kongpass" \
>   -e "KONG_PASSWORD=test" \
>  kong/kong-gateway:3.2.1.0 kong migrations bootstrap

Bootstrapping database...
migrating core on database 'kong'...
core migrated up to: 000_base (executed)
core migrated up to: 003_100_to_110 (executed)
core migrated up to: 004_110_to_120 (executed)
core migrated up to: 005_120_to_130 (executed)
core migrated up to: 006_130_to_140 (executed)
core migrated up to: 007_140_to_150 (executed)
core migrated up to: 008_150_to_200 (executed)
core migrated up to: 009_200_to_210 (executed)
core migrated up to: 010_210_to_211 (executed)
core migrated up to: 011_212_to_213 (executed)
core migrated up to: 012_213_to_220 (executed)
core migrated up to: 013_220_to_230 (executed)
core migrated up to: 014_230_to_260 (executed)
core migrated up to: 015_260_to_270 (executed)
core migrated up to: 016_270_to_280 (executed)
core migrated up to: 016_280_to_300 (executed)
core migrated up to: 017_300_to_310 (executed)
core migrated up to: 018_310_to_320 (executed)
migrating acl on database 'kong'...
acl migrated up to: 000_base_acl (executed)
acl migrated up to: 002_130_to_140 (executed)
acl migrated up to: 003_200_to_210 (executed)
acl migrated up to: 004_212_to_213 (executed)
migrating acme on database 'kong'...
acme migrated up to: 000_base_acme (executed)
acme migrated up to: 001_280_to_300 (executed)
migrating basic-auth on database 'kong'...
basic-auth migrated up to: 000_base_basic_auth (executed)
basic-auth migrated up to: 002_130_to_140 (executed)
basic-auth migrated up to: 003_200_to_210 (executed)
migrating bot-detection on database 'kong'...
bot-detection migrated up to: 001_200_to_210 (executed)
migrating canary on database 'kong'...
canary migrated up to: 001_200_to_210 (executed)
migrating degraphql on database 'kong'...
degraphql migrated up to: 000_base (executed)
migrating graphql-rate-limiting-advanced on database 'kong'...
graphql-rate-limiting-advanced migrated up to: 000_base_gql_rate_limiting (executed)
migrating hmac-auth on database 'kong'...
hmac-auth migrated up to: 000_base_hmac_auth (executed)
hmac-auth migrated up to: 002_130_to_140 (executed)
hmac-auth migrated up to: 003_200_to_210 (executed)
migrating http-log on database 'kong'...
http-log migrated up to: 001_280_to_300 (executed)
migrating ip-restriction on database 'kong'...
ip-restriction migrated up to: 001_200_to_210 (executed)
migrating jwt on database 'kong'...
jwt migrated up to: 000_base_jwt (executed)
jwt migrated up to: 002_130_to_140 (executed)
jwt migrated up to: 003_200_to_210 (executed)
migrating jwt-signer on database 'kong'...
jwt-signer migrated up to: 000_base_jwt_signer (executed)
jwt-signer migrated up to: 001_200_to_210 (executed)
migrating key-auth on database 'kong'...
key-auth migrated up to: 000_base_key_auth (executed)
key-auth migrated up to: 002_130_to_140 (executed)
key-auth migrated up to: 003_200_to_210 (executed)
migrating key-auth-enc on database 'kong'...
key-auth-enc migrated up to: 000_base_key_auth_enc (executed)
key-auth-enc migrated up to: 001_200_to_210 (executed)
migrating konnect-application-auth on database 'kong'...
konnect-application-auth migrated up to: 000_base_konnect_applications (executed)
migrating mtls-auth on database 'kong'...
mtls-auth migrated up to: 000_base_mtls_auth (executed)
mtls-auth migrated up to: 001_200_to_210 (executed)
mtls-auth migrated up to: 002_2200_to_2300 (executed)
migrating oauth2 on database 'kong'...
oauth2 migrated up to: 000_base_oauth2 (executed)
oauth2 migrated up to: 003_130_to_140 (executed)
oauth2 migrated up to: 004_200_to_210 (executed)
oauth2 migrated up to: 005_210_to_211 (executed)
migrating openid-connect on database 'kong'...
openid-connect migrated up to: 000_base_openid_connect (executed)
openid-connect migrated up to: 001_14_to_15 (executed)
openid-connect migrated up to: 002_200_to_210 (executed)
openid-connect migrated up to: 003_280_to_300 (executed)
migrating post-function on database 'kong'...
post-function migrated up to: 001_280_to_300 (executed)
migrating pre-function on database 'kong'...
pre-function migrated up to: 001_280_to_300 (executed)
migrating proxy-cache-advanced on database 'kong'...
proxy-cache-advanced migrated up to: 001_035_to_050 (executed)
migrating rate-limiting on database 'kong'...
rate-limiting migrated up to: 000_base_rate_limiting (executed)
rate-limiting migrated up to: 003_10_to_112 (executed)
rate-limiting migrated up to: 004_200_to_210 (executed)
migrating response-ratelimiting on database 'kong'...
response-ratelimiting migrated up to: 000_base_response_rate_limiting (executed)
migrating session on database 'kong'...
session migrated up to: 000_base_session (executed)
session migrated up to: 001_add_ttl_index (executed)
migrating vault-auth on database 'kong'...
vault-auth migrated up to: 000_base_vault_auth (executed)
vault-auth migrated up to: 001_280_to_300 (executed)
vault-auth migrated up to: 002_300_to_310 (executed)
migrating enterprise on database 'kong'...
enterprise migrated up to: 000_base (executed)
enterprise migrated up to: 006_1301_to_1500 (executed)
enterprise migrated up to: 006_1301_to_1302 (executed)
enterprise migrated up to: 010_1500_to_2100 (executed)
enterprise migrated up to: 007_1500_to_1504 (executed)
enterprise migrated up to: 008_1504_to_1505 (executed)
enterprise migrated up to: 007_1500_to_2100 (executed)
enterprise migrated up to: 009_1506_to_1507 (executed)
enterprise migrated up to: 009_2100_to_2200 (executed)
enterprise migrated up to: 010_2200_to_2211 (executed)
enterprise migrated up to: 010_2200_to_2300 (executed)
enterprise migrated up to: 010_2200_to_2300_1 (executed)
enterprise migrated up to: 011_2300_to_2600 (executed)
enterprise migrated up to: 012_2600_to_2700 (executed)
enterprise migrated up to: 012_2600_to_2700_1 (executed)
enterprise migrated up to: 013_2700_to_2800 (executed)
enterprise migrated up to: 014_2800_to_3000 (executed)
enterprise migrated up to: 015_3000_to_3100 (executed)
enterprise migrated up to: 016_3100_to_3200 (executed)
migrating enterprise.acl on database 'kong'...
enterprise.acl migrated up to: 001_1500_to_2100 (executed)
migrating enterprise.basic-auth on database 'kong'...
enterprise.basic-auth migrated up to: 001_1500_to_2100 (executed)
migrating enterprise.hmac-auth on database 'kong'...
enterprise.hmac-auth migrated up to: 001_1500_to_2100 (executed)
migrating enterprise.jwt on database 'kong'...
enterprise.jwt migrated up to: 001_1500_to_2100 (executed)
migrating enterprise.key-auth on database 'kong'...
enterprise.key-auth migrated up to: 001_1500_to_2100 (executed)
migrating enterprise.key-auth-enc on database 'kong'...
enterprise.key-auth-enc migrated up to: 001_1500_to_2100 (executed)
enterprise.key-auth-enc migrated up to: 002_3100_to_3200 (executed)
enterprise.key-auth-enc migrated up to: 002_2800_to_3200 (executed)
migrating enterprise.mtls-auth on database 'kong'...
enterprise.mtls-auth migrated up to: 001_1500_to_2100 (executed)
enterprise.mtls-auth migrated up to: 002_2200_to_2300 (executed)
migrating enterprise.oauth2 on database 'kong'...
enterprise.oauth2 migrated up to: 001_1500_to_2100 (executed)
enterprise.oauth2 migrated up to: 002_2200_to_2211 (executed)
migrating enterprise.request-transformer-advanced on database 'kong'...
enterprise.request-transformer-advanced migrated up to: 001_1500_to_2100 (executed)
migrating enterprise.response-transformer-advanced on database 'kong'...
enterprise.response-transformer-advanced migrated up to: 001_1500_to_2100 (executed)
103 migrations processed
103 executed
Database is up-to-date
[root@localhost ~]# 
[root@localhost ~]#
[root@localhost ~]# docker run -d --name kong-gateway \
>   --network=kong-net \
>   -e "KONG_DATABASE=postgres" \
>   -e "KONG_PG_HOST=kong-database" \
>   -e "KONG_PG_USER=kong" \
>   -e "KONG_PG_PASSWORD=kongpass" \
>   -e "KONG_PROXY_ACCESS_LOG=/dev/stdout" \
>   -e "KONG_ADMIN_ACCESS_LOG=/dev/stdout" \
>   -e "KONG_PROXY_ERROR_LOG=/dev/stderr" \
>   -e "KONG_ADMIN_ERROR_LOG=/dev/stderr" \
>   -e "KONG_ADMIN_LISTEN=0.0.0.0:8001" \
>   -e "KONG_ADMIN_GUI_URL=http://localhost:8002" \
>   -e KONG_LICENSE_DATA \
>   -p 8000:8000 \
>   -p 8443:8443 \
>   -p 8001:8001 \
>   -p 8444:8444 \
>   -p 8002:8002 \
>   -p 8445:8445 \
>   -p 8003:8003 \
>   -p 8004:8004 \
>   kong/kong-gateway:3.2.1.0
79d104d9375a5a48dc8d3c88c27ac850aed392ba616642a34285fca2982fac86
[root@localhost ~]#  curl -i -X GET --url http://localhost:8001/services
HTTP/1.1 200 OK
Date: Tue, 28 Mar 2023 01:44:53 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Access-Control-Allow-Origin: http://localhost:8002
X-Kong-Admin-Request-ID: iKndHgIERaYNfR1xdqO1DqvJVV8zgXKv
vary: Origin
Access-Control-Allow-Credentials: true
Content-Length: 23
X-Kong-Admin-Latency: 667
Server: kong/3.2.1.0-enterprise-edition

{"next":null,"data":[]}[root@localhost ~]#