Complete failure of the event system after an attempt to add a hook event

[mister-proger]

Expected Behavior

Working events after adding a web hook and restarting the server

Current Behavior

An error is displayed at the standard startup via the command line:

Error: Unable to initialize notification target(s): unexpected scheme found  (*fmt.wrapError)
       9: internal\logger\logger.go:259:logger.LogIf()
       8: cmd\config-current.go:542:cmd.lookupConfigs()
       7: cmd\config.go:219:cmd.initConfig()
       6: cmd\config.go:190:cmd.(*ConfigSys).Init()
       5: cmd\server-main.go:557:cmd.initConfigSubsystem()
       4: cmd\server-main.go:528:cmd.initServerConfig()
       3: cmd\server-main.go:831:cmd.serverMain.func15()
       2: cmd\server-main.go:489:cmd.bootstrapTrace()
       1: cmd\server-main.go:830:cmd.serverMain()

Full console output:

D:\scripts\startup>minio

D:\scripts\startup>E:\Program\MinIO\minio.exe server "E:\Program\MinIO\data" "D:\S3-data"
Formatting 1st pool, 1 set(s), 2 drives per set.
WARNING: Host local has more than 1 drives of set. A host failure will result in data becoming unavailable.

API: SYSTEM()
Time: 11:17:36 UTC 01/25/2024
DeploymentID: c7711960-fe47-41f6-940d-f580a707e2f9
Error: Unable to initialize notification target(s): unexpected scheme found  (*fmt.wrapError)
       9: internal\logger\logger.go:259:logger.LogIf()
       8: cmd\config-current.go:542:cmd.lookupConfigs()
       7: cmd\config.go:219:cmd.initConfig()
       6: cmd\config.go:190:cmd.(*ConfigSys).Init()
       5: cmd\server-main.go:557:cmd.initConfigSubsystem()
       4: cmd\server-main.go:528:cmd.initServerConfig()
       3: cmd\server-main.go:831:cmd.serverMain.func15()
       2: cmd\server-main.go:489:cmd.bootstrapTrace()
       1: cmd\server-main.go:830:cmd.serverMain()
MinIO Object Storage Server
Copyright: 2015-2024 MinIO, Inc.
License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Version: RELEASE.2024-01-18T22-51-28Z (go1.21.6 windows/amd64)

Status:         2 Online, 0 Offline.
S3-API: http://10.242.223.170:9000  http://26.141.18.68:9000  http://192.168.196.64:9000  http://192.168.196.62:9000  http://192.168.196.60:9000  http://192.168.80.53:9000  http://192.168.56.1:9000  http://127.0.0.1:9000
RootUser: minio
RootPass: password

Console: http://10.242.223.170:64993 http://26.141.18.68:64993 http://192.168.196.64:64993 http://192.168.196.62:64993 http://192.168.196.60:64993 http://192.168.80.53:64993 http://192.168.56.1:64993 http://127.0.0.1:64993
RootUser: minio
RootPass: password

Command-line: https://min.io/docs/minio/linux/reference/minio-mc.html#quickstart
   $ mc alias set 'myminio' 'http://10.242.223.170:9000' 'minio' 'password'

Documentation: https://min.io/docs/minio/linux/index.html

After that, the server turns on normally, only the events do not work. If I try to add a new event, but there will be no error, just please restart so that the changes take effect. After the reboot, the same error occurs, and a new event is not added.

Possible Solution

Most likely, I messed up something while try to adding events

Steps to Reproduce (for bugs)

1. Starting the server and logging into the control panel
2. Adding a new hook event
3. Restarting the server

Context

Before adding the hook event, I tried adding ElasticSearch hooks. But maybe I was doing something wrong, they stubbornly did not want to see the endpoint on the local address. Then, after another reboot, I decided to add just a point of hooks.

Regression

Not

Your Environment

• minio version RELEASE.2024-01-18T22-51-28Z (commit-id=19387cafab76133c2e7642de4aac8c81b9f4f8c7)
• Runtime: go1.21.6 windows/amd64
• Configuration: default

[mister-proger]

Before adding events on hooks, I tried to add PSQL events:

panic: "PUT /minio/admin/v3/set-config-kv": setting PGLOCALEDIR not supported
goroutine 875 [running]:
runtime/debug.Stack()
        runtime/debug/stack.go:24 +0x5e
github.com/minio/minio/cmd.serverMain.func8.setCriticalErrorHandler.func2.1()
        github.com/minio/minio/cmd/generic-handlers.go:555 +0x97
panic({0x2fbb440?, 0xc011cbc9f0?})
        runtime/panic.go:914 +0x21f
github.com/lib/pq.parseEnviron.func2(...)
        github.com/lib/pq@v1.10.9/conn.go:2019
github.com/lib/pq.parseEnviron({0xc00f66aa00, 0x48, 0x3569eef?})
        github.com/lib/pq@v1.10.9/conn.go:2074 +0xd45
github.com/lib/pq.NewConnector({0xc00006084d, 0x28})
        github.com/lib/pq@v1.10.9/connector.go:61 +0x10a
github.com/lib/pq.DialOpen({0x5540ce0?, 0xc000d683f0}, {0xc00006084d?, 0xc011ed2510?})
        github.com/lib/pq@v1.10.9/conn.go:335 +0x31
github.com/lib/pq.Open(...)
        github.com/lib/pq@v1.10.9/conn.go:330
github.com/lib/pq.Driver.Open({}, {0xc00006084d, 0x28})
        github.com/lib/pq@v1.10.9/conn.go:57 +0x85
database/sql.dsnConnector.Connect(...)
        database/sql/sql.go:758
database/sql.(*DB).conn(0xc00133e820, {0x5548db8, 0x6ca8840}, 0x1)
        database/sql/sql.go:1387 +0x717
database/sql.(*DB).PingContext.func1(0xb8?)
        database/sql/sql.go:850 +0x3a
database/sql.(*DB).retry(0x5540b00?, 0xc011ed2800)
        database/sql/sql.go:1538 +0x42
database/sql.(*DB).PingContext(0xc00133e820, {0x5548db8?, 0x6ca8840})
        database/sql/sql.go:849 +0x94
database/sql.(*DB).Ping(...)
        database/sql/sql.go:867
github.com/minio/minio/internal/event/target.(*PostgreSQLTarget).initPostgreSQL(0xc000fc8c80)
        github.com/minio/minio/internal/event/target/postgresql.go:370 +0xe7
github.com/minio/minio/internal/once.(*Init).do(0xc000fc8c80, 0xc000ac34a8?)
        github.com/minio/minio/internal/once/init.go:49 +0x9a
github.com/minio/minio/internal/once.(*Init).Do(...)
        github.com/minio/minio/internal/once/init.go:40
github.com/minio/minio/internal/event/target.(*PostgreSQLTarget).init(...)
        github.com/minio/minio/internal/event/target/postgresql.go:353
github.com/minio/minio/internal/event/target.(*PostgreSQLTarget).IsActive(0xc000fc8c80)
        github.com/minio/minio/internal/event/target/postgresql.go:174 +0x45
github.com/minio/minio/internal/config/notify.TestSubSysNotificationTargets({0x5549100, 0xc00f41e660}, 0xc0000608c0?, {0xc0000608c0, 0xf}, 0xc000e93e40?)
        github.com/minio/minio/internal/config/notify/parse.go:68 +0x34f
github.com/minio/minio/cmd.validateSubSysConfig({0x5549100, 0xc00f41e660}, 0x1e4a1560108?, {0xc0000608c0, 0xf}, {0x5575a28?, 0xc0001d5ec0?})
        github.com/minio/minio/cmd/config-current.go:432 +0x106f
github.com/minio/minio/cmd.validateConfig({0x5549100, 0xc00f41e660}, 0x2fbc440?, {0xc0000608c0, 0xf})
        github.com/minio/minio/cmd/config-current.go:458 +0x131
github.com/minio/minio/cmd.setConfigKV({0x5549100, 0xc0009be270}, {0x5575a28, 0xc0001d5ec0}, {0xc00107a200, 0x92, 0x200})
        github.com/minio/minio/cmd/admin-handlers-config-kv.go:214 +0x1da
github.com/minio/minio/cmd.adminAPIHandlers.SetConfigKVHandler({}, {0x5543800, 0xc011e9a140}, 0xc011a5a400)
        github.com/minio/minio/cmd/admin-handlers-config-kv.go:170 +0x3a6
net/http.HandlerFunc.ServeHTTP(0x5be6bd6?, {0x5543800?, 0xc011e9a140?}, 0x5575a28?)
        net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.adminMiddleware.func1.httpTraceHdrs.httpTrace.func2({0x5543800, 0xc011e9a140}, 0xc011e9a140?)
        github.com/minio/minio/cmd/http-tracer.go:189 +0xef
github.com/minio/minio/cmd.adminMiddleware.func1({0x5543800?, 0xc011e9a140}, 0xc011a5a300)
        github.com/minio/minio/cmd/admin-router.go:132 +0x2ff
net/http.HandlerFunc.ServeHTTP(0x3424720?, {0x5543800?, 0xc011e9a140?}, 0x4?)
        net/http/server.go:2136 +0x29
github.com/klauspost/compress/gzhttp.NewWrapper.func1.1({0x5542ed0, 0xc000c5c540}, 0x1b?)
        github.com/klauspost/compress@v1.17.4/gzhttp/compress.go:501 +0x4fc
net/http.HandlerFunc.ServeHTTP(0xc0009be060?, {0x5542ed0?, 0xc000c5c540?}, 0x0?)
        net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.setBucketForwardingMiddleware.func1({0x5542ed0, 0xc000c5c540}, 0xc011a5a300)
        github.com/minio/minio/cmd/generic-handlers.go:460 +0x2fc
net/http.HandlerFunc.ServeHTTP(0xc?, {0x5542ed0?, 0xc000c5c540?}, 0x10?)
        net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.setUploadForwardingMiddleware.func1({0x5542ed0?, 0xc000c5c540}, 0xc011a5a300)
        github.com/minio/minio/cmd/generic-handlers.go:573 +0x1e6
net/http.HandlerFunc.ServeHTTP(0x343cd80?, {0x5542ed0?, 0xc000c5c540?}, 0x39?)
        net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.setRequestValidityMiddleware.func1({0x5542ed0?, 0xc000c5c540}, 0xc011a5a300)
        github.com/minio/minio/cmd/generic-handlers.go:443 +0x168d
net/http.HandlerFunc.ServeHTTP(0xc0009be120?, {0x5542ed0?, 0xc000c5c540?}, 0xc011bdd518?)
        net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.setRequestLimitMiddleware.func1({0x5542ed0?, 0xc000c5c540}, 0xc011a5a300)
        github.com/minio/minio/cmd/generic-handlers.go:134 +0x662
net/http.HandlerFunc.ServeHTTP(0x3551ea1?, {0x5542ed0?, 0xc000c5c540?}, 0x2b751eb?)
        net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.setCrossDomainPolicyMiddleware.func1({0x5542ed0?, 0xc000c5c540?}, 0x6?)
        github.com/minio/minio/cmd/crossdomain-xml-handler.go:42 +0xf1
net/http.HandlerFunc.ServeHTTP(0xc011a5a300?, {0x5542ed0?, 0xc000c5c540?}, 0xc010e9c2b0?)
        net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.setBrowserRedirectMiddleware.func1({0x5542ed0, 0xc000c5c540}, 0xc011a5a300)
        github.com/minio/minio/cmd/generic-handlers.go:164 +0x19c
net/http.HandlerFunc.ServeHTTP(0x0?, {0x5542ed0?, 0xc000c5c540?}, 0x213ed0c0?)
        net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.setAuthMiddleware.func1({0x5542ed0?, 0xc000c5c540}, 0xc011a5a300)
        github.com/minio/minio/cmd/auth-handler.go:636 +0x6a2
net/http.HandlerFunc.ServeHTTP(0x5549100?, {0x5542ed0?, 0xc000c5c540?}, 0x5521770?)
        net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.httpTracerMiddleware.func1({0x5543470, 0xc00f65c000}, 0xc011a5a100)
        github.com/minio/minio/cmd/http-tracer.go:89 +0x3b4
net/http.HandlerFunc.ServeHTTP(0x32fa500?, {0x5543470?, 0xc00f65c000?}, 0xa?)
        net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.addCustomHeadersMiddleware.func1({0x5543470, 0xc00f65c000}, 0xc0009be0c0?)
        github.com/minio/minio/cmd/generic-handlers.go:538 +0x438
net/http.HandlerFunc.ServeHTTP(0xc011a5a000?, {0x5543470?, 0xc00f65c000?}, 0xfde11f?)
        net/http/server.go:2136 +0x29
github.com/minio/mux.(*Router).ServeHTTP(0xc0001d4fc0, {0x5543470, 0xc00f65c000}, 0xc011a5a000)
        github.com/minio/mux@v1.9.0/mux.go:214 +0x1f7
github.com/minio/minio/cmd.corsHandler.(*Cors).Handler.func2({0x5543470, 0xc00f65c000}, 0xc011a5a000)
        github.com/rs/cors@v1.10.1/cors.go:281 +0x184
net/http.HandlerFunc.ServeHTTP(0xd49088?, {0x5543470?, 0xc00f65c000?}, 0xe?)
        net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.serverMain.func8.setCriticalErrorHandler.func2({0x5543470?, 0xc00f65c000?}, 0x6c3fb00?)
        github.com/minio/minio/cmd/generic-handlers.go:562 +0x78
net/http.HandlerFunc.ServeHTTP(0xc001379ac8?, {0x5543470?, 0xc00f65c000?}, 0x106cde5?)
        net/http/server.go:2136 +0x29
github.com/minio/minio/internal/http.(*Server).Init.func1({0x5543470?, 0xc00f65c000?}, 0xc011cfe010?)
        github.com/minio/minio/internal/http/server.go:123 +0x19c
net/http.HandlerFunc.ServeHTTP(0xd412e5?, {0x5543470?, 0xc00f65c000?}, 0xc00f65c001?)
        net/http/server.go:2136 +0x29
net/http.serverHandler.ServeHTTP({0x553c1d8?}, {0x5543470?, 0xc00f65c000?}, 0x6?)
        net/http/server.go:2938 +0x8e
net/http.(*conn).serve(0xc00109e000, {0x5549100, 0xc0011ca120})
        net/http/server.go:2009 +0x5f4
created by net/http.(*Server).Serve in goroutine 103
        net/http/server.go:3086 +0x5cb

I tried to add ES events, but it just said that the endpoint did not respond (that is, ES does not exist on the endpoint)
I'm not an expert, but it looks very much like ES events go to the address specified in the environment variables, and endpoint monitoring in the control panel turns out to be based on the configuration file.
This explains why the server did not see ES - on the first attempt to add, I mixed up the ports, and it [event] was not added, although the environment variables were created. However, deleting them and restarting the server led to the fact that ES stopped seeing at all.

[klauspost]

lib/pq#414 - seems to be intentional.

Since you don't share the actual settings you use, I can't really help you more.

[mister-proger]

GitHub does not allow attaching .conf files, I renamed and attached them as .txt. This is the file that was downloaded in the control panel in the Configuration tab on the Export button. Just in case, I also attach a link to the disk where everything storage. Is it possible that I'm giving you the wrong configuration that you need?

minio-server-config-01-25-2024-15-37-23.txt

https://disk.yandex.ru/d/HyCYydAd_gduPQ

[harshavardhana]

Just do mc admin config reset alias notify_webhook:CC_test

[mister-proger]

After the command: minio-server-config-01-25-2024-20-51-32.txt

D:\scripts\startup>minio

D:\scripts\startup>E:\Program\MinIO\minio.exe server "E:\Program\MinIO\data" "D:\S3-data"

API: SYSTEM()
Time: 17:51:15 UTC 01/25/2024
DeploymentID: c7711960-fe47-41f6-940d-f580a707e2f9
Error: Unable to initialize notification target(s): unexpected scheme found  (*fmt.wrapError)
       9: internal\logger\logger.go:259:logger.LogIf()
       8: cmd\config-current.go:542:cmd.lookupConfigs()
       7: cmd\config.go:219:cmd.initConfig()
       6: cmd\config.go:190:cmd.(*ConfigSys).Init()
       5: cmd\server-main.go:557:cmd.initConfigSubsystem()
       4: cmd\server-main.go:528:cmd.initServerConfig()
       3: cmd\server-main.go:831:cmd.serverMain.func15()
       2: cmd\server-main.go:489:cmd.bootstrapTrace()
       1: cmd\server-main.go:830:cmd.serverMain()
MinIO Object Storage Server
Copyright: 2015-2024 MinIO, Inc.
License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Version: RELEASE.2024-01-18T22-51-28Z (go1.21.6 windows/amd64)

Status:         2 Online, 0 Offline.
S3-API: http://10.242.223.170:9000  http://26.141.18.68:9000  http://192.168.196.64:9000  http://192.168.196.62:9000  http://192.168.196.60:9000  http://192.168.80.53:9000  http://192.168.56.1:9000  http://127.0.0.1:9000
RootUser: minio
RootPass: password

Console: http://10.242.223.170:56527 http://26.141.18.68:56527 http://192.168.196.64:56527 http://192.168.196.62:56527 http://192.168.196.60:56527 http://192.168.80.53:56527 http://192.168.56.1:56527 http://127.0.0.1:56527
RootUser: minio
RootPass: password

Command-line: https://min.io/docs/minio/linux/reference/minio-mc.html#quickstart
   $ mc alias set 'myminio' 'http://10.242.223.170:9000' 'minio' 'password'

Documentation: https://min.io/docs/minio/linux/index.html

After trying to add a new event of as hook: minio-server-config-01-25-2024-20-55-56.txt

The error remains, events still refuse to be added.