fix(cube-master): panic if http server fails to start#127
Conversation
yyyang91
commented
Apr 30, 2026
- Fix the conditional logic for port availability detection.
- Add explicit success logs to print the listening address upon startup.
- Add detailed error logs when the HTTP server fails to start.
- Enhance service observability for better troubleshooting.
|
What's the purpose of this change? |
| addr = ":http" | ||
| } | ||
|
|
||
| ln, err := net.Listen("tcp", addr) |
There was a problem hiding this comment.
Does this change work with cubemastercli?
There was a problem hiding this comment.
This logic is ported from the Go standard library (net/http/server.go, v1.24.8).
I migrated the original implementation here to gain more granular control over the startup sequence. Specifically, it allows us to inject the startup/failure logs and port check logic directly into the serving loop, which addresses the observability requirements for cube-master.
There was a problem hiding this comment.
You did not answer my question.
|
|
||
| actualAddr := ln.Addr().String() | ||
| CubeLog.Infof("cube-master listening on on %s", actualAddr) | ||
| stdlog.Printf("cube-master listening on on %s", actualAddr) |
There was a problem hiding this comment.
Why do we need a stdlog?
There was a problem hiding this comment.
This is because:
- CubeLog.Infof only writes logs to a specific business log file /data/log/CubeMaster/cubemaster-req.log
- start_with_pidfile fucniton just redirect stdout/stderr to /var/log/cube-sandbox-one-click/cubemaster.log, we cannot find errors in /var/log/cube-sandbox-one-click/cubemaster.log
There was a problem hiding this comment.
Only the logs under /data/log/CubeMaster/meet the requirements.
There was a problem hiding this comment.
But, the installation logs directed me to the /var/log/cube-sandbox-one-click directory to check for errors, You can view the screenshot. #127 (comment)
There was a problem hiding this comment.
Do you mean just keep stdlog when the server start error? If the server start successfully, just logs to /data/log/CubeMaster/
There was a problem hiding this comment.
How about panic when the requested port is in use?
There was a problem hiding this comment.
I also think a panic is more clearer. I didn't use it because I'm not sure about the project's policy on panicking. Additionally, I found out that CubeLog.Fatal doesn't exit the program.
There was a problem hiding this comment.
How about panic when the requested port is in use?
I think the root cause of the current issue is likely that CubeLog.Errorf does not trigger a panic or exit the program.😅
- Fix the conditional logic for port availability detection. - Add explicit success logs to print the listening address upon startup. - Add detailed error logs when the HTTP server fails to start. - Enhance service observability for better troubleshooting.
83c5952 to
9e7ebd3
Compare
| addr = ":http" | ||
| } | ||
|
|
||
| ln, err := net.Listen("tcp", addr) |
There was a problem hiding this comment.
If the real issue is that the requested port is in use, we can simply panic here.
There was a problem hiding this comment.
It seems that a panic only terminates the program, while a non-panic means the process is still running. To speed up troubleshooting, it's necessary to have logs indicating whether the service started successfully. Should we write success and error messages to `/var/log/cube-sandbox-one-click/cubemaster.log? This is because when the installation script encounters an error, it prompts us to check the files in the directory.
There was a problem hiding this comment.
Panic will log to stderr and should be captured, no?
There was a problem hiding this comment.
Panic will log to stderr and can be captured. but it will print the stack. Is it ok to print the stack?
…ilure logs" This reverts commit 9e7ebd3.
|
I revert the preious commit, and add a commit just panic. |
|
I believe a better solution here would be to introduce a synchronization point before the startup completes. If an issue is detected, trigger a graceful shutdown instead of panicking outright. We should already have a graceful exit mechanism in place. @yyyang91 |
On the master branch, when serverTmp.Run() occur errors would not triger a graceful exit mechanism. CubeSandbox/CubeMaster/cmd/cubemaster/app/main.go Lines 112 to 114 in 09101b2 I revert the commit that can tigger graceful exit yesterday. |
|
When the port is already in use, cube-master hangs silently instead of exiting. The graceful shutdown mechanism is not triggered, so the process just sits there. At the same time, no error is written to /var/log/cube-sandbox-one-click/cubemaster.log — the directory the installation script points you to when something goes wrong. What is the project's expected behavior here? CubeLog alone cannot emit errors to the path the installation script expects, so should the process exit immediately, trigger a graceful shutdown, or is there another mechanism to make startup failures visible in the installation script's diagnostic directory? @chenhengqi @fslongjin |
chenhengqi
left a comment
There was a problem hiding this comment.
I am OK with this change.
