Skip to content

Conversation

@huchaogithup
Copy link
Contributor

@huchaogithup huchaogithup commented Apr 20, 2025

The initial cause of the crash was that vty->buf was null. The crash occurred after executing the function fgets(vty->buf, VTY_BUFSIZ, fp). The root cause was traced back to mgmtd loading /etc/frr/zebra.conf, during which the cmd_exit function was called and attempted to write something to vty->wfd (standard error stream). In daemon mode, this stream points to /dev/null and is read-only, which caused an error. As a result, the buffer vty->buf was closed, leading to the crash.

clashlog:
MGMTD: Received signal 11 at 1745121262 (si_addr 0x0, PC 0x7b13a09988f9); aborting...
MGMTD: zlog_signal+0xf9 7b13a0cd2ec9 7ffce3846830 /usr/lib/frr/libfrr.so.0 (mapped at 0x7b13a0c00000)
MGMTD: core_handler+0xb5 7b13a0d13715 7ffce3846970 /usr/lib/frr/libfrr.so.0 (mapped at 0x7b13a0c00000)
MGMTD: __sigaction+0x50 7b13a0845330 7ffce3846ac0 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7b13a0800000)
MGMTD: ---- signal ----
MGMTD: __nss_database_lookup+0x10919 7b13a09988f9 7ffce38477c8 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7b13a0800000)
MGMTD: _IO_getline_info+0x124 7b13a0886ff4 7ffce38477d0 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7b13a0800000)
MGMTD: fgets+0xa4 7b13a0885bd4 7ffce3847830 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7b13a0800000)
MGMTD: config_from_file+0x45 7b13a0c9f6d5 7ffce3847860 /usr/lib/frr/libfrr.so.0 (mapped at 0x7b13a0c00000)
MGMTD: mgmt_vty_read_configs+0x16c 7b13a0d3263c 7ffce38478a0 /usr/lib/frr/libfrr.so.0 (mapped at 0x7b13a0c00000)
MGMTD: event_call+0xae 7b13a0d269be 7ffce3848910 /usr/lib/frr/libfrr.so.0 (mapped at 0x7b13a0c00000)
MGMTD: frr_run+0xc8 7b13a0cc9bd8 7ffce3848a40 /usr/lib/frr/libfrr.so.0 (mapped at 0x7b13a0c00000)
MGMTD: main+0x157 597895ef2b57 7ffce3848b60 /usr/lib/frr/mgmtd (mapped at 0x597895edd000)
MGMTD: __libc_init_first+0x8a 7b13a082a1ca 7ffce3848ba0 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7b13a0800000)
MGMTD: __libc_start_main+0x8b 7b13a082a28b 7ffce3848c40 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7b13a0800000)
MGMTD: _start+0x25 597895ef3045 7ffce3848ca0 /usr/lib/frr/mgmtd (mapped at 0x597895edd000)
MGMTD: in thread mgmt_config_read_in scheduled from mgmtd/mgmt_vty.c:655 mgmt_vty_init()

#18140

@frrbot frrbot bot added the mgmt FRR Management Infra label Apr 20, 2025
@donaldsharp
Copy link
Member

Please put the analysis into the commit message itself. Without it being in the commit message this is going no-where

@huchaogithup huchaogithup changed the title mgmtd: Fix the null pointer exception issue when mgmtd reads vty->buf. mgmtd: Fixed bug,In daemon mode, this stream points to /dev/null and is read-only, which caused an error. As a result, the buffer vty->buf was closed, leading to the crash. Apr 21, 2025
@huchaogithup
Copy link
Contributor Author

Please put the analysis into the commit message itself. Without it being in the commit message this is going no-where

Is this modification okay?

…is read-only, which caused an error. As a result, the buffer vty->buf was closed, leading to the crash.

clashlog:
MGMTD: Received signal 11 at 1745121262 (si_addr 0x0, PC 0x7b13a09988f9); aborting...
MGMTD: zlog_signal+0xf9 7b13a0cd2ec9 7ffce3846830 /usr/lib/frr/libfrr.so.0 (mapped at 0x7b13a0c00000)
MGMTD: core_handler+0xb5 7b13a0d13715 7ffce3846970 /usr/lib/frr/libfrr.so.0 (mapped at 0x7b13a0c00000)
MGMTD: __sigaction+0x50 7b13a0845330 7ffce3846ac0 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7b13a0800000)
MGMTD: ---- signal ----
MGMTD: __nss_database_lookup+0x10919 7b13a09988f9 7ffce38477c8 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7b13a0800000)
MGMTD: _IO_getline_info+0x124 7b13a0886ff4 7ffce38477d0 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7b13a0800000)
MGMTD: fgets+0xa4 7b13a0885bd4 7ffce3847830 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7b13a0800000)
MGMTD: config_from_file+0x45 7b13a0c9f6d5 7ffce3847860 /usr/lib/frr/libfrr.so.0 (mapped at 0x7b13a0c00000)
MGMTD: mgmt_vty_read_configs+0x16c 7b13a0d3263c 7ffce38478a0 /usr/lib/frr/libfrr.so.0 (mapped at 0x7b13a0c00000)
MGMTD: event_call+0xae 7b13a0d269be 7ffce3848910 /usr/lib/frr/libfrr.so.0 (mapped at 0x7b13a0c00000)
MGMTD: frr_run+0xc8 7b13a0cc9bd8 7ffce3848a40 /usr/lib/frr/libfrr.so.0 (mapped at 0x7b13a0c00000)
MGMTD: main+0x157 597895ef2b57 7ffce3848b60 /usr/lib/frr/mgmtd (mapped at 0x597895edd000)
MGMTD: __libc_init_first+0x8a 7b13a082a1ca 7ffce3848ba0 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7b13a0800000)
MGMTD: __libc_start_main+0x8b 7b13a082a28b 7ffce3848c40 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7b13a0800000)
MGMTD: _start+0x25 597895ef3045 7ffce3848ca0 /usr/lib/frr/mgmtd (mapped at 0x597895edd000)
MGMTD: in thread mgmt_config_read_in scheduled from mgmtd/mgmt_vty.c:655 mgmt_vty_init()

Signed-off-by: huachao01 <huachao01@corp.netease.com>
@vayetze vayetze requested a review from choppsv1 April 22, 2025 15:20
@donaldsharp
Copy link
Member

I do not think that this is the correct approach to fixing this problem. Opening /dev/null as RW is not something I think we should be doing. @eqvinox opinions here?

@huchaogithup
Copy link
Contributor Author

I do not think that this is the correct approach to fixing this problem. Opening /dev/null as RW is not something I think we should be doing. @eqvinox opinions here?

I think I more or less understand what you mean now. In the case of systemctl start frr or running docker run -d quay.io/frrouting/frr:10.2.1, it's true that only the standard input stream is kept as /dev/null, while the standard output and error streams are typically replaced by pipes.
image
or
image

I might have encountered the issue because I manually started /usr/lib/frr/mgmtd -d -F traditional -A 127.0.0.1 or sudo containerlab deploy -t frr.yaml while trying to troubleshoot a different problem.
image
Because they both use stty.

image

@github-actions
Copy link

This PR is stale because it has been open 180 days with no activity. Comment or remove the autoclose label in order to avoid having this PR closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants