Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add cmds/core/netstat #2976

Open
wants to merge 21 commits into
base: main
Choose a base branch
from
Open

Add cmds/core/netstat #2976

wants to merge 21 commits into from

Conversation

ChriMarMe
Copy link
Contributor

@ChriMarMe ChriMarMe commented Apr 30, 2024 •
edited

Snapshot of status quo netstat implementation in u-root compared to net-tools/netstat

netstat has five main capabilities:

  • List socket states
  • List route table (flags: -r, -r4,-r6,-r46)
  • List multicast group membership (flags: -g, -g4, -g6, -g46)
  • Print interface information(s) (flags: -i, -I=)
  • Print network statistics (flags: -s, -s4, -s6)

Status of implementation:

  • printing of sockets (tcp, udp, raw for IPv4/IPv6, unix) (netstat -4, netstat -6, netstat -46, netstat -r, netstat -x)
  • printing of routing table (IPv4, IPv6) (netstat -r, netstat -r4, netstat -r6, netstat -r46)
  • printing of statistics (IPv4, IPv6) (netstat -s, netstat -s4, netstat -s6)
  • printing of multicast group memberships (IPv4, IPv6) (netstat -g, netstat -g4, netstat -g6, netstat -g46)
  • printing of interfaces (netstat -i)
  • printing of single interface (netstat -I="ifacename")
  • flag --tcp, --udp (for IPv4, IPv6)
  • flag --all (display all connections, default: connected)
  • flag --verbose, -v (Provides errors for unsupported address families of the kernel)
  • --wide (don't truncate IP addresses)
  • --extend(display other/more information)
  • --programs (display PID/Program name for sockets)
  • --timers (display timers)
  • --listening (display listening server sockets)
  • --continuous (continuous listing)
  • --fib (display Forwarding Information Base (default))
  • --cache (display routing cache instead of FIB)
  • --numeric (don't resolve names)
  • --numeric-host (don't resolve host names)
  • --numeric-user (don't resolve user names)
  • --numeric-port (don't resolve port names)
  • --symbolic (resolve hardware names)
  • support socket type tcp
  • support socket type udp
  • support socket type raw
  • support socket type unix
  • support socket type ax25
  • support socket type sctp
  • support socket type ipx
  • support socket type netrom

@ChriMarMe ChriMarMe force-pushed the add/netstat branch 7 times, most recently from 28ca4ad to 6e8b4f7 Compare April 30, 2024 11:45
@jensdrenhaus jensdrenhaus mentioned this pull request Apr 30, 2024
Open
@jensdrenhaus
Copy link
Member

What socket types and address families should be supported?

Socket types:

  • raw
  • unix
  • tcp
  • udp
  • ax25
  • sctp (Stream Control Transmission Protocol)
  • ipx
  • netrom

Address families:

  • IPv4
  • IPv6
  • ax25 (Amateur radio AX.25 protocol)
  • netrom (AMPR NET/ROM)
  • ipx (Novell IPX)
  • ddp(Appletalk DDP)
  • x25 (ITU-T X.25 / ISO-8208 protocol)

@ChriMarMe evaluated the socket type usage as follows:
raw
raw sockets allow the manipulation of payload and header for the given address family in layer 2 (data-link-layer) or layer 3 (network-layer)
https://stackoverflow.com/questions/14774668/what-is-raw-socket-in-socket-programming
This is a common, broadly used socket type.

unix
unix sockets provide a mechanism for inter-process communication defined in POSIX standard and implemented in the linux kernel.
This is a common, broadly used socket type.

tcp/udp
sockets of type tcp/udp usually used on top of IPv4/IPv6 address families.
This is a common, broadly used socket type

ax25
Broadly used in packet radio application
IIuc: Drop-in replacement for ethernet for a special use case (digital radio stuff) There is no usage in u-root so far.
sctp
The “Stream Control Transmission Protocol” is a transport layer protocol and provides UDP-like connections with TCP reliability build-in.
There is no usage in u-root so far.

ipx
Proprietary protocol. Works like IP/UDP. Popular in the past, not so much anymore.
Does not see usage in u-root

netrom
netrom is a protocol used extensively by radio amateurs. The Linux netrom protocol family permits access to these protocols via the standard networking socket metaphor.
Does not see use in u-root

Proposal:
Processing of the RAW and UNIX socket type data is a viable thing to do. Others do not see any usage in u-root nor vendored dependencies. We suggest implementing RAW and UNIX socket type data processing and skipping the rest. Same goes for the address families linked to the skipped socket types. (ax25,x25,ddp,ipx, netrom)

@jensdrenhaus
Copy link
Member

cc @10000TB @andrewsun2898

@ChriMarMe ChriMarMe force-pushed the add/netstat branch 2 times, most recently from 38f6fd7 to 4375dfa Compare May 2, 2024 11:21
rminnich
rminnich reviewed May 2, 2024
View reviewed changes
Copy link
Member

@rminnich rminnich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is pretty amazing. See my one comment. You might put the info on what is done, and what is not done, in doc.go

I would be ready to approve as it is now if you want.

pkg/netstat/route.go Outdated Show resolved Hide resolved
@rminnich rminnich added the Awaiting author Waiting for new changes or feedback for author. label May 2, 2024
@ChriMarMe ChriMarMe force-pushed the add/netstat branch 5 times, most recently from f26af0f to 29d9aed Compare May 7, 2024 11:20
@ChriMarMe
Copy link
Contributor Author

ChriMarMe commented May 8, 2024 •
edited

When implementing the --cache option for routing tables of IPv6, I encountered legacy compatibility for really old kernel versions. There is a file opened and if that fails, it falls back to a the -fib function but with a filter passed to it.
I asked the maintainer of net-tools what this is about. See: ecki/net-tools#34
Here we have some additional informations about routing cache:
https://serverfault.com/questions/1091128/why-i-get-cache-in-the-output-of-ip-route-get
https://workshop.netfilter.org/2013/wiki/images/2/2a/DaveM_route_cache_removed_nfws2013.pdf

So my takeaway is that routing cache was removed for IPv4/6 some time ago already and the --cache-flag is dragged along for legacy reasons.

For now I will skip the implementation of --cache. Let me know what you think @10000TB @andrewsun2898

@ChriMarMe ChriMarMe force-pushed the add/netstat branch 5 times, most recently from 16e4b6d to e69dd3d Compare May 15, 2024 13:17
@10000TB
Copy link
Member

10000TB commented May 15, 2024 •
edited

@ChriMarMe @jensdrenhaus Hi, srry for the delay! -- re. socket types and address family.

I mostly agree with the assesment put by @jensdrenhaus , except we would like TCP/UDP included too. I think that is beneficial to include. So the requirements from my end are as follows.

Socket types:

  • Required

    • Raw
    • Unix
    • Tcp
    • Udp

  • Not Required

    • ax25
    • sctp
    • Ipx
    • Netrom

Address Family

  • Required

    • IPv4
    • IPv6

  • Not Required

    • ax25 (Amateur radio AX.25 protocol)
    • netrom (AMPR NET/ROM)
    • ipx (Novell IPX)
    • ddp(Appletalk DDP)
    • x25 (ITU-T X.25 / ISO-8208 protocol)

@10000TB
Copy link
Member

10000TB commented May 15, 2024

When implementing the --cache option for routing tables of IPv6, I encountered legacy compatibility for really old kernel versions. There is a file opened and if that fails, it falls back to a the -fib function but with a filter passed to it. I asked the maintainer of net-tools what this is about. See: ecki/net-tools#34 Here we have some additional informations about routing cache: https://serverfault.com/questions/1091128/why-i-get-cache-in-the-output-of-ip-route-get https://workshop.netfilter.org/2013/wiki/images/2/2a/DaveM_route_cache_removed_nfws2013.pdf

So my takeaway is that routing cache was removed for IPv4/6 some time ago already and the --cache-flag is dragged along for legacy reasons.

For now I will skip the implementation of --cache. Let me know what you think @10000TB @andrewsun2898

In that case, how do we display routing cache ? -- and how do we different route from cache versus new lookup need happen ?

I tried on my 6.6 kernel devbox, which has netstat 2.10

xuehaohu@worldpeace:~$ netstat --version
net-tools 2.10
Fred Baumgarten, Alan Cox, Bernd Eckenfels, Phil Blundell, Tuan Hoang, Brian Micek and others
+NEW_ADDRT +RTF_IRTT +RTF_REJECT +FW_MASQUERADE +I18N +SELINUX
AF: (inet) +UNIX +INET +INET6 +IPX +AX25 +NETROM +X25 +ATALK +ECONET +ROSE -BLUETOOTH
HW:  +ETHER +ARC +SLIP +PPP +TUNNEL -TR +AX25 +NETROM +X25 +FR +ROSE +ASH +SIT +FDDI +HIPPI +HDLC/LAPB +EUI64
xuehaohu@worldpeace:~$ uname -a
Linux 6.6.15-2rodete2-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.6.15-2rodete2 (2024-03-19) x86_64 GNU/Linux
xuehaohu@worldpeace:

xuehaohu@worldpeace:~$ netstat --help
...
...
        -C, --cache              display routing cache instead of FIB

xuehaohu@worldpeace:~$ netstat -6 -C
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp6       0      0 localhost:52868         localhost:9557          ESTABLISHED
tcp6       0      0 localhost:46904         localhost:9557          ESTABLISHED
tcp6       0      0 localhost:57110         localhost:9557          ESTABLISHED
tcp6       0      0 (redacted):49230     (redacted):5031 ESTABLISHED
tcp6       0      0 localhost:57150         localhost:9557          ESTABLISHED
tcp6       0      0 (redacted):35642     (redacted):5031 ESTABLISHED
tcp6       0      0 localhost:41834         localhost:9557          ESTABLISHED
tcp6       0      0 (redacted):58684     (redacted):5031    ESTABLISHED

@ChriMarMe
Copy link
Contributor Author

@ChriMarMe @jensdrenhaus Hi, srry for the delay! -- re. socket types and address family.

I mostly agree with the assesment put by @jensdrenhaus , except we would like TCP/UDP included too. I think that is beneficial to include. So the requirements from my end are as follows.

Socket types:

* Required
  
  * Raw
  * Unix
  * Tcp
  * Udp

* Not Required
  
  * ax25
  * sctp
  * Ipx
  * Netrom

Address Family

* Required
  
  * IPv4
  * IPv6

* Not Required
  
  * ax25 (Amateur radio AX.25 protocol)
  * netrom (AMPR NET/ROM)
  * ipx (Novell IPX)
  * ddp(Appletalk DDP)
  * x25 (ITU-T X.25 / ISO-8208 protocol)

Thanks for the clarification. I didnt mention TCP/UDP and IPv4/IPv6 because I assumed it as required already :)

When implementing the --cache option for routing tables of IPv6, I encountered legacy compatibility for really old kernel versions. There is a file opened and if that fails, it falls back to a the -fib function but with a filter passed to it. I asked the maintainer of net-tools what this is about. See: ecki/net-tools#34 Here we have some additional informations about routing cache: https://serverfault.com/questions/1091128/why-i-get-cache-in-the-output-of-ip-route-get https://workshop.netfilter.org/2013/wiki/images/2/2a/DaveM_route_cache_removed_nfws2013.pdf
So my takeaway is that routing cache was removed for IPv4/6 some time ago already and the --cache-flag is dragged along for legacy reasons.
For now I will skip the implementation of --cache. Let me know what you think @10000TB @andrewsun2898

In that case, how do we display routing cache ? -- and how do we different route from cache versus new lookup need happen ?

I tried on my 6.6 kernel devbox, which has netstat 2.10

xuehaohu@worldpeace:~$ netstat --version
net-tools 2.10
Fred Baumgarten, Alan Cox, Bernd Eckenfels, Phil Blundell, Tuan Hoang, Brian Micek and others
+NEW_ADDRT +RTF_IRTT +RTF_REJECT +FW_MASQUERADE +I18N +SELINUX
AF: (inet) +UNIX +INET +INET6 +IPX +AX25 +NETROM +X25 +ATALK +ECONET +ROSE -BLUETOOTH
HW:  +ETHER +ARC +SLIP +PPP +TUNNEL -TR +AX25 +NETROM +X25 +FR +ROSE +ASH +SIT +FDDI +HIPPI +HDLC/LAPB +EUI64
xuehaohu@worldpeace:~$ uname -a
Linux 6.6.15-2rodete2-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.6.15-2rodete2 (2024-03-19) x86_64 GNU/Linux
xuehaohu@worldpeace:

xuehaohu@worldpeace:~$ netstat --help
...
...
        -C, --cache              display routing cache instead of FIB

xuehaohu@worldpeace:~$ netstat -6 -C
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp6       0      0 localhost:52868         localhost:9557          ESTABLISHED
tcp6       0      0 localhost:46904         localhost:9557          ESTABLISHED
tcp6       0      0 localhost:57110         localhost:9557          ESTABLISHED
tcp6       0      0 (redacted):49230     (redacted):5031 ESTABLISHED
tcp6       0      0 localhost:57150         localhost:9557          ESTABLISHED
tcp6       0      0 (redacted):35642     (redacted):5031 ESTABLISHED
tcp6       0      0 localhost:41834         localhost:9557          ESTABLISHED
tcp6       0      0 (redacted):58684     (redacted):5031    ESTABLISHED

Will have a look at it again. Prolly misunderstood some of the information.

@ChriMarMe
Copy link
Contributor Author

When implementing the --cache option for routing tables of IPv6, I encountered legacy compatibility for really old kernel versions. There is a file opened and if that fails, it falls back to a the -fib function but with a filter passed to it. I asked the maintainer of net-tools what this is about. See: ecki/net-tools#34 Here we have some additional informations about routing cache: https://serverfault.com/questions/1091128/why-i-get-cache-in-the-output-of-ip-route-get https://workshop.netfilter.org/2013/wiki/images/2/2a/DaveM_route_cache_removed_nfws2013.pdf
So my takeaway is that routing cache was removed for IPv4/6 some time ago already and the --cache-flag is dragged along for legacy reasons.
For now I will skip the implementation of --cache. Let me know what you think @10000TB @andrewsun2898

In that case, how do we display routing cache ? -- and how do we different route from cache versus new lookup need happen ?

I tried on my 6.6 kernel devbox, which has netstat 2.10

xuehaohu@worldpeace:~$ netstat --version
net-tools 2.10
Fred Baumgarten, Alan Cox, Bernd Eckenfels, Phil Blundell, Tuan Hoang, Brian Micek and others
+NEW_ADDRT +RTF_IRTT +RTF_REJECT +FW_MASQUERADE +I18N +SELINUX
AF: (inet) +UNIX +INET +INET6 +IPX +AX25 +NETROM +X25 +ATALK +ECONET +ROSE -BLUETOOTH
HW:  +ETHER +ARC +SLIP +PPP +TUNNEL -TR +AX25 +NETROM +X25 +FR +ROSE +ASH +SIT +FDDI +HIPPI +HDLC/LAPB +EUI64
xuehaohu@worldpeace:~$ uname -a
Linux 6.6.15-2rodete2-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.6.15-2rodete2 (2024-03-19) x86_64 GNU/Linux
xuehaohu@worldpeace:

xuehaohu@worldpeace:~$ netstat --help
...
...
        -C, --cache              display routing cache instead of FIB

xuehaohu@worldpeace:~$ netstat -6 -C
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp6       0      0 localhost:52868         localhost:9557          ESTABLISHED
tcp6       0      0 localhost:46904         localhost:9557          ESTABLISHED
tcp6       0      0 localhost:57110         localhost:9557          ESTABLISHED
tcp6       0      0 (redacted):49230     (redacted):5031 ESTABLISHED
tcp6       0      0 localhost:57150         localhost:9557          ESTABLISHED
tcp6       0      0 (redacted):35642     (redacted):5031 ESTABLISHED
tcp6       0      0 localhost:41834         localhost:9557          ESTABLISHED
tcp6       0      0 (redacted):58684     (redacted):5031    ESTABLISHED

BTW: The output you have here is not route cache. It's IPv6 sockets listing.

@rminnich rminnich added Awaiting author Waiting for new changes or feedback for author. and removed Awaiting reviewer Waiting for a reviewer. labels May 24, 2024
@ChriMarMe
Copy link
Contributor Author

I hate to be picky, but the way this is written, it's a lot of code needing tests.

A friend at meta has had good luck with asking chatgpt to write tests for Go.

Being picky is what helps me being a better dev. So thank you :)

@ChriMarMe ChriMarMe force-pushed the add/netstat branch 2 times, most recently from eff2190 to 2c981c2 Compare May 24, 2024 08:12
@ChriMarMe ChriMarMe requested a review from rminnich May 27, 2024 06:26
@ChriMarMe ChriMarMe added Awaiting reviewer Waiting for a reviewer. and removed Awaiting author Waiting for new changes or feedback for author. labels May 27, 2024
@ChriMarMe
Netstat:
9099c21 
* Interface table printing
* specific Interface printing

Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
* Add processing of output format flags
738d5ac 
* Add structure and member-functions to handle the different flag mechanism

Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Add multicast group membership printing
1d97ce5 
Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Add Statistics printing for IPv4 and IPv6
93fcfe1 
Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Add route printing capability for IPv4 and IPv6
a53db79 
Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Add socket printing for IPv4/IPv6 (TCP, UDP, RAW) and Unix sockets
ae58aff 
Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Add some coverage tests
4fc4aa4 
Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Implement --cache for IPv6 and adapt tests accordingly
2709a81 
Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Add doc.md with report about implemented flags and some additional in…
3c90062 
…formation

regarding route cache printing for IPv4 and IPv6

Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Fix formatting error in output for IPv6 route printing
9d065b8 
Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Add some coverage tests for output/format
ac9a6a3 
Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Add _linux to cmds/core/netstat/netstat.go
fdb6166 
Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Change Socket interface function PrintSockets to SocketsString:
be84bed 
* now returns a string to the caller. Allows for more control where printing happens

Change AddressFamily interface function PrintRoutes to RoutesFormatString
* now returns a string to the caller. Allows for more control where printing happens

Add ClearOutput function to AddressFamily interface and implement for IPv4/6
* Allows to reset the output string builder. Required for continuous printing

Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Adapt tests according to latest changes to interfaces
d304ce5 
Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
fix parseIPv6Flags
3a6e94d 
Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Fix inode/progname printing
c8d7fb5 
Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
cleaned up evalFlags function to return errors up to calling main fun…
a626224 
…ction as it should be done in the first place

Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Add coverage tests for netstat command
bc1b7a6 
Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Rename evalFlags function to run
77edb60 
Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Rework flag parsing:
46f8325 
* Implement flag struct and move parsing into main function
* Add io.Writer to run function to have more control over output (important for tests)

Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
@ChriMarMe
Add tests to cmds/core/netstat
99e6ebb 
Signed-off-by: Christopher Meis <christopher.meis@9elements.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@10000TB 10000TB Awaiting requested review from 10000TB

@andrewsun2898 andrewsun2898 Awaiting requested review from andrewsun2898

@rminnich rminnich Awaiting requested review from rminnich

At least 1 approving review is required to merge this pull request.

Assignees

@ChriMarMe ChriMarMe

Labels
Awaiting reviewer Waiting for a reviewer.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@ChriMarMe @jensdrenhaus @10000TB @rminnich