Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support packet capture of multiple interfaces simultaneously and support NetScaler capture format extra info #2871

Closed
KyferEz opened this issue Nov 3, 2018 · 16 comments
Assignees
Labels
feature Adding new functionality
Milestone

Comments

@KyferEz
Copy link

KyferEz commented Nov 3, 2018

We need the ability to capture traces of multiple interfaces simultaneously. Currently the option to capture a trace only allows the ability to select one interface at a time. Even if you open multiple tabs and try to take 2 traces at the same time, this doesn't work and you only end up with the last trace you took. When troubleshooting Asynchronous routes, this is an issue and one must resort to CLI and using TCPdump.

Also in these sorts of traces, it is beneficial to see if the packet is RX or TX, and what Interface and VLAN the packet was on. Standard packet captures do not show this, however NetScaler adds support in the protocol for another set of information that is very useful. If support for this can also be added that would make firewall traces far easier. Below image is a screenshot showing the extra fields such as Operation, VLAN, and NIC Number.

examplenstrace

.
@KyferEz KyferEz changed the title Support packet capture of multiple interfaces simultaneously and support NetScaler capture format features Support packet capture of multiple interfaces simultaneously and support NetScaler capture format extra info Nov 3, 2018
@AdSchellevis
Copy link
Member

Underneath it's using tcpdump to capture the data, so you could easily check the possibilities command line. To capture more interfaces at once, we probably would have to allow starting multiple tcpdump's at once, although I'm not sure.

If you can test command line and share the results, that would help extending the existing feature.

@KyferEz
Copy link
Author

KyferEz commented Nov 3, 2018

Are you referring to capturing multiple interfaces? If so, yes, that works.

If you are referring to capturing the extra data such as NIC number, VLAN, Rx/Tx, then I'm not sure as NetScaler uses some custom command for creating the traces, and if you use TCPdump it doesn't get that data. I could send you a trace with the extra data if you want to look at it's format. Wireshark reads it so I'm sure the format must be supported by some library.

@AdSchellevis
Copy link
Member

If tcpdump won't support the output, it's unlikely that it would get in our product (I doubt some of that information is anywhere in the flow to be honest).

That would leave only the request to capture multiple interfaces at once.

@fichtner fichtner self-assigned this Nov 3, 2018
@fichtner fichtner added the feature Adding new functionality label Nov 3, 2018
@fichtner fichtner added this to the 19.1 milestone Nov 3, 2018
@fichtner
Copy link
Member

fichtner commented Nov 3, 2018

assigning myself as promised

@KyferEz
Copy link
Author

KyferEz commented Nov 3, 2018

Perhaps tcpdump developers could be asked what is achievable for those extra fields? I know they can tell which interface it was captured on. I also know the newer versions of TCPDump support filtering by direction, so they can identify Tx/Rx; we just need TCPDump to support the header injection into the captured packet for Wiresheak to read it.

@AdSchellevis
Copy link
Member

Maybe, was just reading a bit about it, it's proprietary, but quite useful to have indeed

@AdSchellevis
Copy link
Member

small question, where did the pcap file come from in the sample? did wireshark capture that info?

@KyferEz
Copy link
Author

KyferEz commented Nov 3, 2018

NetScaler captured the data in the screenshot. Wireshark just understands and displays it.

@AdSchellevis
Copy link
Member

I kind of expect the data isn't actually in the packet, but inserted by the netscaler. The pcap from tcpdump should already be complete if I'm not mistaken.

@KyferEz
Copy link
Author

KyferEz commented Nov 3, 2018

That may be the case, I do not know. But even if it is, that just means TCPdump should be able to easily insert the data given some code and a switch to do so.

@AdSchellevis
Copy link
Member

How would it do that?

@KyferEz
Copy link
Author

KyferEz commented Nov 3, 2018

"Given some code and a switch to do so" - I am assuming with modification by the devs. It's already writing the packets to a pcap format to file, it surely could be modified to add a header to the packet with the extra info.

Of course this is for TCPdump devs; though perhaps coming from another dev the request might have more weight? That's the only reason I haven't looked at putting a request in for TCPdump yet.

@AdSchellevis
Copy link
Member

My point is what information should it pull from where according to which standard? If it's not in the packet, it sounds quite impossible to guess what the netscaler knows.... but maybe I'm missing something. Which information are you looking for specifically? (and is it something the firewall could know)

@KyferEz
Copy link
Author

KyferEz commented Nov 3, 2018

Not sure if I'm following you, but what I'm thinking is that TCPDump should already know the info I want, which is primarily Interface and direction.

So let's assume you tell TCPDump to capture all interfaces. It already knows what interfaces are on your system, otherwise it can't capture all interfaces. So therefore, it should know which interface the packet was captured on. It should also know if that packet was in the RX buffer or TX buffer of that interface. So it can take what it knows and append it to the packet before writing it out.

@AdSchellevis
Copy link
Member

Maybe you better try your ideas here https://www.tcpdump.org , we’ll try to fix the multiple interfaces as mentioned earlier.

@KyferEz
Copy link
Author

KyferEz commented Nov 3, 2018

Found this. Some I didn't follow, but the last comment looks like they achieved showing both the Interface and Direction: the-tcpdump-group/libpcap#127

@fichtner fichtner modified the milestones: 19.1, 19.7 Jan 20, 2019
@fichtner fichtner modified the milestones: 19.7, 20.1 Jul 1, 2019
@AdSchellevis AdSchellevis assigned AdSchellevis and unassigned fichtner Jul 7, 2019
AdSchellevis added a commit that referenced this issue Jul 8, 2019
It's a short term solution, it would be better to refactor the legacy page and use configd calls, but since the "any" keyword wasn't possible in fbsd, it seemed like a good idea to allow multiple selections.

Changes in this commit:

- start tcpdump captures to files using /tmp/packetcapture_[INTERFACE].cap in stead of the single file before
- refactor "remove" action to delete all /tmp/packetcapture_*.cap
- refactor "view" action to iterate over all /tmp/packetcapture_*.cap files and return a named array per interface
- change download action to point to the actual filename and return content of found.
@fichtner fichtner modified the milestones: 20.1, 19.7 Jul 9, 2019
EugenMayer pushed a commit to KontextWork/opnsense_core that referenced this issue Jul 22, 2019
It's a short term solution, it would be better to refactor the legacy page and use configd calls, but since the "any" keyword wasn't possible in fbsd, it seemed like a good idea to allow multiple selections.

Changes in this commit:

- start tcpdump captures to files using /tmp/packetcapture_[INTERFACE].cap in stead of the single file before
- refactor "remove" action to delete all /tmp/packetcapture_*.cap
- refactor "view" action to iterate over all /tmp/packetcapture_*.cap files and return a named array per interface
- change download action to point to the actual filename and return content of found.
EugenMayer pushed a commit to KontextWork/opnsense_core that referenced this issue Jul 22, 2019
It's a short term solution, it would be better to refactor the legacy page and use configd calls, but since the "any" keyword wasn't possible in fbsd, it seemed like a good idea to allow multiple selections.

Changes in this commit:

- start tcpdump captures to files using /tmp/packetcapture_[INTERFACE].cap in stead of the single file before
- refactor "remove" action to delete all /tmp/packetcapture_*.cap
- refactor "view" action to iterate over all /tmp/packetcapture_*.cap files and return a named array per interface
- change download action to point to the actual filename and return content of found.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Adding new functionality
Development

No branches or pull requests

3 participants