-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added SMTP layer #587
Added SMTP layer #587
Conversation
@gconnell what's your take on tcp-protocols? tcp-based protocols don't really fit into the per-packet layer model gopacket uses. There should be a reassembly step in between that merges the packets into a stream which is then processed by the application layer protocol. |
Correct, doing per-packet decoding of stream-based protocols comes with additional problems/complexity. |
I'm happy to add tcp stream decoding, but I think it should be in the context of reassembly/, not layers/. That said, we don't currently have any interfaces or other things built up (like gopacket.Layer*) in reassembly to support starting this, so it'd be a pretty big undertaking. I have had some pushback in the past (and relented) for TLS in particular, which you'll note is a stream-based protocol that's got layer-level support, but that's because 99.99% of the time, it's sent in single packet messages which are in fact decodable (unless fragmented by some layer). In short, I agree that we should not do FTP, SMTP, HTTP, etc. in layers/, but also that we should probably consider how best we can build up a system for supporting that type of contribution in reassembly/. |
Isn't FTP still fundamentally a Layer that can be decoded? I don't see the dependence of these layers on the reassembly package itself. I understand that in order to decode an FTP payload, you must first reassemble the relevant packets in the TCP stream. The way I currently do it is implement the Stream interface with a struct that has a Payload field. Whenever Stream.ReassembledSG is called, I write all of the bytes available from ScatterGather.Fetch() to that field. Then, when ReassemblyComplete is finished, I send a bool value on a channel to StreamFactory.New(), which blocks on a separate goroutine to receive from that ReassemblyComplete channel. This lets me know that the Stream has been fully reassembled and my Stream object contains the entire application payload (if any was present). I can then pass this Stream object to various protocol-specific TCP protocol analyzers, depending on the contexts of the Payload field. This way of decoding TCP streams separates the reassembly process from the layer decoding process. I think that layers like FTP, SMTP, etc should still be placed in the Layers package because the decoding process should be separate from the reassembly process. |
Well yes and no. The problem here is that the layers functionality doesn't really work, since it was built with packets in mind, which is reflected in the interfaces (e.g.,
This would be one way to solve that. But others probably want to decode streams as soon as enough data is available (e.g., the SCADA protocol IEC 60870–5–104 would even make that necessary since it uses endless TCP connections).
I agree, that this shouldn't be part of reassembly (would be nice to not be tied in into a single source interface - so, e.g., one could read a http stream from a txt file). However, it should be different from the layers interface. Something where the decoder can say: I need more data; I consumed x bytes and I'm finished, ... So the input should be some kind of stream, that could be already finished and have all the data, as in your case, and also something that can wait for additional data, so it could be used with reassembly. |
added (basic) unencrypted SMTP decoding and added smtps and submission port with TLS decoder