Capture network traffic in a .pcap file

[marty90]

Hi,
I think a useful feature could be to create the .pcap file containing the network traffic exchanged by the browser and the server, like webpagetest does.
It should not be difficult to implement. Just start tcpdump before starting the browser, and close it immediately after.

[worenga]

I like the idea for pure convenience and for having the feature to further analyze page loads on a deeper level. It may help future analysis and the development of novel insights.

However, it will introduce browsertime into the packet capture business and will create a lot of new things we may need to worry about in the future, if we support this. I am a bit worried that it is more than simply starting/stopping tcpdump. e.g. for inspecting layer 7 we may need the encryption keys used for https, browsers do handle that differently.

[marty90]

Capturing the raw .pcap file should be simple. WebPageTest uses WinPcap just to read packets from an interface and dump them to a file (here). TcpDump works in the same way.
Having the raw Pcap file would be interesting to see TCP related information, or data about TLS handshake.

I agree that having the TLS keylog would be useful to decrypt the stream. That is simple for Chrome and Firefox (just set an environment variable), but for mobile could be hard. That could be left as future work, as the raw PCAP file already is useful.

[soulgalore]

I'll be busy with preparing the coming sitespeed.io release the coming weeks, but if you have time @marty90 to do a PR of what it would look like, it would be interesting!

[marty90]

I can try to implement it ;)
I may create a class tcpdump in lib/support used by the method run of the class Engine, like it does for xvfb.
The class may create a file called capture.pcap in storageManager.directory ;)
This would add an option like --pcap.

What do you think?

[soulgalore]

@marty90 have a go if you have time and then we can help out in a PR to find a good structure. best to test out and if you find it useful we can help you get it in.

[soulgalore]

This was released today in 8.0.0. --tcpdump