Mime Types

[neu5ron]

I was wondering if we could discuss/add mime_type as a nested field.
There is a bit discussed on this already over at #554, however since it started as HTTP specific header fields, I think it may have gotten lost in the mix and mime type's deserve their own issue :)

I think this would be a pretty quick win that would benefit setting up some nice analytics and just all around schema for a lot of logs.

a really easy "analytic" is something like file.mime_type:"application/x-dosexec" AND NOT file.extension:(exe OR dll OR msi)

Some type of logs and things that would benefit:

• Suricata HTTP
• Suricata Alert
• Yara and other file scanning
• AV, EDR, and or Sandbox
• Zeek HTTP
• Zeek SMTP
• Zeek File
• Zeek FTP
• Anything else containing HTTP/SMTP logs (Squid, PaloAlto, etc...)

Miss you @webmat ;)

[webmat]

Ah right, good point. Where would you see adding a .mime_type field?

• file.mime_type
• http.request.mime_type
• http.response.mime_type
• Somewhere under email.* when we get around to it, of course
• Anywhere else?

Is it worth breaking up the mime_type into type, subtype & parameter to discrete fields, you think?

BTW you're still hanging around the cool places, and that's what counts ;-)

[neu5ron]

I think all those fields are perfect start.
Seems good to me!

I don't think necessary to break up.

[webmat]

Solved by #760

[webmat]

Just realized #760 only added file.mime_type, but not the http request/response ones.

@ebeahan Let's see if we can add http.request.mime_type and http.response.mime_type for 1.6, WDYT?

[ebeahan]

Sounds good 👍

[ebeahan]

Reading over the past discussion here, #760, and #554, appears only the file.mime_type field was added due to the ongoing conversations around how to best handle HTTP headers in ECS such as content-type.

I do see a case for populating a dedicated http.response.mime_type field alongside the nested HTTP header fields. The first extracts the MIME type (that happens to be transmitted in the Content-Type header) while the second captures all HTTP headers from the request/response (which would capture the content-type as well, but not in a specialized field).

However, I'm not sure http.request.mime_type aligns with http.response.mime_type and file.mime_type. The Accept header advertises which content types the client making the request is able to understand. This provides a different function than http.response.mime_type or file.mime_type, which is stating specifically the type of the content.

Also while http.response.mime_type and file.mime_type can be limited to one value, Accept headers often contain numerous types. This can be handled, but it's another difference worth noting.

[webmat]

Yes, you're right. So the idea behind both http mime_type fields is not for them to be a replacement for the http headers.

We don't have http headers defined yet in ECS, so perhaps it's a bit early to tackle this, after all.

But the overall plan is the content-type headers would always go to their appropriate place in the header fields. The mime_type should be populated only when doing analysis of the body.

We could likely make a concession that the http mime_type can also be populated based on the http headers if the source of the header is trusted (e.g. I trust my web server's access logs to set http.response.mime_type correctly). I think this concession would only make sense for the response mime_type, probably never on the request side.

The goal is specifically to enable analysis on Content-Type headers that say something, when the actual mime type is different. Which could indicate an attack or something misconfigured.

And indeed I would keep the accept headers out of this completely. They're telling us a different thing :-)