Azure Blob Storage Sink Content-Type Not Reflecting Underlying Media Type

[stemjacobs]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

When using the Azure Blob Storage sink it works splendidly, but will indicate the Content-Type in the blob's metadata in relation to how the payload was compressed vs its underlying encoding type. The sink already correctly sets the Content-Encoding type based on compression chosen. It currently, however, will also set Content-Type based on the type of compression used vs the original media/content type. This is contrary to the expectation touched on in RFC2616 Section 14.11

This causes no issues with actual functionality of the sink and is a minor concern at best. Depending on consuming client implementations there could be some confusion on the expeted returned media type, but that is about it.

With the following settings for the sink:

...
compression = "gzip"
encoding.codec = "json"
...

Actual:

Content-Type = "application/gzip"

Expected:

Content-Type = "application/json"

Configuration

[sources.generate_syslog]
type = "demo_logs"
format = "syslog"
count = 100

[transforms.remap_syslog]
inputs = ["generate_syslog"]
type = "remap"
source = '''
  structured = parse_syslog!(.message)
  . = merge(., structured)
'''

[sinks.emit_syslog]
inputs = ["remap_syslog"]
type = "console"
encoding.codec = "json"

[sinks.azure_storage]
inputs = ["remap_syslog"]
type = "azure_blob"
container_name = "testlogs"
compression = "gzip"
connection_string = "<test_blob_storage_string_here>"
encoding.codec = "json"

Version

0.32.0

Debug Output

No response

Example Data

No response

Additional Context

No response

References

No response