Decoders for sources #257
Labels
domain: codecs
Anything related to Vector's codecs (encoding/decoding)
domain: data model
Anything related to Vector's internal data model
domain: logs
Anything related to Vector's log events
domain: sources
Anything related to the Vector's sources
have: should
We should have this feature, but is not required. It is medium priority.
needs: requirements
Needs a a list of requirements before work can be begin
type: enhancement
A value-adding code change that enhances its existing functionality.
For applicable sources, you should be able to configure a decoder (e.g. JSON) that will fully parse and structure the incoming data. The difference between a source-configured decoder and a downstream parsing transform is that a source-configured decoder will be expected to completely parse the entire message into structured data and drop the original raw bytes representation.
This will also require figuring out how downstream transforms and sinks will handle this new type of record with no raw data. Similar to #256, we may need to add a conditional or
match
to detect this case and provide a different default behavior that makes sense for this kind of data.This would also benefit greatly from #235, so that we could detect and prevent cases where downstream components would try to read the raw original data by default and end up with surprising behavior.
Related to both of the above, there may be decoders (syslog comes to mind) where it makes sense to set one parsed field (e.g. message) as the default target of downstream parsing components.
The text was updated successfully, but these errors were encountered: