Skip to content

1.3.0
Compare
Choose a tag to compare
@dlvenable dlvenable released this 23 Mar 01:35
· 1513 commits to main since this release
1.3.0
f817ef3
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

2022-03-22 Version 1.3.0

Important Deprecation

  • We have updated the pipeline definition to support processor: as replacement of prepper: which has been deprecated and will be fully removed in 2.0. (#655, #667)

Features

  • AggregateProcessor for generic stateful aggregation (README) (#839, #850, #931, #969, #1022, #1046)
  • DateProcessor to extract dates from fields in events (README) (#971, #1014)
  • Processors to support mutate, alter, and delete fields from Events (README) (#1002)
  • KeyValueProcessor to support parsing messages with key-value strings such as queries and properties (README) (#872)
  • DropProcessor to filter out (remove/drop) entire events based off confitional expression (README) (#801, #1174)

Enhancements

  • Add dependency Injection support for Data Prepper Core and Plugins. (#815, #846, #1140)
  • Add Data Prepper expression evaluator (#1024, #1027, #1090, #1065, #1153, #1155, #1157, #1169, #1177, #1178)
  • Support for nested syntax in LogStashConfigConverter (#1088)
  • Support default values for attributes in LogstashConfigConverter mapping files (#1095)
  • Support converting index with a date-time pattern in LogstashConfigConverter (#1045, #1095)
  • Support creation from raw text string in JacksonEvent builder (#770, #1074)
  • Support loading plugins from multiple packages (#948)
  • Support of date and time patterns in opensearch sink index names (#788, #833)
  • Support for passing a PipelineDescription in @DataPrepperPluginConstructor (#825)
  • Validate Plugin Configurations using JSR-303 (#826)
  • Support Logstash configuration conversion for OpenSearch Logstash output (#756)
  • Support negation of boolean attribute values in Logstash configuration converter while mapping plugins (#756)

Bug Fixes

  • Allow stdout and file sink to output generic object type (#1192)
  • Fixed issue where Spring was unable to find the PrometheusMeterRegistry Bean (#1019)
  • Use BlockingTaskExecutor in OtelTraceSource (#745)

Infrastructure

  • Upload Maven artifacts as part of Release build (#1181)
  • Updated the release build to push the Docker image and upload archives (#1151)
  • Update Gradle project to produce only tar.gz archives (#1132)
  • Add simple integration tests for AggregateProcessor (#1046)
  • Assemble the data-prepper-core uber-jar using Zip64 (#820)

Documentation

  • Example on log ingestion from Kubernetes containers (#729)
  • Update copyright headers in release scripts (#933)
  • Update copyright headers in data-prepper subprojects (#776, #928)
  • Example on Data-Prepper ECS Firelens integration (#704)

Maintenance

  • Delete outdated Kibana trace analytics example (#1135)
  • Update Open Distro usages to OpenSearch in scripts (#1086)
  • Upgrade docker-compose.yml files from ODFE to OpenSearch (#847)
  • Update the ADOT example to OpenSearch (#703)
  • Migrate demo sample application to use opensearch and opensearch dashboards instead of ODFE and kibana (#666)
  • Update log ingestion example to use latest data-prepper docker image (#752)

Refactoring

  • Deprecating Record and RecordMetaData (#954)
  • FileSink to use Event model (#750)
  • StringPrepper to use Event model (#753)
  • StdInSource, RandomStringSource to use Event model (#625)
Assets 2