sqrl 0.6.1

What's Changed

• Fix logger configuration by @velo in #1114
• Enable dependabot by @velo in #1116
• Link and README update by @mbroecheler in #1106
• Homepage commands by @mbroecheler in #1139
• Reduce bottleneck build around vertx vmp by @velo in #1111
• Explain ENV vars for local data access by @mbroecheler in #1162
• Write server-config.json to plan by @velo in #1163
• Dag picture by @mbroecheler in #1164
• updated docs and LICENSE by @mbroecheler in #1183
• Add badges by @mbroecheler in #1196
• Fix glue and iceberg runtime issues by @velo in #1198
• Change vertx.json parser to fallback to format including model container by @velo in #1202
• Move to docs.datasqrl.com by @mbroecheler in #1203
• Add add-opens to java by @velo in #1205
• update CNAME by @mbroecheler in #1204
• Add extra export and opens by @velo in #1211
• Fix flink warnings by @velo in #1212
• Fix Submodule Link by @mbroecheler in #1213
• Consolidate modules by @velo in #1108
• Test if cmd image print flink warning logs by @velo in #1206
• Auto register aggregation functions by @velo in #1214
• Refactor and Removal by @mbroecheler in #1223
• Add plugins to format files by @velo in #1224
• minor docs update by @mbroecheler in #1227
• Use docker hub as staging area for test images by @velo in #1225
• Simplify CREATE TABLE mapping by @mbroecheler in #1232
• adding test coverage for timestamp arguments with interval conditions by @mbroecheler in #1239

---

List of PRs that updated libraries versions

• Bump vertx.version from 4.5.8 to 4.5.14 by @dependabot in #1119
• Bump com.opencsv:opencsv from 5.7.1 to 5.11 by @dependabot in #1120
• Bump testcontainers.version from 1.18.0 to 1.21.0 by @dependabot in #1126
• Bump org.apache.hadoop:hadoop-mapreduce-client-core from 3.1.0 to 3.4.1 by @dependabot in #1131
• Bump com.h2database:h2 from 2.1.214 to 2.3.232 by @dependabot in #1132
• Bump com.google.guava:guava from 31.1-jre to 33.4.8-jre by @dependabot in #1133
• Bump io.reactiverse:vertx-maven-plugin from 1.0.28 to 2.0.1 by @dependabot in #1135
• Bump org.apache.maven.plugins:maven-release-plugin from 3.0.0 to 3.1.1 by @dependabot in #1136
• Bump org.apache.commons:commons-lang3 from 3.12.0 to 3.17.0 by @dependabot in #1138
• Bump org.apache.maven.plugins:maven-surefire-plugin from 2.22.2 to 3.5.3 by @dependabot in #1125
• Bump org.immutables:value from 2.8.8 to 2.10.1 by @dependabot in #1128
• Bump log4j.version from 2.23.1 to 2.24.3 by @dependabot in #1129
• Bump junit.jupiter.version from 5.8.2 to 5.12.2 by @dependabot in #1134
• Bump snakeyaml to 2.2 by @velo in #1143
• Bump org.freemarker:freemarker from 2.3.32 to 2.3.34 by @dependabot in #1144
• Bump org.assertj:assertj-core from 3.24.2 to 3.27.3 by @dependabot in #1145
• Bump io.micrometer:micrometer-registry-prometheus from 1.13.3 to 1.14.6 by @dependabot in #1146
• Bump commons-beanutils:commons-beanutils from 1.9.4 to 1.10.1 by @dependabot in #1148
• Bump io.netty:netty-codec from 4.1.92.Final to 4.2.1.Final by @dependabot in #1149
• Bump io.agroal:agroal-pool from 1.11 to 2.6 by @dependabot in #1150
• Bump org.duckdb:duckdb_jdbc from 0.10.3 to 1.2.2.0 by @dependabot in #1152
• Bump jakarta.annotation:jakarta.annotation-api from 1.3.5 to 3.0.0 by @dependabot in #1153
• Bump software.amazon.awssdk:bom from 2.24.5 to 2.31.38 by @dependabot in #1155
• Bump commons-io:commons-io from 2.11.0 to 2.19.0 by @dependabot in #1157
• Bump info.picocli:picocli from 4.7.6 to 4.7.7 by @dependabot in #1159
• Bump org.testcontainers:redpanda from 1.20.1 to 1.21.0 by @dependabot in #1160
• Bump org.codehaus.mojo:build-helper-maven-plugin from 3.4.0 to 3.6.0 by @dependabot in #1161
• Bump org.apache.iceberg:iceberg-flink-runtime-1.19 from 1.6.0 to 1.9.0 by @dependabot in #1154
• Bump software.amazon.awssdk:bom from 2.31.38 to 2.31.39 by @dependabot in #1179
• Bump org.apache.avro:avro from 1.11.3 to 1.12.0 by @dependabot in #1181
• Bump com.google.auto.service:auto-service from 1.0.1 to 1.1.1 by @dependabot in #1165
• Bump com.fasterxml.jackson:jackson-bom from 2.16.2 to 2.19.0 by @dependabot in #1167
• Bump org.apache.maven.plugins:maven-failsafe-plugin from 3.3.1 to 3.5.3 by @dependabot in #1168
• Bump org.testng:testng from RELEASE to 7.11.0 by @dependabot in #1169
• Bump com.google.protobuf:protobuf-java from 3.21.1 to 4.30.2 by @dependabot in #1171
• Bump commons-codec:commons-codec from 1.17.2 to 1.18.0 by @dependabot in #1172
• Bump org.apache.zookeeper:zookeeper from 3.7.1 to 3.9.3 by @dependabot in #1173
• Bump net.snowflake:snowflake-jdbc from 3.18.0 to 3.24.0 by @dependabot in #1174
• Bump org.apache.maven.plugins:maven-dependency-plugin from 3.6.0 to 3.8.1 by @dependabot in #1175
• Bump org.slf4j:slf4j-simple from 1.7.36 to 2.0.17 by @dependabot in #1176
• Bump org.apache.commons:commons-configuration2 from 2.9.0 to 2.12.0 by @dependabot in #1177
• Bump org.apache.maven.plugins:maven-compiler-plugin from 3.10.1 to 3.14.0 by @dependabot in #1178
• Bump org.postgresql:postgresql from 42.5.0 to 42.7.5 by @dependabot in #1180
• Bump com.graphql-java:graphql-java-extended-scalars from 19.1 to 22.0 by @dependabot in #1166
• Bump com.google.guava:guava from 31.1-jre to 33.4.8-jre by @dependabot in #1170
• Bump org.antlr:antlr4-runtime from 4.9.2 to 4.13.2 by @dependabot in #1184
• Bump org.apache.commons:commons-collections4 from 4.4 to 4.5.0 by @dependabot in #1186
• Bump org.apache.calcite:calcite-core from 1.36.0 to 1.39.0 by @dependabot in #1187
• Bump com.networknt:json-schema-validator from 1.5.2 to 1.5.6 by @dependabot in #1188
• Bump software.amazon.awssdk:bom from 2.31.39 to 2.31.40 by @dependabot in #1190
• Bump org.slf4j:slf4j-api from 1.7.36 to 2.0.17 by @dependabot in #1191
• Bump org.yaml:snakeyaml from 2.2 to 2.4 by @dependabot in #1192
• Bump com.theokanning.openai-gpt3-java:service from 0.11.0 to 0.18.2 by @dependabot in #1193
• Bump io.projectreactor:reactor-core from 3.5.6 to 3.7.5 by @dependabot in #1194
• Bump net.lingala.zip4j:zip4j from 2.11.2 to 2.11.5 by @dependabot in #1195
• Bump software.amazon.awssdk:bom from 2.31.40 to 2.31.41 by @dependabot in #1199
• Bump io.micrometer:micrometer-registry-prometheus from 1.14.6 to 1.15.0 by @dependabot in #1200
• Bump org.apache.hadoop:hadoop-common from 2.10.2 to 3.4.1 by @dependabot in #1127
• Bump org.slf4j:slf4j-api from 1.7.36 to 2.0.17 by @dependabot in #1201
• Bump log4j.version from 2.23.1 to 2.24.3 by @dependabot in #1207
• Bump org.apache.hadoop:hadoop-mapreduce-client-core from 3.1.0 to 3.4.1 by @dependabot in #1208
• Bump software.amazon.awssdk:bom from 2.31.41 to 2.31.42 by @dependabot in https:...

sqrl 0.6.0

What's Changed

• Add tests for avro schema by @henneberger in #973
• Add date type for avro schemas by @henneberger in #977
• Add graphql subscription support for postgres-log engine in vertx by @nandorsoma in #850
• Move snapshots for fullusescases by @henneberger in #978
• Add time type for avro schemas by @henneberger in #979
• Add Avro row support by @henneberger in #980
• Add avro single row datatype support by @henneberger in #981
• Avro array datatype support by @henneberger in #982
• Fix serialize to bytes by @henneberger in #983
• Avro unstructured map datatype support by @henneberger in #984
• Fully migrate to flink function system by @henneberger in #952
• Allow test goal with no visible queries by @henneberger in #942
• Copy jar to build dir by @henneberger in #985
• Add async function test by @henneberger in #986
• Expose vector udt as common type by @henneberger in #987
• Add additional error checking for schema walker by @henneberger in #991
• Flink logical functions test by @Zesky665 in #954
• Fix function aliasing for function names with an underscore by @henneberger in #994
• Flink arithmetic function tests by @Zesky665 in #955
• Allow the percent operator for mod by @henneberger in #996
• Re-enable iceberg tests by @henneberger in #998
• Fix nested map types by @henneberger in #1003
• Improve Error Handling by @mbroecheler in #1004
• Flink collections functions test by @Zesky665 in #961
• Flink string function tests by @Zesky665 in #964
• Flink json function tests by @Zesky665 in #972
• Flink temporal functions test by @Zesky665 in #971
• Restore vertx from accidental removal by @henneberger in #1009
• Set prometheus metrics by default, support setting conf directory by @henneberger in #1010
• updating snapshots by @mbroecheler in #1021
• Fix java type for event_time by @velo in #1023
• Add DoubleToVector to the std library by @nandorsoma in #1029
• Revert "Add DoubleToVector to the std library" by @henneberger in #1037
• Fix datatype parsing to support map types by @henneberger in #1032
• Update developer documentation by @echauchot in #1040
• Improve Specifications documentation by @echauchot in #1042
• Support BigInteger in GraphQL (#744 and #909) by @echauchot in #1041
• Document server-core server-vertx by @echauchot in #1044
• Add missing java.util.List by @velo in #1048
• Made project compatible with eclipse by @velo in #1050
• Update release procedure by @echauchot in #1049
• Only create topics if it doesn't exist by @velo in #1056
• Include schema on compiled flink plan by @velo in #1045
• Deploy snapshot builds to github packages by @velo in #1035
• Isolate docker and build phases into different gh-actions by @velo in #1057
• Revert "Isolate docker and build phases into different gh-actions" by @velo in #1058
• Lock ubuntu version to fix multi-platform build by @velo in #1059
• Dynamically tag Docker images based on Maven versioning by @velo in #1062
• Publish docker images to ghcr.io for PR snapshots by @velo in #1065
• Remove profiles support by @velo in #1047
• Remove external repositories by @velo in #1053
• Fix warning 'No table file for schema file' by @velo in #1073
• Fix/rowtime nullability by @mbroecheler in #1076
• V0.6.0 by @mbroecheler in #1071
• Bring 0.6 branch changes to main by @velo in #1078
• Add Flink Function Test by @mbroecheler in #1080
• Get subscription testing wired by @velo in #1081
• bugfixes by @mbroecheler in #1082
• fix validation of parameter nullability, fix validation messages. by @mbroecheler in #1084
• Make it possible to discover sqrl standard libraries by @velo in #1079
• Simplify queries in Server & fix query lookup by @mbroecheler in #1085
• Easy code coverage by @velo in #1086
• Move types, formats and connector to flink-sql-runner by @velo in #1083
• Migrate code to java 17 by @velo in #1087
• Use SimpleTypeSerializerSnapshot by @velo in #1088
• Fix cyclic schema detection. by @mbroecheler in #1089
• Vector test case by @mbroecheler in #1091
• Classpath Function Loading by @mbroecheler in #1093
• Add test for connector/format dependencies by @mbroecheler in #1092
• Additional Kafka Test Coverage by @mbroecheler in #1095
• CircleCI setup by @velo in #1096
• Moving & updating docs by @mbroecheler in #1098
• Fix docs deploy by @mbroecheler in #1100
• fix broken links, update deployment process by @mbroecheler in #1101
• Fix Docusaurus Build by @mbroecheler in #1102
• Rename functions project by @velo in #1097
• Explicitly set nullability by @mbroecheler in #1022
• Moving logic with a graph to a separate library in NPM. by @ustymenkopavlodevima in #939
• Remove unused code by @velo in #1090
• fix topicName reference by @mbroecheler in #1104
• Docs update for 0.6 by @mbroecheler in #1103
• add redpanda volume by @wutianchen in #1105
• Use latest flink-sql-runner release by @velo in #1109
• fix index selection for text and vector by @mbroecheler in #1107
• Bump flink-sql-runner to version 0.6.0 by @velo in #1110

New Contributors

• @echauchot made their first contribution in #1040
• @ustymenkopavlodevima made their first contribution in #939

Full Changelog: v0.5.10...0.6.0

v0.5.10

[maven-release-plugin] copy for tag v0.5.10

v0.5.9

What's Changed

• Add Filtered Distinct Feature by @mbroecheler in #1018

Full Changelog: v0.5.8...v0.5.9

v0.5.8

Resolves an issue when using the avro schema with json formats.

v0.5.7

What's Changed

• Consistent jar handling for Local and Remote UDFs by @nandorsoma in #849
• Fix null values for case insensitive datafetcher (#871) by @henneberger in #872
• Run unit tests on main by @henneberger in #876
• Add checkpoint monitoring to test infrastructure by @henneberger in #929
• Update instructions by @Zesky665 in #936
• Fixed sensor-teaser tests by @Zesky665 in #935
• Include url on timeout exceptions by @velo in #946
• Flink functions test by @Zesky665 in #944
• Add comparison tests by @Zesky665 in #948
• Exclude like from imported flink functions by @henneberger in #950
• Enable Like test by @Zesky665 in #953

New Contributors

• @velo made their first contribution in #946

Full Changelog: v0.5.6...v0.5.7

v0.5.6

What's Changed

• Add more tests, install views by @henneberger in #843
• Fix avro config value by @henneberger in #844
• Adding some more logging to the tests by @Zesky665 in #852
• Enable Separate Validation for the Package Field During Publishing by @nandorsoma in #846
• Add udf support to run/test by @henneberger in #851
• Add compiled plan to flink plan by @henneberger in #853
• Move compiled plan to flink folder by @henneberger in #854

Full Changelog: v0.5.5...v0.5.6

v0.5.5

DataSQRL v0.5.5 🚀🌟

We're excited to bring you another packed release filled with improvements, new features, and fixes to further enhance your DataSQRL experience!

🌟 Major Enhancements:

• Flink 1.19 Support: We've upgrade to Flink 1.19 so you can take advantage of all of the features and bug fixes it offeres.
• Support for Multiple Database Engines: Now, you can connect and operate across multiple database engines, such as Postgres and DuckDB over iceberg effortlessly.
• Prometheus Metrics Endpoint: Added a new Prometheus endpoint for better monitoring and observability of your systems.
• Extended Avro Support: Added more support for avro data types. Be sure to see the timestamp_mapping.legacy flag for avro sources to take advantage of the new data types.
• Revamped Run and Test command: The docker command now supports a run and test target for faster iteration. We've removed the extra docker images so it can run in github actions and more.

📈 General Improvements:

• Better Error Handling in Schema Validation: We've improved error handling in schema validation, reducing the noise and providing clearer feedback when issues arise.
• Timestamps and Avro Schema Enhancements: Improved timestamp reassignment handling and expanded datatype support for Avro schema, making processing more reliable.
• IndexHints in DagPlanner: Added IndexHints for more optimized query execution plans.
• Improved Documentation: Updated and moved documentation to make it more accessible, including new details for Avro schema handling.

🛠️ Change Log:

• Fix generating subscriptions with nullable primary keys #782
• Fix tests in UseCasesFullIT #777
• Change postgres-cdc TableType to VERSIONED_STATE as a temporary fix #766
• Add support for log engine import/export #747
• Add Prometheus metrics endpoint #783
• Support for multiple database engines #784
• Clean up serializer code #791
• Add postgres-cdc to Vert.x #790
• Put subscriptions behind a feature flag #801
• Update README.md #799
• Fixes for timestamp reassignment #805
• Bump Flink to 1.19 #808
• Use greatest function for timestamp calculation #813
• Include more datatypes for Avro schema #811
• Move documentation to the sqrl repository #789
• Fix distinct casing in queries #817
• Add virtual columns and clean up code #815
• Remove count from Flink functions and clean up tests #816
• Set file monitoring to 10 seconds in discovery #820
• Add lowercase to string library #823
• IndexHints in DagPlanner #824
• Add views to Postgres plan #835
• Move to old run command and Flink 1.19 dependencies #836
• Re-enable the new run command #838
• Create Kafka topics from values array #841

New Contributors

• @Zesky665 made their first contribution in #799

Full Changelog: v0.5.4...v0.5.5

v0.5.4

DataSQRL v0.5.4 🚀🌟

In this release, we are thrilled to announce a host of enhancements and new features that significantly improve the capabilities and user experience of DataSQRL.

🌟 Major Enhancements:

• Create Table syntax: Log engine tables can now be created with CREATE TABLE statements to simply adding new tables.
• Subscription and Mutation Generation: New capability to generate GraphQL subscriptions and mutations for all stream tables automatically.
• Iceberg Integration: Added modularity and partition fixes for better Iceberg connector performance with Flink.
• Snowflake Integration: Added snowflake as a query engine
• DuckDB Integration: Extended support with DuckDB to bolster our analytical processing capabilities.

📈 General Improvements:

• String and Array Handling Functions: Improved handling of strings and arrays in data streams for more efficient processing.

🛠️ Change Log:

• Update README documentation by @mbroecheler in #726 and #730
• Add primary key in discovery by @mbroecheler in #739
• Generate and refine subscriptions/mutations by @henneberger in #735, #750, #753
• Add and adjust partitioning for Iceberg by @henneberger in #751
• Implement minor changes in Snowflake and string/array handling by @henneberger in #757, #774, #769
• Add Snowflake in Vert.x by @henneberger in #775
• Additional SQL features and fixes by @mbroecheler in #762, #771

Full Changelog: v0.5.3...v0.5.4

v0.5.3

[maven-release-plugin] copy for tag v0.5.3