v0.6.0

Release 0.6.0

We've added deferred query handles for GQL and EFCore, giving steps explicit control over when data is fetched from external sources. This should give users more control about when data is materialized within their pipelines.

What's New

• Deferred Query Handles: You can now use GqlQuery<TResult, T> and DbQuery<T> to declare queries in your catalog without triggering data fetches. Steps call ToListAsync() to materialize data when needed, making I/O boundaries explicit in step code. This keeps the catalog focused on structural declaration while steps control execution timing.
  • GQL Query Object: The GqlQuery<TResult, T> handle supports non-paginated, Relay-paginated, and offset-paginated queries, with composable filters via GqlQuery<TFilter, TResult, T> and a WithFilter() method.
  • EFCore Deferred Queries: The DbQuery<T> handle offers fluent composition (Where, OrderBy, Take, Skip) and server-side INSERT-FROM-SELECT fusion when saving to the same database and scope.
  • KedroSpaceflightsGQL Example: Demonstrates step-based parameterization using deferred GQL queries to fetch company analytics on demand. See examples/advanced/KedroSpaceflightsGQL.
• Improved Spaceflights Distributed: The distributed Spaceflights example now includes comprehensive FUnit test coverage for all data science and reporting steps, showing how to write testable multi-step pipelines. See examples/advanced/SpaceflightsDistributed.
• Enhanced Spark Support: The Spark extension now includes more Spark-native operations — better expression translation, support for additional scalar functions, and optimizations for ranking and windowing. See KedroSpaceflightsSpark.

Bug Fixes

• Spaceflights Distributed Tests: All data processing, model training, evaluation, and reporting steps now have corresponding FUnit tests, ensuring step-level transformations are validated before integration.

❤️ Thank You

• Spencer Elkington