Skip to content

Releases: juarezr/spark-streaming-google-pubsub

v0.9.4

Choose a tag to compare

@juarezr juarezr released this 04 Oct 02:07
4a4d04c

What's Changed

  • fix: continue inferring Spark batch interval from idle gaps until gaps are consistent by @juarezr in #22

Full Changelog: v0.9.3...v0.9.4

v0.9.3

Choose a tag to compare

@juarezr juarezr released this 02 Oct 07:13
5128bde

What's Changed

  • fix: implement connection error handling to halt on non-recoverable configuration errors by @juarezr in #21

Full Changelog: v0.9.2...v0.9.3

v0.9.2

Choose a tag to compare

@juarezr juarezr released this 01 Oct 07:03
d0cde08

What's Changed

  • fix: update AutoBatchTime logic for receiveTime estimation with a write-first approach by @juarezr in #20

Full Changelog: 0.9.1...v0.9.2

0.9.1

Choose a tag to compare

@juarezr juarezr released this 29 Sep 02:57
95263a1

What's Changed

  • feat: add code coverage support with JaCoCo and Coveralls by @juarezr in #19
    • Introduced new Maven profiles for coverage reporting and Coveralls integration.
    • Updated README and documentation to include coverage instructions and usage.
    • Enhanced CI workflow to run coverage tests and publish reports to Coveralls when applicable.
    • Added integration tests for PubSubClient, validating message publishing and acknowledgment against the Pub/Sub emulator

Full Changelog: v0.9.0...0.9.1

v0.9.0

Choose a tag to compare

@juarezr juarezr released this 26 Sep 09:37
18cc84d

What's Changed

  • feat: implement streaming pull PubSub Subscriber + bounded queue instead of unary Pulls on the driver by @juarezr in #18
    • [fix: refine AutoBatchTime logic for adjusting batch interval based on idle micro-batch conditions
    • [fix: Connector default batchSize = 64m to reduce pressure while still yielding 38 MiB parquet files

Full Changelog: v0.8.2...v0.9.0

v0.8.2

Choose a tag to compare

@juarezr juarezr released this 22 Sep 02:32
03e9271

What is Changed

  • fix: freeze AutoBatchTime logic after processing time becomes stable by @juarezr in #17

Full Changelog: v0.8.1...v0.8.2

v0.8.1

Choose a tag to compare

@juarezr juarezr released this 21 Sep 21:44
b0ab256

What is Changed

  • feat: enhance AutoBatchTime to improve busy-wait handling by @juarezr in #15

Full Changelog: v0.8.0...v0.8.1

v0.8.0

Choose a tag to compare

@juarezr juarezr released this 21 Sep 19:26
b070bc9

What is Changed

  • feat: enhance AutoBatchTime to handle startup noise detection by @juarezr in #14 :
    • feat: enhance AutoBatchTime to handle startup noise detection
    • refactor[ci]: update release workflow to improve artifact handling
    • reorder publishing to release first on GitHub

Full Changelog: v0.7.0...v0.8.0

v0.7.0

Choose a tag to compare

@juarezr juarezr released this 18 Sep 07:40
b54a71b

What is Changed

Multiple fixes and enhancements by @juarezr in #13

  • feat: enhance build information and logging in connector startup
  • fix: handle AutoBatchTime with zero trigger mode
  • fix: raise processing time from idle start-to-start gaps
  • feat: implement debounce logic for idle pulls within a gather batch
  • feat: implement Trigger.AvailableNow with drain-until-idle behavior for recovery purposes
  • feat: add limitTime parameter to set an exclusive upper bound for message processing based on publish time.
  • docs: update README for clarity and add usage examples for Java, Scala, and PySpark

Full Changelog: v0.6.3...v0.7.0

v0.6.3

Choose a tag to compare

@juarezr juarezr released this 17 Sep 07:16
5992f42

What is Changed

Full Changelog: v0.6.2...v0.6.3