feat: add kafka, redis, and schema registry packages #33
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…tegration Added a new `kafka` package offering robust Kafka client functionality, including message publishing, consuming, and lifecycle management. Integrated FX module for simplified dependency injection. Packaged features include distributed tracing, header injection, and structured logging. Comprehensive unit and integration tests ensure reliability.
Enhanced the Kafka package with serializer and deserializer APIs for seamless message encoding and decoding. Introduced configurable serializers and deserializers with support for JSON, Protobuf, and Avro formats. Updated client and message handling methods to integrate serialization logic. Expanded documentation with examples for serialization use cases.
Integrated schema registry with support for JSON, Protobuf, and Avro serializers and deserializers. Introduced schema management APIs, caching, compatibility checks, and wire format encoding/decoding. Added Fx module for dependency injection and extensive unit tests to ensure functionality. Updated documentation with usage examples.
…client Introduced the `DataType` field in Kafka client configuration for automatic serializer and deserializer selection, supporting JSON (default), Gob, string, bytes, and custom implementations like Protobuf and Avro. Added helper methods for setting and validating default serializers. Updated documentation with detailed usage examples for serialization.
…egration tests Introduced a Redis client with robust configuration options, including support for single-node, cluster, and sentinel setups. Integrated Fx modules for simplified dependency injection. Added structured logging, TLS support, and default connection pooling. Comprehensive unit and integration tests validate key-value, hash, list, set, and pub/sub operations. Updated documentation with detailed examples and usage guidelines.
Added checks to skip Kafka integration tests when running in short mode (`testing.Short`). This ensures tests are not executed in environments where integration tests are explicitly excluded.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces three new packages to the
stdlibrary: Kafka, Redis, and Schema Registry. All packages follow the established patterns in the library and include comprehensive testing with testcontainers.Changes Overview
📦 New Packages
1. Kafka (
v1/kafka)2. Redis (
v1/redis)3. Schema Registry (
v1/schema_registry)🧪 Testing
All packages include comprehensive testing:
-shortflag for CI/CD)🔧 Integration
All packages follow the established
stdlibrary patterns:📝 Documentation
doc.gofilesdocs/v1/Dependencies Added
Breaking Changes
None - all new packages with no changes to existing code.