#2673: Microservice pattern: Polling publisher #3243

[quantdevv]

Pull Request

What does this PR do?

This PR introduces a microservice-based architecture that includes a polling-publisher and a subscriber-service communicating via Apache Kafka. It demonstrates an in-memory data flow mechanism using Kafka topics.

The PR includes:

• Kafka producer REST endpoint for sending messages
• Kafka consumer that listens automatically and processes messages
• Spring Boot configuration for both microservices
• Initial testing and local setup for message flow validation

Fixes #2673

Type of change

• New feature (non-breaking change)
• Documentation update

How has this been tested?

• Manually tested Kafka message publishing via /send endpoint
• Verified automatic listening and processing in subscriber microservice
• Logs and terminal output confirm end-to-end communication

Additional context

Both microservices are designed for local Kafka communication using port 9092. Make sure the Kafka broker is running before testing

[github-actions]

PR Summary

This PR introduces a microservice-based architecture using Kafka for communication between a polling publisher and a subscriber service. It includes REST endpoints, Kafka producers and consumers, Spring Boot configurations, and testing for message flow validation.

Changes

File	Summary
polling-publisher/README.md	This file provides a comprehensive guide to the Polling Publisher-Subscriber pattern, explaining its intent, architecture, real-world examples, and implementation using Spring Boot and Kafka. It also covers when to use this pattern, its benefits and trade-offs, and related design patterns.
polling-publisher/etc/polling-publisher.urm.puml	New file.
polling-publisher/polling-service/etc/polling-service.urm.puml	New file.
polling-publisher/polling-service/pom.xml	This file configures the Maven project for the polling service microservice, defining dependencies and build settings. It includes Spring Boot, Kafka, and testing dependencies.
polling-publisher/polling-service/src/main/java/com/iluwatar/polling/App.java	This is the main application class for the polling service microservice. It uses Spring Boot annotations to enable the application context and scheduling.
polling-publisher/polling-service/src/main/java/com/iluwatar/polling/DataRepository.java	This class defines an in-memory repository for storing and retrieving data. It uses a HashMap for data storage and provides methods for saving, retrieving, deleting, and finding all data.
polling-publisher/polling-service/src/main/java/com/iluwatar/polling/DataSourceService.java	This service is responsible for managing data, including adding, retrieving, and removing data. It uses a DataRepository and a scheduler to periodically generate and add new data.
polling-publisher/polling-service/src/main/java/com/iluwatar/polling/KafkaProducer.java	This class handles sending messages to Kafka. It uses Spring's KafkaTemplate to publish messages to specified topics.
polling-publisher/polling-service/src/main/java/com/iluwatar/polling/PollingController.java	This REST controller provides a /send endpoint for manually sending messages to Kafka and a health check endpoint.
polling-publisher/polling-service/src/main/java/com/iluwatar/polling/PollingScheduler.java	This class schedules the data polling and publishing tasks using Spring's @Scheduled annotation. It retrieves data from the DataSourceService and sends it to Kafka.
polling-publisher/polling-service/src/main/resources/application.yml	This file contains the Spring Boot configuration for the polling service, including Kafka broker settings, consumer group ID, and port number.
polling-publisher/polling-service/src/test/java/com/iluwatar/polling/AppTest.java	This file contains a test case to verify the application context loading.
polling-publisher/polling-service/src/test/java/com/iluwatar/polling/DataRepositoryTest.java	This test suite verifies the functionality of the DataRepository class, including saving, retrieving, deleting, and finding all data.
polling-publisher/polling-service/src/test/java/com/iluwatar/polling/DataSourceServiceTest.java	This test suite contains test cases for the DataSourceService, covering adding, retrieving, removing, and getting all data.
polling-publisher/pom.xml	This is the main pom file for the polling-publisher module. It aggregates the polling-service and subscriber-service modules and defines common dependencies.
polling-publisher/subscriber-service/etc/subscriber-service.urm.puml	New file.
polling-publisher/subscriber-service/pom.xml	This file configures the Maven project for the subscriber service microservice, defining dependencies and build settings. It includes Spring Boot, Kafka, and testing dependencies.
polling-publisher/subscriber-service/src/main/java/com/iluwatar/subscriber/App.java	This is the main application class for the subscriber service microservice. It uses Spring Boot annotations to enable the application context.
polling-publisher/subscriber-service/src/main/java/com/iluwatar/subscriber/KafkaConsumer.java	This class consumes messages from Kafka topics. It uses Spring's @KafkaListener annotation to listen for messages on the updates and API topics.
polling-publisher/subscriber-service/src/main/resources/application.yml	This file contains the Spring Boot configuration for the subscriber service, including Kafka broker settings, consumer group ID, and port number.
polling-publisher/subscriber-service/src/test/java/com/iluwatar/subscriber/AppTest.java	This file contains a test case to verify the application context loading.
pom.xml	The polling-publisher module was added to the list of modules.

---

autogenerated by presubmit.ai

[github-actions]

✅ LGTM!

Review Summary

Commits Considered (30)

• e49025f: Merge branch 'master' into issue-2673
• e0dbd48: Merge branch 'issue-2673' of https://github.com/quantdevv/java-design-patterns into issue-2673
• 256d81b: addressed review comment: removed unnecessary dependency
• ff872e7: used logger to print statement
• f16fb4f: fixed the printing statements & used log function
• 6b1fefd: deleted unnecessary README.md files
• ee8c228: improved the README.md file & fixed it's format
• c2848da: Updated README.md file based on new template
• 589fe40: merged changes from origin/master
• f799e6f: Merge branch 'iluwatar:master' into issue-2673
• 8c50536: Merge branch 'master' into issue-2673
• 059f292: added polling-publisher module in pom.xml
• 84d08f0: added scope before class definition in Test classes & removed duplicate dependency from pom.xml & added required one
• 5e3c951: synced with remote branch
• 24b87bb: added description on both application.yaml file
• 4eeb011: updated all README.md
• 5aa46ac: added new listener for other topic
• a59362c: Fixed topic name
• 1b3d933: Fixed subscriber-service & updated pom.xml
• 46abbfb: Fixed Kafka bug & added a controller to send message using API
• a7cc2c3: implemented subscriber microservice using kafkaConsumer
• a7497c4: implemented Data repository, DataSource Service & added unittest-case & implemented polling-service microservice
• 78f1ea9: resolved dependency & structured classes
• 931ab84: added unit test-case for DataSourceService class
• bb065d3: added entry point for subscriber service & unit test-case
• 4988d2e: subscriber-service implementation
• a2b8743: fixed checkstyle errors
• 0051d3f: polling-publisher service implementation
• c4e9276: added two service polling service & publisher service to handle two different operation & prepared initial skeleton
• faf0a12: added new microservice pattern polling-publisher, created folder structure, README.md & App.java

Files Processed (22)

• polling-publisher/README.md (1 hunk)
• polling-publisher/etc/polling-publisher.urm.puml (1 hunk)
• polling-publisher/polling-service/etc/polling-service.urm.puml (1 hunk)
• polling-publisher/polling-service/pom.xml (1 hunk)
• polling-publisher/polling-service/src/main/java/com/iluwatar/polling/App.java (1 hunk)
• polling-publisher/polling-service/src/main/java/com/iluwatar/polling/DataRepository.java (1 hunk)
• polling-publisher/polling-service/src/main/java/com/iluwatar/polling/DataSourceService.java (1 hunk)
• polling-publisher/polling-service/src/main/java/com/iluwatar/polling/KafkaProducer.java (1 hunk)
• polling-publisher/polling-service/src/main/java/com/iluwatar/polling/PollingController.java (1 hunk)
• polling-publisher/polling-service/src/main/java/com/iluwatar/polling/PollingScheduler.java (1 hunk)
• polling-publisher/polling-service/src/main/resources/application.yml (1 hunk)
• polling-publisher/polling-service/src/test/java/com/iluwatar/polling/AppTest.java (1 hunk)
• polling-publisher/polling-service/src/test/java/com/iluwatar/polling/DataRepositoryTest.java (1 hunk)
• polling-publisher/polling-service/src/test/java/com/iluwatar/polling/DataSourceServiceTest.java (1 hunk)
• polling-publisher/pom.xml (1 hunk)
• polling-publisher/subscriber-service/etc/subscriber-service.urm.puml (1 hunk)
• polling-publisher/subscriber-service/pom.xml (1 hunk)
• polling-publisher/subscriber-service/src/main/java/com/iluwatar/subscriber/App.java (1 hunk)
• polling-publisher/subscriber-service/src/main/java/com/iluwatar/subscriber/KafkaConsumer.java (1 hunk)
• polling-publisher/subscriber-service/src/main/resources/application.yml (1 hunk)
• polling-publisher/subscriber-service/src/test/java/com/iluwatar/subscriber/AppTest.java (1 hunk)
• pom.xml (1 hunk)

Actionable Comments (0)

Skipped Comments (0)

[sonarqubecloud]

Quality Gate failed

Failed conditions
2 Security Hotspots
79.7% Coverage on New Code (required ≥ 80%)
D Reliability Rating on New Code (required ≥ A)
E Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

[quantdevv]

Hi @iluwatar , can you please have a look on this..?