Adding support for M1 #35

oleksandrivan · 2023-09-27T13:01:31Z

Hello.
As in this PR there is an upgrade to RocksDB library to support M1 ARM architecture, this changes were not propagated to this repository causing it to fail when trying the solution of the KTables exercise.
With this PR the streams library is updated so now it correctly works.

cla-assistant · 2023-09-27T13:01:43Z

All committers have signed the CLA.

bbejeck

Thanks for the PR @oleksandrivan - we just need to revert the changes indicated below.

bbejeck · 2023-10-02T15:23:53Z

kafka-streams/build.gradle

    implementation('io.confluent:kafka-streams-avro-serde:6.1.1') {
        exclude group: 'org.apache.kafka', module: 'kafka-clients'
        exclude group: 'org.apache.kafka', module: 'kafka-streams'
    }

-    testImplementation "org.apache.kafka:kafka-streams-test-utils:2.8.0"
+    testImplementation "org.apache.kafka:kafka-streams-test-utils:3.3.0"
    testImplementation "junit:junit:4.13.2"


Might as well go with 3.5.1 for all of these since that's currently the latest version

Updated to 3.5.1

bbejeck · 2023-10-02T15:25:17Z

kafka-streams/docker/docker-compose.yml

+    hostname: broker
+    container_name: broker
+    ports:
+      - 29092:29092


Suggested change

- 29092:29092

- 9092:9092

This way, users can use the more familiar 9092 port for connecting to Kafka

For this change I'll create another PR with the appropiate documentation so it can be ran locally.

bbejeck · 2023-10-02T15:26:08Z

kafka-streams/docker/docker-compose.yml

+    environment:
+      KAFKA_BROKER_ID: 1
+      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,CONTROLLER:PLAINTEXT
+      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:9092,PLAINTEXT_HOST://localhost:29092


Suggested change

KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:9092,PLAINTEXT_HOST://localhost:29092

KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092

This way, users can use the more familiar 9092 port for connecting to Kafka

bbejeck · 2023-10-02T15:27:18Z

kafka-streams/docker/docker-compose.yml

+      KAFKA_PROCESS_ROLES: broker,controller
+      KAFKA_NODE_ID: 1
+      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@broker:29093
+      KAFKA_LISTENERS: PLAINTEXT://broker:9092,CONTROLLER://broker:29093,PLAINTEXT_HOST://0.0.0.0:29092


Suggested change

KAFKA_LISTENERS: PLAINTEXT://broker:9092,CONTROLLER://broker:29093,PLAINTEXT_HOST://0.0.0.0:29092

KAFKA_LISTENERS: PLAINTEXT://broker:29092,CONTROLLER://broker:29093,PLAINTEXT_HOST://0.0.0.0:9092

bbejeck · 2023-10-02T15:27:40Z

kafka-streams/docker/docker-compose.yml

+      - 8081:8081
+    environment:
+      SCHEMA_REGISTRY_HOST_NAME: schema-registry
+      SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: broker:9092


Suggested change

SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: broker:9092

SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: broker:29092

bbejeck · 2023-10-02T17:14:12Z

kafka-streams/src/main/java/io/confluent/developer/aggregate/StreamsAggregate.java

@@ -39,7 +39,12 @@ public static void main(String[] args) throws IOException {

        // Now take the electronicStream object, group by key and perform an aggregation
        // Don't forget to convert the KTable returned by the aggregate call back to a KStream using the toStream method
-        electronicStream.groupByKey().aggregate(null, null);


These tutorials were intentionally left incomplete, so users can finish the code on their own. Full solutions are found in the solution package for each tutorial.

Yep sorry, I pushed to my main branch after completing the course and the PR auto-updated to the latest commit.

bbejeck · 2023-10-02T17:14:40Z

kafka-streams/src/main/java/io/confluent/developer/errors/StreamsErrorHandling.java

            // This return null statement is here so the code will compile
            // You need to replace it with some logic described below
-            return null;
+            return DeserializationHandlerResponse.CONTINUE;


same as above

bbejeck · 2023-10-02T17:15:31Z

kafka-streams/src/main/java/io/confluent/developer/errors/StreamsErrorHandling.java

+            if(exception instanceof RecordTooLargeException){
+                return ProductionExceptionHandlerResponse.CONTINUE;
+            }
+            return ProductionExceptionHandlerResponse.FAIL;
            // If the exception type is RecordTooLargeException continue working


Same as above - there are a bunch of other changes below that follow the same pattern, but I won't repeat the comments.

oleksandrivan · 2023-10-03T15:58:54Z

kafka-streams/gradle/wrapper/gradle-wrapper.properties

@@ -1,5 +1,5 @@
 distributionBase=GRADLE_USER_HOME
 distributionPath=wrapper/dists
-distributionUrl=https\://services.gradle.org/distributions/gradle-7.0-bin.zip
+distributionUrl=https\://services.gradle.org/distributions/gradle-7.3-bin.zip


Bumped gradle so it uses Java 17 as Java 16 is non-LTS and no longer has support.

oleksandrivan · 2023-10-03T15:59:49Z

kafka-streams/gradlew.bat

Auto-generated by ./gradlew wrapper --gradle-version=7.3

bbejeck

LGTM! Thanks for the contribution @oleksandrivan !

Adding support for M1

c8b4375

oleksandrivan requested a review from a team as a code owner September 27, 2023 13:01

oleksandrivan mentioned this pull request Sep 27, 2023

Kafka Streams: Building on M1 mac fails #33

Open

bbejeck requested changes Oct 2, 2023

View reviewed changes

Kafka to use 3.5.1 and gradle to 7.3

fd09d8c

oleksandrivan force-pushed the main branch from 7e9214d to fd09d8c Compare October 3, 2023 15:51

oleksandrivan commented Oct 3, 2023

View reviewed changes

kafka-streams/gradlew.bat

Copy link

Contributor Author

oleksandrivan Oct 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto-generated by ./gradlew wrapper --gradle-version=7.3

bbejeck approved these changes Oct 3, 2023

View reviewed changes

bbejeck merged commit 6ab7a39 into confluentinc:main Oct 3, 2023
1 check failed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding support for M1 #35

Adding support for M1 #35

oleksandrivan commented Sep 27, 2023

cla-assistant bot commented Sep 27, 2023 •

edited

Loading

bbejeck left a comment

bbejeck Oct 2, 2023

oleksandrivan Oct 3, 2023

bbejeck Oct 2, 2023

oleksandrivan Oct 3, 2023

bbejeck Oct 2, 2023

bbejeck Oct 2, 2023

bbejeck Oct 2, 2023

bbejeck Oct 2, 2023

oleksandrivan Oct 3, 2023

bbejeck Oct 2, 2023

bbejeck Oct 2, 2023

oleksandrivan Oct 3, 2023

oleksandrivan Oct 3, 2023

bbejeck left a comment

	KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:9092,PLAINTEXT_HOST://localhost:29092
	KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092

	KAFKA_LISTENERS: PLAINTEXT://broker:9092,CONTROLLER://broker:29093,PLAINTEXT_HOST://0.0.0.0:29092
	KAFKA_LISTENERS: PLAINTEXT://broker:29092,CONTROLLER://broker:29093,PLAINTEXT_HOST://0.0.0.0:9092

	SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: broker:9092
	SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: broker:29092

Adding support for M1 #35

Adding support for M1 #35

Conversation

oleksandrivan commented Sep 27, 2023

cla-assistant bot commented Sep 27, 2023 • edited Loading

bbejeck left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bbejeck left a comment

Choose a reason for hiding this comment

cla-assistant bot commented Sep 27, 2023 •

edited

Loading