Julie doesn't like Placement contraints #241
Labels
Comments
|
Update: I got curious and did some more digging: We start again using a "plain" descriptor That won't get any constraint settings Now I add the following to the very same descriptor: And rerun julie and voila: Hope that helps somehow in finding the error 😅 |
Fobhep
changed the title
|
another update - I do realize now also that Julie uses not the default replicator factor that is set in the broker - in contrast to the manual topic creation |
|
Hi @Fobhep, thanks for reporting this. FYI: we never tested this tool with Confluent replica placements, so the behaviour and thought are completely green field. |
yap, this is true. Do you think it would be better to rely on cluster defaults? |
|
ok so - I would then open another issue to discuss the defaults 😅 and we can leave this one open for the constraints |
Fobhep
changed the title
|
I am actually happy to change it if useful for others! :-)
Missatge de Fobhep ***@***.***> del dia dt., 30 de març 2021
a les 11:17:
… ok so - I would then open another issue to discuss the defaults 😅 and we
can leave this one open for the constraints
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#241 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAQXPE4VCJNJMPFULVOJH3TGGJKFANCNFSM42AOGSXQ>
.
--
Pere Urbon-Bayes
Software Architect
https://twitter.com/purbon
https://www.linkedin.com/in/purbon/
|
|
Moin, with merge if #243, there not anymore the always on use of replicationFactor and partitionCount, this makes me thing now this would work as expected. Do you mind testing this for me? that would be very much appreciated. You can find the latest releases (nightly build) here: nightly build: https://github.com/kafka-ops/julie/actions/runs/704912720 As well as in docker hub https://hub.docker.com/repository/docker/purbon/kafka-topology-builder (latest tag). Thanks again for all your help! -- Pere |
|
Ola - so I had now the time to do some test. This is my properties file: I try to deploy this topology: And this is the error I receive: I can use the exact same properties file! without any problems with builtin kafka-admin-tools like |
|
Update: I changed my cluster to GSSAPI and retried testing: Again builtin admin-client tools work just fine. Julie: |
|
@Fobhep I tried reproducing your SSL issue in my environment, however, I can't seem to manage to do it. I setup an RBAC on TLS cluster, installed in another redhat machine the lastest julie release 2.1.0. and all 🍏 julie-ops --brokers kafka.operator.purbon.de:9092 --clientConfig topology-builder-rbac-tls.properties --topology descriptor.yml
[INFO ] 2021-04-29 10:53:44.702 [main] TopologyBuilderAdminClientBuilder - Connecting AdminClient to kafka.operator.purbon.de:9092
[INFO ] 2021-04-29 10:53:44.702 [main] TopologyBuilderAdminClientBuilder - Connecting AdminClient to kafka.operator.purbon.de:9092
log4j:WARN No appenders could be found for logger (org.apache.kafka.clients.admin.AdminClientConfig).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
[INFO ] 2021-04-29 10:53:46.025 [main] MDSApiClientBuilder - Connecting to an MDS server at https://kafka.operator.purbon.de:443
[INFO ] 2021-04-29 10:53:46.025 [main] MDSApiClientBuilder - Connecting to an MDS server at https://kafka.operator.purbon.de:443
[INFO ] 2021-04-29 10:53:46.025 [main] MDSApiClientBuilder - Connecting to an MDS server at https://kafka.operator.purbon.de:443
[INFO ] 2021-04-29 10:53:46.184 [main] MDSApiClientBuilder - Connecting to an MDS server at https://kafka.operator.purbon.de:443
[INFO ] 2021-04-29 10:53:46.184 [main] MDSApiClientBuilder - Connecting to an MDS server at https://kafka.operator.purbon.de:443
[INFO ] 2021-04-29 10:53:46.184 [main] MDSApiClientBuilder - Connecting to an MDS server at https://kafka.operator.purbon.de:443
List of Topics:
_confluent-controlcenter-0-MetricsAggregateStore-repartition
_confluent-controlcenter-0-MonitoringMessageAggregatorWindows-THREE_HOURS-changelog
operator.connectors-configs
_confluent-controlcenter-0-aggregatedTopicPartitionTableWindows-ONE_MINUTE-changelog
_confluent-controlcenter-0-KSTREAM-OUTEROTHER-0000000106-store-repartition
_confluent-controlcenter-0-TriggerEventsStore-repartition
_confluent-license
_confluent-controlcenter-0-Group-ONE_MINUTE-changelog
_confluent-controlcenter-0-MonitoringStream-THREE_HOURS-repartition
context.foo.foo
_confluent-controlcenter-0-aggregate-topic-partition-store-changelog
_confluent-controlcenter-0-MonitoringMessageAggregatorWindows-ONE_MINUTE-repartition
_confluent-controlcenter-0-AlertHistoryStore-repartition
operator.connectors-status
_confluent-controlcenter-0-MonitoringStream-ONE_MINUTE-repartition
_confluent-controlcenter-0-monitoring-message-rekey-store
_confluent-metrics
_confluent-controlcenter-0-TriggerEventsStore-changelog
_confluent-controlcenter-0-aggregate-topic-partition-store-repartition
_confluent-controlcenter-0-MonitoringMessageAggregatorWindows-ONE_MINUTE-changelog
_confluent-controlcenter-0-aggregatedTopicPartitionTableWindows-THREE_HOURS-changelog
_confluent-controlcenter-0-MonitoringTriggerStore-changelog
_confluent-controlcenter-0-MonitoringVerifierStore-repartition
_confluent-metadata-auth
_confluent-controlcenter-0-Group-THREE_HOURS-repartition
_confluent-controlcenter-0-group-aggregate-store-ONE_MINUTE-repartition
_confluent-command
_confluent-controlcenter-0-aggregatedTopicPartitionTableWindows-ONE_MINUTE-repartition
_confluent-controlcenter-0-MonitoringTriggerStore-repartition
_confluent-controlcenter-0-Group-THREE_HOURS-changelog
_confluent-controlcenter-0-TriggerActionsStore-repartition
operator.connectors-offsets
.... redacted...
_confluent-monitoring
_confluent-telemetry-metrics
List of ACLs:
List of Principles:
Kafka Topology updatedi guess you do as well, and the error reported is certainly not related, but I pass the SSL keystores globally for the MDS server like this. export JULIE_OPS_OPTIONS=-Djavax.net.ssl.keyStore=/tmp/keystore.jks -Djavax.net.ssl.keyStorePassword=password -Djavax.net.ssl.trustStore=/tmp/truststore.jks -Djavax.net.ssl.trustStorePassword=passwordproperty file is very similar to yours. topology.builder.access.control.class = com.purbon.kafka.topology.roles.RBACProvider
topology.builder.mds.server = https://kafka.operator.purbon.de:443
topology.builder.mds.user = kafka
topology.builder.mds.password = kafka-secret
topology.builder.mds.kafka.cluster.id = <cluster-id>
topology.builder.mds.schema.registry.cluster.id = schema-registry-cluster
topology.builder.mds.kafka.connect.cluster.id = connect-cluster
security.protocol=SSL
ssl.truststore.location=/tmp/truststore.jks
ssl.truststore.password=password
ssl.keystore.location=/tmp/keystore.jks
ssl.keystore.password=passwordThe error mentioned it really looks as a dependency is missing, but it would as well fail on my test as I did. java.lang.NoSuchMethodError: org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.castOrThrow(Lorg/apache/kafka/common/security/auth/SslEngineFactory;)Lorg/apache/kafka/common/security/ssl/DefaultSslEngineFactory;
at io.confluent.security.auth.client.rest.RestClient.createSslSocketFactory(RestClient.java:176)how does the call you do look like? which version are you using? java version? |
|
I guess I see where the error might come, I am testing with pure SSL, but you are using OAUTHBEARER. sasl.mechanism=OAUTHBEARER
sasl.login.callback.handler.class=io.confluent.kafka.clients.plugins.auth.token.TokenUserLoginCallbackHandler
sasl.jaas.config=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required \
username="user1" password="1234" \
metadataServerUrls="https://test-broker:8090";" |
|
I am able to reproduce the problem with the OAUTH config you provided. Caused by: org.apache.kafka.common.KafkaException: java.lang.NoSuchMethodError: org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.castOrThrow(Lorg/apache/kafka/common/security/auth/SslEngineFactory;)Lorg/apache/kafka/common/security/ssl/DefaultSslEngineFactory;
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:172)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:157)
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:73)
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:105)
at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:508)
... 7 more
Caused by: java.lang.NoSuchMethodError: org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.castOrThrow(Lorg/apache/kafka/common/security/auth/SslEngineFactory;)Lorg/apache/kafka/common/security/ssl/DefaultSslEngineFactory;
at io.confluent.security.auth.client.rest.RestClient.createSslSocketFactory(RestClient.java:176)
at io.confluent.security.auth.client.rest.RestClient.<init>(RestClient.java:126)
at io.confluent.security.auth.client.rest.RestClient.<init>(RestClient.java:96)
at io.confluent.kafka.clients.plugins.auth.token.TokenUserLoginCallbackHandler.configure(TokenUserLoginCallbackHandler.java:67)
at io.confluent.kafka.clients.plugins.auth.token.AbstractTokenLoginCallbackHandler.configure(AbstractTokenLoginCallbackHandler.java:86)investigating |
|
problem looks to be introduced by 402553e and the shading process after it to build the final fat jar. |
|
@Fobhep I was able to sort out the problem you experienced here and I fixed them f2cd0c5 the RCA was a dependency problem, from 5.5 to 6.0 the SSL dependencies changes, so i now use all the jars coming out of confluent directly to avoid any problem with and RBAC usage. This should be fixed with the previously commented commit, I will close this for now, feel free to reopen if necessary. NOTE: this fix will be available in the next bug fix release available in the normal channels. |
|
@purbon I retested this and sadly also in Julie version 2.1.2 Julie is not accepting globally set placements contraints. descriptor.yaml |
|
@Fobhep do you see any particular error from julie ? |
|
no errors - nope |
|
@purbon let me know if you have any specific suggestions on how to turn on debug log for Julie that would provide more information |
|
I did a test with the latest version of Julie, using ---
context: "o"
projects:
- name: "f"
consumers:
- principal: "User:NewApp2"
topics:
- name: "t"
config:
confluent.placement.constraints: "{\"version\":1,\"replicas\":[{\"count\":1,\"constraints\":{\"rack\":\"rack-1\"}},{\"count\":1,\"constraints\":{\"rack\":\"rack-2\"}}],\"observers\":[]}"and you can see the placement constraints properly set. $ docker exec kafka kafka-topics --bootstrap-server kafka:29092 \ 2.7.0
--describe --topic o.f.t
Topic: o.f.t TopicId: dJImanTbSd2sbUjLVDMoVA PartitionCount: 1 ReplicationFactor: 2 Configs: confluent.placement.constraints={"version":1,"replicas":[{"count":1,"constraints":{"rack":"rack-1"}},{"count":1,"constraints":{"rack":"rack-2"}}],"observers":[]}
Topic: o.f.t Partition: 0 Leader: 1 Replicas: 1,2 Isr: 1,2 Offline:I do have the feeling this was caused when Julie use to send a specific replication factor, but now when not configured, as seen in the example, all set as expected. As you can see when using this topology: ---
context: "o"
projects:
- name: "f"
consumers:
- principal: "User:NewApp2"
topics:
- name: "b"
config:
replication.factor: "1"
confluent.placement.constraints: "{\"version\":1,\"replicas\":[{\"count\":1,\"constraints\":{\"rack\":\"rack-1\"}},{\"count\":1,\"constraints\":{\"rack\":\"rack-2\"}}],\"observers\":[]}"that causes this error: re set. Both cannot be used at the same time.
java.io.IOException: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.InvalidRequestException: Both replicationFactor and confluent.placement.constraints are set. Both cannot be used at the same time.
at com.purbon.kafka.topology.api.adminclient.TopologyBuilderAdminClient.createTopic(TopologyBuilderAdminClient.java:200)
at com.purbon.kafka.topology.actions.topics.CreateTopicAction.createTopic(CreateTopicAction.java:41)
at com.purbon.kafka.topology.actions.topics.CreateTopicAction.run(CreateTopicAction.java:36)
at com.purbon.kafka.topology.ExecutionPlan.execute(ExecutionPlan.java:126)
at com.purbon.kafka.topology.ExecutionPlan.run(ExecutionPlan.java:101)
at com.purbon.kafka.topology.JulieOps.run(JulieOps.java:212)
at com.purbon.kafka.topology.JulieOps.run(JulieOps.java:227)
at com.purbon.kafka.topology.CommandLineInterface.processTopology(CommandLineInterface.java:212)
at com.purbon.kafka.topology.CommandLineInterface.run(CommandLineInterface.java:161)
at com.purbon.kafka.topology.CommandLineInterface.main(CommandLineInterface.java:147)
Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.InvalidRequestException: Both replicationFactor and confluent.placement.constraints are set. Both cannot be used at the same time.
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:165)
at com.purbon.kafka.topology.api.adminclient.TopologyBuilderAdminClient.createAllTopics(TopologyBuilderAdminClient.java:211)
at com.purbon.kafka.topology.api.adminclient.TopologyBuilderAdminClient.createTopic(TopologyBuilderAdminClient.java:193)
... 9 more
Caused by: org.apache.kafka.common.errors.InvalidRequestException: Both replicationFactor and confluent.placement.constraints are set. Both cannot be used at the same time.as seen in this issue. closing this issue for now, feel free to reopen if necessary. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
I am trying to use Julie together with replica placement contraints and that seems to raise some errors.
To Reproduce
My brokers have the following config:
So if I create a topic
test1manually - the constraints will be respected:However if I create now a topic via julie using the following descriptor:
descriptor.yaml
Sadly the constraints seem not to be respected:
If I go further and try to force a constraint on the topic descriptor like this:
I get this error
The text was updated successfully, but these errors were encountered: