Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DryRun mode outputs Kafka AdminClient logs #548

Closed
dimmy-timmy opened this issue Oct 20, 2022 · 2 comments · Fixed by sverrehu/julie#5
Closed

DryRun mode outputs Kafka AdminClient logs #548

dimmy-timmy opened this issue Oct 20, 2022 · 2 comments · Fixed by sverrehu/julie#5
Labels
bug Something isn't working

Comments

@dimmy-timmy
Copy link

Dry-run mode julie-ops-cli.sh --clientConfig julie.properties --dryRun --overridingClientConfig kafka.properties --topology topics.yaml outputs Kafka AdminClient logs

Run julie-ops from official docker image version 4.4 or latest in dry-run mode
It will output Kafka client log together with json diff

08:15:16.290 [main] DEBUG org.apache.kafka.common.security.ssl.DefaultSslEngineFactory - Created SSL context with keystore null, truststore null, provider SunJSSE.
08:15:16.318 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'awt.toolkit' was supplied but isn't a known config.
08:15:16.318 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'topology.topic.prefix.separator' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'java.specification.version' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'julie.s3.endpoint' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'sun.cpu.isalist' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'sun.jnu.encoding' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'topology.project.prefix.format' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'julie.kafka.config.topic' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'confluent.command.topic' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'sun.arch.data.model' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'topology.dlq.topics.generate' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'allow.delete.artefacts.ksql' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'topology.file.type' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'sun.boot.library.path' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'sun.java.command' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'jdk.debug' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'java.specification.vendor' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'java.home' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'file.separator' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'topology.builder.state.processor.class' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'java.vm.compressedOopsMode' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'line.separator' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'java.specification.name' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'java.vm.specification.vendor' was supplied but isn't a known config.
08:15:16.319 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'julie.enable.principal.management' was supplied but isn't a known config.
08:15:16.320 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'topology.subject.managed.prefixes' was supplied but isn't a known config.
08:15:16.320 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'confluent.monitoring.topic' was supplied but isn't a known config.
08:15:16.320 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'sun.management.compiler' was supplied but isn't a known config.
08:15:16.320 [main] WARN org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'java.runtime.version' was supplied but isn't a known config.
......................ommited.............................
08:15:18.420 [kafka-admin-client-thread | adminclient-1] DEBUG org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Sending DESCRIBE_ACLS request with header RequestHeader(apiKey=DESCRIBE_ACLS, apiVersion=2, clientId=adminclient-1, correlationId=170) and timeout 30000 to node 2: {resource_type_filter=1,resource_name_filter=null,pattern_type_filter=1,principal_filter=null,host_filter=null,operation=1,permission_type=1,_tagged_fields={}}
08:15:18.422 [kafka-admin-client-thread | adminclient-1] DEBUG org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Received DESCRIBE_ACLS response from node 2 for request with header RequestHeader(apiKey=DESCRIBE_ACLS, apiVersion=2, clientId=adminclient-1, correlationId=170): org.apache.kafka.common.requests.DescribeAclsResponse@3f4880f4
[WARN ] 2022-10-20 08:15:18.449 [main] AccessControlManager - Remote state verification disabled, this is not a good practice, be awarein future versions, this check is going to become mandatory.
[WARN ] 2022-10-20 08:15:18.449 [main] AccessControlManager - Remote state verification disabled, this is not a good practice, be awarein future versions, this check is going to become mandatory.
[WARN ] 2022-10-20 08:15:18.458 [main] ArtefactManager - Remote state verification disabled, this is not a good practice, be awarein future versions, this check is going to become mandatory.
[WARN ] 2022-10-20 08:15:18.458 [main] ArtefactManager - Remote state verification disabled, this is not a good practice, be awarein future versions, this check is going to become mandatory.
[WARN ] 2022-10-20 08:15:18.462 [main] ArtefactManager - Remote state verification disabled, this is not a good practice, be awarein future versions, this check is going to become mandatory.
[WARN ] 2022-10-20 08:15:18.462 [main] ArtefactManager - Remote state verification disabled, this is not a good practice, be awarein future versions, this check is going to become mandatory.
{
  "Operation" : "com.purbon.kafka.topology.actions.topics.UpdateTopicConfigAction",
  "Topic" : "dev.cs.cdr",
  "Action" : "update",
  "Changes" : {
    "UpdatedPartitionCount" : 6
  }
}

Expected behavior
It should display just diff, like it was previosly (docker image with tag 4.2.9, 4.3.0)

[WARN ] 2022-10-20 16:31:54.493 [main] TopicManager - Remote state verification disabled, this is not a good practice, be awarein future versions, this check is going to become mandatory.
[WARN ] 2022-10-20 16:31:54.493 [main] TopicManager - Remote state verification disabled, this is not a good practice, be awarein future versions, this check is going to become mandatory.
[WARN ] 2022-10-20 16:31:55.106 [main] AccessControlManager - Remote state verification disabled, this is not a good practice, be awarein future versions, this check is going to become mandatory.
[WARN ] 2022-10-20 16:31:55.106 [main] AccessControlManager - Remote state verification disabled, this is not a good practice, be awarein future versions, this check is going to become mandatory.
[WARN ] 2022-10-20 16:31:55.110 [main] ArtefactManager - Remote state verification disabled, this is not a good practice, be awarein future versions, this check is going to become mandatory.
[WARN ] 2022-10-20 16:31:55.110 [main] ArtefactManager - Remote state verification disabled, this is not a good practice, be awarein future versions, this check is going to become mandatory.
[WARN ] 2022-10-20 16:31:55.113 [main] ArtefactManager - Remote state verification disabled, this is not a good practice, be awarein future versions, this check is going to become mandatory.
[WARN ] 2022-10-20 16:31:55.113 [main] ArtefactManager - Remote state verification disabled, this is not a good practice, be awarein future versions, this check is going to become mandatory.
{
  "Operation" : "com.purbon.kafka.topology.actions.topics.UpdateTopicConfigAction",
  "Topic" : "test-topic",
  "Action" : "update",
  "Changes" : {
    "UpdatedPartitionCount" : 6
  }
}
@dimmy-timmy dimmy-timmy added the bug Something isn't working label Oct 20, 2022
@hansh0801
Copy link

track

@hansh0801
Copy link

@dimmy-timmy did you solved this issue?

sverrehu added a commit to sverrehu/julie that referenced this issue Mar 29, 2023
The Zookeeper dependency is required for a single test method only. This PR moves all ZK* classes to src/test, and marks the zookeeper dependency as test-coped.

Unfortunately, recent versions of zookeeper drag in a logback dependency, that appears to turn on global DEBUG logging. So this PR also exludes those logback dependencies.

Fixes kafka-ops#548
sverrehu added a commit to sverrehu/julie that referenced this issue Sep 12, 2023
The Zookeeper dependency is required for a single test method only. This PR moves all ZK* classes to src/test, and marks the zookeeper dependency as test-coped.

Unfortunately, recent versions of zookeeper drag in a logback dependency, that appears to turn on global DEBUG logging. So this PR also exludes those logback dependencies.

Fixes kafka-ops#548
@purbon purbon closed this as completed in 3fae19d Sep 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants