Go through all Kafka passwordless quickstart/migration-guide to make sure following them can work #30800
Assignees
Labels
Milestone
Comments
Quick start feedback:Link to quick start: https://review.docs.microsoft.com/en-us/azure/developer/java/spring-framework/configure-spring-cloud-stream-binder-java-app-kafka-azure-event-hub?branch=pr-en-us-3583&tabs=passwordless
1.1. Azure Active Directory (Azure AD) authentication
Event Hubs provides an endpoint compatible with the Apache Kafka® producer and consumer APIs. This quick start is not using Apache Kafka. My suggestion:
This problem has been solved by this:
Refs: https://github.com/git-for-windows/build-extra/blob/main/ReleaseNotes.md We can write this down in the doc in case other reader face the same problem.
|
Feedback of Migration guide
BTW, why this feature is not written in the changelog? https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/spring/CHANGELOG.md
8, Change link from .net to java Current link is about .net: https://review.docs.microsoft.com/en-us/dotnet/api/overview/azure/Identity-readme#defaultazurecredential
13... |
chenrujun
changed the title
Here is <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.azure.spring.password.less.migration.kafka.java</groupId>
<artifactId>before-migration</artifactId>
<version>1.0.0-SNAPSHOT</version>
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>3.2.3</version>
</dependency>
</dependencies>
</project>import org.apache.kafka.clients.CommonClientConfigs;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.Producer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.clients.producer.RecordMetadata;
import org.apache.kafka.common.config.SaslConfigs;
import org.apache.kafka.common.serialization.StringSerializer;
import java.util.Properties;
import java.util.concurrent.Future;
public class BeforeMigrationApplication {
public static void main(String[] args) {
try (Producer<String, String> producer = createProducer()) {
ProducerRecord<String, String> record = new ProducerRecord<>("sample-topic", "sample-key", "sample-value");
Future<RecordMetadata> future = producer.send(record);
RecordMetadata metadata = future.get();
System.out.println(metadata.toString());
} catch (Exception e) {
e.printStackTrace();
System.out.println("Error happened when produce message. " + e);
}
}
private static Producer<String, String> createProducer() {
String hostName = System.getProperty("hostName");
String connectionString = System.getProperty("connectionString");
Properties properties = new Properties();
properties.put(CommonClientConfigs.BOOTSTRAP_SERVERS_CONFIG, String.format("%s:9093", hostName));
properties.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, "SASL_SSL");
properties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
properties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
properties.put(SaslConfigs.SASL_MECHANISM, "PLAIN");
properties.put(SaslConfigs.SASL_JAAS_CONFIG,
String.format("org.apache.kafka.common.security.plain.PlainLoginModule required username=\"$ConnectionString\" password=\"%s\";", connectionString));
return new KafkaProducer<>(properties);
}
} |
|
|
Here is the code in my localhost: More context: <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.azure.spring.password.less.migration.kafka.java</groupId>
<artifactId>after-migration</artifactId>
<version>1.0.0-SNAPSHOT</version>
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>3.2.3</version>
</dependency>
<!-- <dependency>-->
<!-- <groupId>org.apache.kafka</groupId>-->
<!-- <artifactId>kafka_2.12</artifactId>-->
<!-- <version>2.3.1</version>-->
<!-- </dependency>-->
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>azure-client-authentication</artifactId>
<version>1.6.15</version>
</dependency>
</dependencies>
</project>java code: import org.apache.kafka.clients.CommonClientConfigs;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.Producer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.clients.producer.RecordMetadata;
import org.apache.kafka.common.config.SaslConfigs;
import java.util.Properties;
import java.util.concurrent.Future;
public class AfterMigrationApplication {
public static void main(String[] args) {
try (Producer<String, String> producer = createProducer()) {
ProducerRecord<String, String> record = new ProducerRecord<>("after-migration-topic", "after-migration--key", "after-migration--value");
Future<RecordMetadata> future = producer.send(record);
RecordMetadata metadata = future.get();
System.out.println(metadata.toString());
} catch (Exception e) {
e.printStackTrace();
System.out.println("Error happened when produce message. " + e);
}
}
private static Producer<String, String> createProducer() {
String hostName = System.getProperty("hostName");
Properties properties = new Properties();
properties.put(CommonClientConfigs.BOOTSTRAP_SERVERS_CONFIG, String.format("%s:9093", hostName));
properties.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, "SASL_SSL");
properties.put(SaslConfigs.SASL_MECHANISM, "OAUTHBEARER");
properties.put(SaslConfigs.SASL_JAAS_CONFIG, "org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required");
properties.put(SaslConfigs.SASL_LOGIN_CALLBACK_HANDLER_CLASS, "CustomAuthenticateCallbackHandler");
return new KafkaProducer<>(properties);
}
} |
Fixed by adding the 2 properties: properties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
properties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
|
Fixed by adding semi-colon: |
Code of import com.microsoft.azure.credentials.AzureCliCredentials;
import com.nimbusds.jwt.JWT;
import com.nimbusds.jwt.JWTClaimsSet;
import com.nimbusds.jwt.JWTParser;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.common.KafkaException;
import org.apache.kafka.common.security.auth.AuthenticateCallbackHandler;
import org.apache.kafka.common.security.oauthbearer.OAuthBearerToken;
import org.apache.kafka.common.security.oauthbearer.OAuthBearerTokenCallback;
import javax.security.auth.callback.Callback;
import javax.security.auth.callback.UnsupportedCallbackException;
import javax.security.auth.login.AppConfigurationEntry;
import java.io.IOException;
import java.net.URI;
import java.text.ParseException;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeoutException;
public class CustomAuthenticateCallbackHandler implements AuthenticateCallbackHandler {
final static AzureCliCredentials CREDENTIALS;
static {
try {
CREDENTIALS = AzureCliCredentials.create();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
// final static MSICredentials CREDENTIALS = new MSICredentials();
// Use AppServiceMSICredentials instead for App Service deployment.
// final static AppServiceMSICredentials CREDENTIALS = new AppServiceMSICredentials(AzureEnvironment.AZURE);
private String sbUri;
@Override
public void configure(Map<String, ?> configs, String mechanism, List<AppConfigurationEntry> jaasConfigEntries) {
String bootstrapServer = Arrays.asList(configs.get(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG)).get(0).toString();
bootstrapServer = bootstrapServer.replaceAll("\\[|\\]", "");
URI uri = URI.create("https://" + bootstrapServer);
this.sbUri = uri.getScheme() + "://" + uri.getHost();
}
public void handle(Callback[] callbacks) throws IOException, UnsupportedCallbackException {
for (Callback callback: callbacks) {
if (callback instanceof OAuthBearerTokenCallback) {
try {
OAuthBearerToken token = getOAuthBearerToken();
OAuthBearerTokenCallback oauthCallback = (OAuthBearerTokenCallback) callback;
oauthCallback.token(token);
} catch (InterruptedException | ExecutionException | TimeoutException | ParseException e) {
e.printStackTrace();
}
} else {
throw new UnsupportedCallbackException(callback);
}
}
}
OAuthBearerToken getOAuthBearerToken() throws InterruptedException, ExecutionException, TimeoutException, IOException, ParseException
{
String accessToken = CREDENTIALS.getToken(sbUri);
JWT jwt = JWTParser.parse(accessToken);
JWTClaimsSet claims = jwt.getJWTClaimsSet();
return new OAuthBearerTokenImp(accessToken, claims.getExpirationTime());
}
public void close() throws KafkaException {
// NOOP
}
}Log: This problem is not solved yet. |
Property like My suggestion: 2 options:
|
|
|
|
|
|
Reasons:
|
But there is no |
|
For example, we can use this link instead: https://github.com/microsoft/vscode-azure-account#signing-inout Otherwise, it's possible that the login step changed, and we (migration guide maintainer) will not know that. |
|
|
|
Reasons: |
This content can be replaced by this link: https://learn.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/qs-configure-portal-windows-vm#enable-system-assigned-managed-identity-on-an-existing-vm |
|
|
|
Screenshot in migration guide: Screenshot in Azure Portal: Workaround: assign role in Azure Spring Apps -> Apps page.
|
|
===========
Screenshots:
|
Here is screenshot of related log: |
2 tasks
|
I have finished go-through related documents. Closing this issue. Next step: #31346 |
|
The #30800 (comment) is tracked with microsoft/spring-cloud-azure#1037. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Context
With all Kafka passwordless-doc being finished, we should check the readability and correctness of those docs.
Goal
Go through the docs in the below PRs to make sure it is easy to understand and execute, and all the commands are correct and operational.
The text was updated successfully, but these errors were encountered: