Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE REQ] Simplify the connection related properties for blob checkpoint store in Spring Cloud Azure Event Hubs libraries #32035

Open
yiliuTo opened this issue Nov 9, 2022 · 1 comment
Assignees
Labels
azure-spring All azure-spring related issues azure-spring-boot Label for spring issues related with Spring Boot Auto-configuration azure-spring-eventhubs Spring event hubs related issues. azure-spring-zinc Client This issue points to a problem in the data-plane of the library. feature-request This issue requires a new behavior in the product in order be resolved.
Milestone

Comments

@yiliuTo
Copy link
Member

yiliuTo commented Nov 9, 2022

Context

When using Storage Blob as the checkpoint store in Spring Cloud Azure Event Hubs libraries, we provide multiple connection related properties to specify the target server entity to connect to. However, in some cases, the usage of those properties may be redundant and cloud be simplified.

The default and simplest usage case is that:

spring:
  cloud:
    azure:
      eventhubs:
        processor:
          checkpoint-store:
            container-name: ...
            account-name: ...
            account-key: ....

However, when there are requests to specify the Storage endpoint, e.g., the server is in Azure China cloud, then developers need to provide properties like:

spring:
  cloud:
    azure:
      eventhubs:
        processor:
          checkpoint-store:
            container-name: ...
            account-name: ...
            endpoint: https://<account-name>.blob.core.chinacloudapi.cn
            account-key: ....

where the endpoint property actually contains the information of account-name but we still require the account-name property to be configured.

Similar cases happen for the property of connection-string, the Storage connection string contains information about the endponit, account name and others. But let's still use the use case that a developer wants to connect to a Storage server in Azure China cloud, then required properties are:

spring:
  cloud:
    azure:
      eventhubs:
        processor:
          checkpoint-store:
            container-name: ...
            account-name: ...
            endpoint: https://<account-name>.blob.core.chinacloudapi.cn
            connection-string: ...

or

spring:
  cloud:
    azure:
      profile:
        cloud-type: azure_china
      eventhubs:
        processor:
          checkpoint-store:
            container-name: ...
            account-name: ...
            connection-string: ...

where multiple properties are required but provide redundant information.

Goal

We should simplify the required properties for the above cases and avoid that developers need to configure multiple properties but provide similar information .

@yiliuTo yiliuTo added Client This issue points to a problem in the data-plane of the library. feature-request This issue requires a new behavior in the product in order be resolved. azure-spring All azure-spring related issues labels Nov 9, 2022
@yiliuTo yiliuTo added this to the Backlog milestone Nov 9, 2022
@stliu
Copy link
Member

stliu commented Nov 24, 2022

nice catch, I think this is because we didn't aware of the different format that the endpoint support.

Nor we dont' have a clear definition of what value is expected for the checkpoint-store.endpoint

For example BlobServiceClientBuilder has this javadoc

 * The following information must be provided on this builder:
 *
 * <ul>
 * <li>the endpoint through {@code .endpoint()}, in the format of {@code https://{accountName}.blob.core.windows.net}.
 * <li>the credential through {@code .credential()} or {@code .connectionString()} if the container is not publicly
 * accessible.
 * </ul>

BlobClientBuilder has this javadoc

 * <li>the endpoint through {@code .endpoint()}, including the container name and blob name, in the format of
 * {@code https://{accountName}.blob.core.windows.net/{containerName}/{blobName}}.
 * <li>the credential through {@code .credential()} or {@code .connectionString()} if the container is not publicly
 * accessible.
 * </ul>
 */

but our doc

Image

@stliu stliu added the azure-spring-eventhubs Spring event hubs related issues. label Dec 10, 2022
@saragluna saragluna modified the milestones: Backlog, 2023-02 Dec 30, 2022
@yiliuTo yiliuTo self-assigned this Jan 6, 2023
@yiliuTo yiliuTo added azure-spring-boot Label for spring issues related with Spring Boot Auto-configuration azure-spring-zinc labels Jan 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
azure-spring All azure-spring related issues azure-spring-boot Label for spring issues related with Spring Boot Auto-configuration azure-spring-eventhubs Spring event hubs related issues. azure-spring-zinc Client This issue points to a problem in the data-plane of the library. feature-request This issue requires a new behavior in the product in order be resolved.
Projects
Status: Todo
Development

No branches or pull requests

3 participants