Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatic schema update can be disabled through admin interface #2691

Merged
merged 6 commits into from Oct 12, 2018

Conversation

ivankelly
Copy link
Contributor

Currently when a producer connects to a topic with a new schema, if
that schema is fully compatible with the current schema, the schema
for the topic is updated to the new schema.

This may not be desirable in some cases.

This patch allows users to set the auto update schema compatibility
strategy at the namespace level. There are 4 compatibility levels,
disabled, backward, forward and full.

Currently when a producer connects to a topic with a new schema, if
that schema is fully compatible with the current schema, the schema
for the topic is updated to the new schema.

This may not be desirable in some cases.

This patch allows users to set the auto update schema compatibility
strategy at the namespace level. There are 4 compatibility levels,
disabled, backward, forward and full.
@ivankelly ivankelly self-assigned this Oct 1, 2018
@ivankelly
Copy link
Contributor Author

rerun cpp tests
rerun integration tests
rerun java8 tests

@ivankelly
Copy link
Contributor Author

rerun java8 tests

1 similar comment
@ivankelly
Copy link
Contributor Author

rerun java8 tests

Copy link
Member

@sijie sijie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall looks good to me. one comment requires clarifications.

@@ -61,6 +61,9 @@
public long offload_threshold = -1;
public Long offload_deletion_lag_ms = null;

public SchemaAutoUpdateCompatibilityStrategy schema_auto_update_compatibility_strategy =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if a namespace was created by pulsar before this change, and we upgrade pulsar to apply this change, what will happen to the existing clients?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

default to full, which is the same behaviour as now.

@sijie sijie added type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages component/schemaregistry labels Oct 3, 2018
@sijie sijie added this to the 2.3.0 milestone Oct 3, 2018
@ivankelly
Copy link
Contributor Author

rerun integration tests

@ivankelly
Copy link
Contributor Author

rerun integration tests

@sijie
Copy link
Member

sijie commented Oct 9, 2018

@merlimat can you review this PR?

AutoUpdateDisabled,

/**
* Messages written in the new schema can be read by the previous schema.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we expand the comments to clarify in terms of what happens when you try to use a producer with a newer/older version of the schema?

Messages written in the new schema can be read by the previous schema.

Does that mean that it will update the topic schema only if the new one can be read by a an old client?

@ivankelly
Copy link
Contributor Author

rerun java8 tests
rerun cpp tests

@ivankelly
Copy link
Contributor Author

@merlimat could you take another look? I have another patch with the CLI waiting for this to merge.

@ivankelly
Copy link
Contributor Author

rerun java8 tests

1 similar comment
@ivankelly
Copy link
Contributor Author

rerun java8 tests

@ivankelly ivankelly merged commit 966c054 into apache:master Oct 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants