-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
error creating Tag Binding / Business Metadata Binding 404 #282
Comments
Thanks for creating an issue @marconak-itera! #253 looks relevant to this issue, we'll take a look at it shortly. |
This was fixed in @marconak-itera could you try again and see whether you can reproduce the issue? Thanks! |
It is more stable as before. Most of terraform runs finished successfully. But there was run which finished with error, we were creating the topic, schema, tags and metadata for topic and schema in one run: logs: module.custom_resource_name_topic.confluent_kafka_topic.main: Creating...
module.custom_resource_name_schema.confluent_schema.main: Creating...
module.custom_resource_name_topic.confluent_kafka_topic.main: Still creating... [10s elapsed]
module.custom_resource_name_schema.confluent_schema.main: Still creating... [10s elapsed]
module.custom_resource_name_topic.confluent_kafka_topic.main: Creation complete after 11s [id=cluster-id/topic-name]
module.custom_resource_name_topic.confluent_tag_binding.tag_topic_entity_type["open"]: Creating...
module.custom_resource_name_topic.confluent_tag_binding.tag_topic_entity_type["regular"]: Creating...
module.custom_resource_name_topic.confluent_tag_binding.tag_topic_entity_type["s"]: Creating...
module.custom_resource_name_topic.confluent_tag_binding.tag_topic_entity_type["no"]: Creating...
module.custom_resource_name_topic.confluent_tag_binding.tag_topic_entity_type["cdc"]: Creating...
module.custom_resource_name_topic.confluent_business_metadata_binding.business_metadata_topic_entity_type["Team"]: Creating...
module.custom_resource_name_schema.confluent_schema.main: Creation complete after 11s [id=cluster-id/topic-name-value/latest]
module.custom_resource_name_schema.confluent_tag_binding.tag_schema_entity_type["cdc"]: Creating...
module.custom_resource_name_schema.confluent_tag_binding.tag_schema_entity_type["regular"]: Creating...
module.custom_resource_name_schema.confluent_tag_binding.tag_schema_entity_type["open"]: Creating...
module.custom_resource_name_schema.confluent_tag_binding.tag_schema_entity_type["no"]: Creating...
module.custom_resource_name_schema.confluent_tag_binding.tag_schema_entity_type["s"]: Creating...
module.custom_resource_name_schema.confluent_business_metadata_binding.business_metadata_schema_entity_type["Team"]: Creating...
module.custom_resource_name_schema.confluent_tag_binding.tag_schema_entity_type["cdc"]: Creation complete after 6s [id=cluster-id/cdc/cluster-id:.:id/sr_schema]
module.custom_resource_name_schema.confluent_tag_binding.tag_schema_entity_type["open"]: Creation complete after 6s [id=cluster-id/open/cluster-id:.:id/sr_schema]
module.custom_resource_name_schema.confluent_tag_binding.tag_schema_entity_type["no"]: Creation complete after 6s [id=cluster-id/gf/cluster-id:.:id/sr_schema]
module.custom_resource_name_schema.confluent_tag_binding.tag_schema_entity_type["s"]: Creation complete after 6s [id=cluster-id/s/cluster-id:.:id/sr_schema]
module.custom_resource_name_schema.confluent_tag_binding.tag_schema_entity_type["regular"]: Creation complete after 7s [id=cluster-id/regular_retention/cluster-id:.:id/sr_schema]
module.custom_resource_name_schema.confluent_business_metadata_binding.business_metadata_schema_entity_type["Team"]: Creation complete after 8s [id=cluster-id/Team/cluster-id:.:id/sr_schema] which failed on creating tags and metadata for topic, the tags and metadata for schema were created:
|
@marconak-itera thanks for testing it out! Could you try new version 1.50.0: https://github.com/confluentinc/terraform-provider-confluent/blob/master/CHANGELOG.md#1500-july-21-2023 too when you've got a moment? We've increased timeouts so it should be even more reliable. Thanks! |
im still receiving 404 errors on tag binding creation when using version 1.50.0 @linouk23 |
Hi @chawleejay By any chance do you still remember how long it took to complete all provisioning after you hit 404? |
its happening instantly after the creation of topic @zhenli00 . Once i hit the 404 i run the code again and it runs successfully. When i try to create a new topic and add tags to it im getting the error but once the topic is created it works |
@chawleejay could you try Thank you for bearing with us! |
@linouk23 darn! tried |
`
confluent-test Terraform used the selected providers to generate the following execution plan.
confluent-test 0 Stacks deploying 1 Stack done 0 Stacks waiting |
@chawleejay When you specify the topic in entity_name, did you reference the topic resource or just hard code the topic name? |
@zhenli00 - It's a reference. (@chawleejay and I work together). It seems like what's happening is the schema registry is not yet "aware" of the newly created topic (even though terraform receives the "Creation complete" message). If the same code is executed a 2nd time, the tag binding completes without error. This doesn't occur on adding a tag to an existing topic, which supports the theory - though it's just a guess. |
@dandrivetime Thanks for sharing. This is our assumption as well, we have added a 5s sleep before creating the binding, the other thing we can do is to increase the sleep time, we will contact our SR as well to investigate. |
I also can confirm - after topic creation in same run - the adding tags/metadata fails also on version 1.51.0 |
@dandrivetime @marconak-itera could you confirm that your SR and Kafka clusters are in the same environment? We'll add more descriptive errors in the next version of TF provider but basically we can see:
when SR and Kafka clusters are in different environments (they should be in the same environment AFAIK). Could you also share a type of your Kafka cluster? Even if I set sleep time to 0 seconds, I can't really reproduce 404 🤔 even when I force a topic to be created right before a tag binding:
|
@linouk23 - yes the SR's are in the same environment. We have a test and prod env. They're both type: I'm not sure why this is so different for us, vs. your local testing! Even if I immediately ( as fast as I can up arrow and press the enter key) run the code again, it will work the 2nd time. This suggests there's not really anything wrong with the inputs/resource setup in our code, but rather something going on with coordination we're not able to see. EDIT: forgot to mention, the topic always goes 1st in our setup. The tag binding doesn't start until after TF gets the "creation complete" message, so the dependency is recognized. |
@linouk23 @zhenli00 - I forced in a timer for 1 minute after the topic is created. This seems to prove out there's a timing issue behind the scenes.
|
@dandrivetime we just released a new version of TF Provider (1.52.0) where we increased a timer to a minute! Let us know if that mitigates the issue. |
Hello, we updated the version, it still fails it from time to time errors:
after rerun
|
Hi @linouk23 can you look at this bug? We are using |
@marconak-itera @HSA72 @dandrivetime we've just merged a new fix internally (added timeout in a different method so hopefully it'll mitigate the issue) that we'll include in our next release of TF Provider that should happen early in January. |
@marconak-itera @HSA72 @dandrivetime Update: we
in our latest 1.57.0 version of TF Provider. Thanks for waiting and let us know if you run into any issues when using the latest 1.57.0 version of TF Provider. |
@marconak-itera @HSA72 @dandrivetime we've just released a new 1.58.0 version of TF Provider with increased timeout that should help to fix
Let us know if the issue persists, thanks! |
@linouk23 - I upgraded to the 1.58 version. Looks like it works (note I only tested once :) ) -
|
Nice! We realized we updated timeouts for |
Update: thanks for waiting, we released
Thanks! |
Hello,
we are using new
confluent_business_metadata_binding
andconfluent_tag_binding
and we are adding the tags and business metadata on topic and schema.The common case for us is create:
in one run. The topics/schema contains usually 3-4 tags, 1-2 business metadata.
But when are running the
terraform
in some case we are getting error:│ Error: Provider produced inconsistent result after apply │ │ When applying changes to │ module.....confluent_business_metadata_binding.business_metadata_topic_entity_type["Team"], │ provider "provider[\"registry.terraform.io/confluentinc/confluent\"]" │ produced an unexpected new value: Root resource was present, but now │ absent. │ │ This is a bug in the provider, which should be reported in the provider's │ own issue tracker.
Error: error creating Tag Binding 404 Not Found
Error creating Business Metadata Binding 404 Not Found
When we rerun terraform again, in some cases it will finish successfully, but in other case there is error regarding the tags or metadata that already exists.
error creating Business Metadata Binding ".../Team/....:topic/kafka_topic": Business-metadata attribute already exists in entity: ....
We have to manually removed tags/metadata to able run the terraform successful.
Configuration looks like something like this:
The version
The text was updated successfully, but these errors were encountered: