Dynamic tag not updated after scheduled_auto_updates request

[HarukaIzumisawa]

Summary:
This issue reports that dynamic tags are not being updated after the scheduled_auto_updates function is called, even though the next_run parameter has passed. The potential cause might be related to the scheduling_status parameter not transitioning to the expected READY state.

Details:
We expect that creating a Dynamic Tag Table configuration with the Refresh Mode set to AUTO and then calling scheduled_auto_updates would update the tags based on the configuration. However, no tag updates were implemented.

The reason for this is that the scheduling_status parameter does not transition to the expected READY state.

The reason for this is as follows.

Looking at the source code, it says that the scheduling_status parameter must be READY as a condition for updating.

datacatalog-tag-engine/TagEngineStoreHandler.py

Lines 1340 to 1349 in b64520a

	def read_ready_configs(self):
	ready_configs = []
	for coll_name in self.get_config_collections():
	config_ref = self.db.collection(coll_name)
	config_ref = config_ref.where("refresh_mode", "==", "AUTO")
	config_ref = config_ref.where("scheduling_status", "==", "READY")
	config_ref = config_ref.where("config_status", "==", "ACTIVE")
	config_ref = config_ref.where("next_run", "<=", datetime.utcnow())

However, looking at the source code, if I create a Dynamic Tag Table configuration with Refresh Mode set to AUTO, the scheduling_status is set to ACTIVE. In fact, the tags were not updated when I ran the scheduled_auto_updates function.

datacatalog-tag-engine/TagEngineStoreHandler.py

Lines 495 to 499 in b64520a

	'tag_history': tag_history,
	'scheduling_status': 'ACTIVE',
	'next_run': next_run,
	'version': 1,
	'service_account': service_account

If there are additional steps required for the update to happen with Refresh Mode set to AUTO, we would appreciate clarification.

Call to action:

• Please advise if there are additional steps required for the update to occur with Refresh Mode set to AUTO.
• If this is a bug, please advise on the next steps or potential workarounds.

[shirleycohen]

Thanks for letting me know, this sounds like a bug. Out of curiosity, how were you calling /scheduled_auto_updates? Was it from Cloud Scheduler?

[HarukaIzumisawa]

With the environment variables set, I used the following command to call /scheduled_auto_updates. I didn't use Cloud Scheduler, because I considered that if a direct call with the curl command doesn't work, it won't work with cloud scheduler.

curl -i -X POST $TAG_ENGINE_URL/scheduled_auto_updates \
  -H "Authorization: Bearer $IAM_TOKEN" \
  -d @examples/configs/dynamic_table/dynamic_table_auto.json

[shirleycohen]

@HarukaIzumisawa apologies for the long delay. This issue has been fixed in v2.2.9

Please note that you need to call the scheduled_auto_updates method as follows:

export IAM_TOKEN=$(gcloud auth print-identity-token)

curl -X POST $TAG_ENGINE_URL/scheduled_auto_updates \
  -H "Authorization: Bearer $IAM_TOKEN"

The expected output looks similar to:

curl -X POST $TAG_ENGINE_URL/scheduled_auto_updates  \
  -H "Authorization: Bearer $IAM_TOKEN"
{"job_ids":"[\"20380bf6340d11ef8ea942004e494300\"]","success":true}

This means that Tag Engine found one config that needed to be run and created a job for it.

When no ready configs are found, the expected output looks like this instead:

curl -X POST $TAG_ENGINE_URL/scheduled_auto_updates   \
  -H "Authorization: Bearer $IAM_TOKEN"
{"job_ids":"[]","success":true}