Workflow get request returns unexpected workflow after unlinking subject set from workflow

[lcjohnso]

[EDIT 2023-02-03 per @mcbouslog ] Originally opened in Panoptes-Front-End. Transferred to panoptes because we believe this might be a workflow caching issue. With a successful request to unlink a subject set from a workflow, the db reflects the link destroyed, but an immediate subsequent get request returns a workflow that still includes the subject set link (unexpected). The same workflow get request 5 minutes later does not include the subject set link (as expected).

Original comment as opened in PFE...

Expected behavior

If you unlink a subject set on the project builder's workflow editor page, the set should no longer be linked and the tick box remain unchecked.

Current behavior

If you unlink a subject set on the project builder's workflow editor page, then reload the page (within ~5 min), the tick box is reticked. However, if you try to untick the box again, a console error appears: Couldn't find SubjectSet with 'id'=73874 [WHERE "subject_sets_workflows"."workflow_id" = $1]

When simultaneously tracking subject_sets_workflows entries in the Panoptes DB, it appears that the untick action successfully unlinks the subject set, however the cached state of the page has not been updated so the tickbox is re-checked on reload. Interesting, the caching not only affects the project builder web interface, but also the state of the workflow links field from the Python client.

Also: it appears that if a second subject set is linked, the state is updated -- i.e., the unlinking of the first subject set is correctly reflected on the workflow editor page (and Client/API response) after reload.

Steps to replicate

1. Link a subject set via a workflow editor page
2. Unlink the subject set
3. Reload the page -- the unlinked subject set's tick box will be ticked again

Additional Comments

It is possible the issue is not one having to do with PFE, but rather Panoptes in how it is caching responses (as evidenced by similar behavior shown by Python Client) -- worth investigating that backend aspect early on.

[eatyourgreens]

Worth checking the browser network panel, when you reload the page, and check workflow.links in the request for the workflow. If the unlinked set is still listed, then this is a bug in the API. Are API responses sent with Cache-Control headers? If they are, then the browser might get the workflow from its cache, rather than hit the API.

[denslowm]

Hi, I've been experiencing this issue for some time so thought I'd comment with the hopes of helping to prioritize a fix.

For some reason, it seems to be getting worse for me and has led to some subject sets being linked / unlinked by mistake to live workflows. Also, this AM I had to really struggle to use the radio buttons and it took refreshing the page several times to get it right.

Screen.Recording.2023-01-30.at.11.46.09.AM.mov

[mcbouslog]

Noting the request to delete a linked subject set from a workflow appears to succeed (i.e. https://panoptes-staging.zooniverse.org/api/workflows/3275/links/subject_sets/4620?http_cache=true&admin=true => status 204), but then immediate subsequent get requests for the workflow (i.e. https://panoptes-staging.zooniverse.org/api/workflows/3275?http_cache=true&admin=true) include workflow.links.subject_sets with an array that includes the just unlinked subject set.

So I think initially (before any workflow inconsistency exists),

1. when a subject is unlinked the component state updates to appear as expected because the component is updating linked subject sets in component state, the delete is successful with the API (204)
2. a refresh or leaving then re-navigating to the workflow editor resets the component state per a get request for the workflow (which includes links.subject_sets with a now unlinked subject set) - <= here's the issue
3. if attempting to uncheck the subject set already unlinked, the request to delete the already deleted linkage returns a 404 (which is good/expected if trying to delete something already deleted)

Nothing new here, reiterating what Cliff notes. I'm not done investigating, but noting what I have for now...

[mcbouslog]

Tested workflow and subject set linking/unlinking with the panoptes-cli and can confirm the following:

1. panoptes workflow info 17744 - confirm workflow has no subject sets linked
2. panoptes workflow add-subject-sets 17744 83176
3. immediately after step 2 panoptes workflow info 17744 - confirm workflow now includes links.subject_sets: ['83176']
4. [wait a few minutes] panoptes workflow remove-subject-sets 17744 83176
5. immediately after step 4 panoptes workflow info 17744 - unexpectedly workflow still includes links.subject_sets: ['83176']
6. [wait 5 minutes] panoptes workflow info 17744 - workflow does not have any subject sets linked

So it appears there's a delay from panoptes in a subject set unlinking from a workflow.

[yuenmichelle1]

This could be due to the Rails upgrades that have been undergone.
See: https://www.bigbinary.com/blog/rails-5-allows-updating-without-updating-timestamps

Cached_serializer utilizes resource's updated_at for serializer_cache_key. See

panoptes/app/serializers/concerns/cached_serializer.rb

Line 18 in 4698c5d

"#{model.class}/#{model.id}/#{model.updated_at.to_i}/context-#{context_hash}"

We did see something similar in update_links where we had to touch the resource [Rails 4.2] (now in Rails 5 updated to set the updated_at time and saving the resource.
See:
https://github.com/zooniverse/panoptes/blob/master/lib/json_api_controller/updatable_resource.rb#L32-L35
We could do the same thing in destroy_links to fix .

panoptes/lib/json_api_controller/updatable_resource.rb

Line 43 in 4698c5d

def destroy_links

(
Tested locally by first disabling the cached_serializer flipper to find out it was the updated_at within cached_serializer. Then remembering seeing something like this when upgrading to Rails 5.
)

[yuenmichelle1]

Closed with #4108