Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix tag-filtered GET issues for Cassandra-backed nodes #5105

Merged
merged 2 commits into from
Oct 14, 2019

Conversation

rainest
Copy link
Contributor

@rainest rainest commented Oct 7, 2019

Summary

Fixes issues with tag-filtered GETs on Cassandra-backed nodes.

Full changelog

  • Modify table index within Cassandra tag query handling to allow passing a subset of all entities found, rather than all entities found, to the next phase.
  • When dereferencing entity IDs found during a Cassandra tag query, skip entities that no longer exist.

Issues resolved

This addresses two fatal errors that appear during admin API GET requests filtered by tags on Cassandra-backed nodes:

  • Kong Enterprise filters the list of tagged entities retrieved to remove entities that are not in the correct workspace (or whose workspace cannot be determined, because the entity has been deleted). Without this change, table indices for filtered entities will have no value, causing the following error when they are later passed to dereference_rows:
2019/09/24 00:52:26 [error] 32#0: *10316 lua coroutine: runtime error: ...ocal/share/lua/5.1/kong/db/strategies/cassandra/init.lua:315: bad argument #1 to 'uuid()' (got nil)
  • Entities are retrieved in a non-atomic two-step process, and entities can be deleted before the second step. This causes dereference_rows to return null values for some entity IDs, later breaking the admin API's attempts to serialize them to JSON:
2019/09/30 20:25:26 [error] 7055#0: *20111 lua coroutine: runtime error: /usr/local/share/lua/5.1/kong/db/schema/init.lua:1463: attempt to index local 'data' (a nil value
)
stack traceback:
coroutine 0:
        /usr/local/share/lua/5.1/kong/db/schema/init.lua: in function 'process_auto_fields'
        /usr/local/share/lua/5.1/kong/db/dao/init.lua:1490: in function 'row_to_entity'
        /usr/local/share/lua/5.1/kong/db/dao/init.lua:1471: in function 'rows_to_entities'
        /usr/local/share/lua/5.1/kong/db/dao/init.lua:1064: in function 'page_collection'
        /usr/local/share/lua/5.1/kong/api/endpoints.lua:303: in function 'fn'
        /usr/local/share/lua/5.1/kong/api/init.lua:50: in function </usr/local/share/lua/5.1/kong/api/init.lua:33>

Travis Raines added 2 commits October 7, 2019 11:40
For tag-filtered queries in Cassandra, use the current entity count as
the entity table index rather than the row iteration counter. If some
entities are excluded from the table, this ensures that no table indices
have a nil value. Kong Enterprise filters these results to exclude
entities in other workspaces.
Exclude entities that cannot be dereferenced from the result of
dereference_rows. Tag queries in C* are a two-step process to retrieve
IDs and then retrieve complete objects, and it's possible to delete
entities between these steps. Returning these entities (or rather, null
entries where they would be) causes errors down the line when the admin
API attempts to serialize them.

Because this excludes some entities that were previously in a page, this
can return fewer entities than requested in a page, though offset/next
page information will still be included.
Copy link
Contributor

@fffonion fffonion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

@hishamhm hishamhm merged commit 53c8c0e into master Oct 14, 2019
@hishamhm hishamhm deleted the fix/cassandra-tag-queries branch October 14, 2019 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants