Skip to content

perf(elastic): filter system fields in es instead of in whelk#1747

Merged
olovy merged 3 commits intodevelopfrom
perf/source-excludes-2
Apr 16, 2026
Merged

perf(elastic): filter system fields in es instead of in whelk#1747
olovy merged 3 commits intodevelopfrom
perf/source-excludes-2

Conversation

@olovy
Copy link
Copy Markdown
Contributor

@olovy olovy commented Apr 15, 2026

Less data that is just transferred and discarded for each query.

catches everything:

$ curl -s "http://localhost:8180/find?_q=" | gron | grep items | cut -d'=' -f1 | grep _ | grep -v _categoryByCollection
$

fixes ..._4_digits_... that slipped through before:

curl -s "https://libris-qa.kb.se/find?_q=" | gron | grep items | cut -d'=' -f1 | grep _ | grep -v _categoryByCollection
json.items[0]["@reverse"].instanceOf[0].publication[0].year_4_digits_keyword
json.items[0]["@reverse"].instanceOf[0].publication[0].year_4_digits_short
json.items[0]["@reverse"].instanceOf[0].publication[0]["librissearch:year_4_digits_keyword"]
json.items[0]["@reverse"].instanceOf[0].publication[0]["librissearch:year_4_digits_short"]
...

Couldn't see any slowdown in response times from elastic.

Copy link
Copy Markdown
Contributor

@kwahlin kwahlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@olovy olovy force-pushed the perf/source-excludes-2 branch from ca35120 to 10e450c Compare April 16, 2026 06:37
@olovy
Copy link
Copy Markdown
Contributor Author

olovy commented Apr 16, 2026

@olovy olovy requested a review from kwahlin April 16, 2026 06:41
@olovy olovy merged commit 592669f into develop Apr 16, 2026
1 check passed
@olovy olovy deleted the perf/source-excludes-2 branch April 16, 2026 08:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants