Skip to content

Don't fetch unneeded ES fields and don't serialize unneeded nested items.#54

Closed
nonword wants to merge 2 commits intomasterfrom
pb/compression-and-item-limits
Closed

Don't fetch unneeded ES fields and don't serialize unneeded nested items.#54
nonword wants to merge 2 commits intomasterfrom
pb/compression-and-item-limits

Conversation

@nonword
Copy link
Copy Markdown
Member

@nonword nonword commented Jul 11, 2017

This PR amends three things to improve response time:

  1. Adds compression library to automatically gzip all responses.
  2. Adds _source.excludes param to search and fetchById resources calls to exclude unnecessary source fields in ES response
  3. Truncates bib items to 10 in a search context. Seems front-end is only displaying first now anyway. (Will need to build way to paginate through nested items in future after indexing them as nested datatype.)

@nonword
Copy link
Copy Markdown
Member Author

nonword commented Jul 21, 2017

I think this PR may mitigate some of the issues with many-item bibs slowing down some searches. For your consideration @nodanaonlyzuul

@nodanaonlyzuul
Copy link
Copy Markdown
Contributor

nodanaonlyzuul commented Jul 21, 2017 via email

@kfriedman
Copy link
Copy Markdown

Great to add gzip. I'm a little concerned that the API Gateway might not support it, but I'm pretty sure it does now.

Also, it might be better to let Nginx handle the gzipping as I think that's best practice. We'd have to update the Beanstalk configuration. Maybe something like: https://gist.github.com/omidfi/aaae08617729d26fce52024cafb59e12

@nodanaonlyzuul
Copy link
Copy Markdown
Contributor

I agree with Kevin. I'm going to open an issue about letting nginx handle gzip.
I'm going to keep this PR open for the other functionality and maybe we can remove gzip from this feature branch.

@nodanaonlyzuul nodanaonlyzuul mentioned this pull request Jul 24, 2017
@nonword
Copy link
Copy Markdown
Member Author

nonword commented Jul 31, 2017

Removed app level gzip compression

@nonword nonword changed the title Add gzip compresssion, items truncation pre serialization Don't fetch unneeded ES fields and don't serialize unneeded nested items. Jul 31, 2017
@nonword
Copy link
Copy Markdown
Member Author

nonword commented Mar 23, 2018

The useful parts of this have been folded into #99

@nonword nonword closed this Mar 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants