fix(ui): Fix download to CSV flow using Scroll across entities api #7629
Conversation
| @@ -94,3 +94,5 @@ export enum UnionType { | |||
| AND, | |||
| OR, | |||
| } | |||
|
|
|||
| export const SCROLL_KEEP_ALIVE_TIME = '10m'; // TODO: Determine if this is sufficient. | |||
There was a problem hiding this comment.
Can we make this an environment variable instead?
There was a problem hiding this comment.
Environment variable? That means extending a config endpoint right?
There was a problem hiding this comment.
Kinda blows up scope to adding this to our appConfig GraphQL API. Let me confirm with Ryan on platform team whether this is something we'd need to allow config on
|
Adding Cypress Tests for large downloads is kinda a scary proposition because it can take a long time to get the requisite amount of data into the instance. Took me ~30 minutes to get ~7k entities into my own local instance. I'd hesitate to do that. Maybe a better solve is to have a download as csv cypress test generally? |
|
Just chatted with Ryan - he mentioned that we can omit the parameter altogether for now! Going to remove it... |
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
Summary
In this PR we move to using scrollAcrossEntities api which allows for deep pagination against elasticsearch. This fixes download as csv which did not previously work for large data sets (greater than a few thousand)
Also, making scroll correctly apply the selected View.
Validation
Tested against 10k entities locally.
Status
Ready for review
Checklist