feat: improve export #316
Merged
feat: improve export #316
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
prevent segfault with more than 25k results
rolljee
requested changes
Aug 30, 2023
| const { | ||
| query, | ||
| sort, | ||
| lang = "elasticsearch", |
There was a problem hiding this comment.
lang was set to koncorde by default before, we should keep the same otherwise it's a breaking change
There was a problem hiding this comment.
It's true, but all other requests use lang='elasticsearch' by default, so it's inconsistent.
But need to be confirmed because for measure export some filters are hard-coded in koncorde language.
There was a problem hiding this comment.
Doesn't it already default to elasticsearch ? See this
|
|
||
| return new HttpStream(stream); | ||
| } catch (error) { | ||
| request.response.configure({ |
There was a problem hiding this comment.
Why adding this instead of throwing a traditional error ?
There was a problem hiding this comment.
Because this endpoint is mostly called by browser and traditional error return a JSON that can be hard to understand for user.
Maybe the better solution could be to improve global error handler in the core
There was a problem hiding this comment.
Add a comment to explain this in the code :)
rolljee
approved these changes
Aug 30, 2023
sebtiz13
force-pushed
the
fix/digitaltwin-export
branch
from
485a82d
to
9ccba3f
Compare
github-actions bot
pushed a commit
that referenced
this pull request
Aug 31, 2023
# [2.4.0-beta.1](v2.3.2...v2.4.0-beta.1) (2023-08-31) ### Bug Fixes * copy-version script ([72b5e49](72b5e49)) ### Features * digitaltwin exports ([#315](#315)) ([aab65af](aab65af)) * improve export ([#316](#316)) ([e4e746e](e4e746e)) * **measures:** add lastMeasuredAt on DigitalTwin ([#314](#314)) ([05b8a53](05b8a53))
github-actions bot
pushed a commit
that referenced
this pull request
Aug 31, 2023
# [2.4.0-beta.1](v2.3.2...v2.4.0-beta.1) (2023-08-31) ### Bug Fixes * copy-version script ([72b5e49](72b5e49)) ### Features * digitaltwin exports ([#315](#315)) ([aab65af](aab65af)) * improve export ([#316](#316)) ([e4e746e](e4e746e)) * **measures:** add lastMeasuredAt on DigitalTwin ([#314](#314)) ([05b8a53](05b8a53))
sebtiz13
added a commit
that referenced
this pull request
Aug 31, 2023
* feat(export): add support elasticsearch lang * feat: improve handle error * feat(export): limit export to 10k documents prevent segfault with more than 25k results * fix(export): prevent error duplicate result
github-actions bot
pushed a commit
that referenced
this pull request
Aug 31, 2023
# [2.4.0-beta.1](v2.3.2...v2.4.0-beta.1) (2023-08-31) ### Bug Fixes * copy-version script ([72b5e49](72b5e49)) ### Features * digitaltwin exports ([#315](#315)) ([aab65af](aab65af)) * improve export ([#316](#316)) ([51db84a](51db84a)) * **measures:** add lastMeasuredAt on DigitalTwin ([#314](#314)) ([05b8a53](05b8a53))
github-actions bot
pushed a commit
that referenced
this pull request
Aug 31, 2023
# [2.4.0-beta.1](v2.3.2...v2.4.0-beta.1) (2023-08-31) ### Bug Fixes * copy-version script ([72b5e49](72b5e49)) ### Features * digitaltwin exports ([#315](#315)) ([aab65af](aab65af)) * improve export ([#316](#316)) ([51db84a](51db84a)) * **measures:** add lastMeasuredAt on DigitalTwin ([#314](#314)) ([05b8a53](05b8a53))
github-actions bot
pushed a commit
that referenced
this pull request
Aug 31, 2023
# [2.4.0-beta.1](v2.3.2...v2.4.0-beta.1) (2023-08-31) ### Bug Fixes * copy-version script ([72b5e49](72b5e49)) ### Features * digitaltwin exports ([#315](#315)) ([aab65af](aab65af)) * improve export ([#316](#316)) ([51db84a](51db84a)) * **measures:** add lastMeasuredAt on DigitalTwin ([#314](#314)) ([05b8a53](05b8a53))
github-actions bot
pushed a commit
that referenced
this pull request
Jan 11, 2024
# [2.4.0-temp-ci-debug.1](v2.3.2...v2.4.0-temp-ci-debug.1) (2024-01-11) ### Bug Fixes * **asset_migration:** fixes multiple assets migration and optimize ([fd546cf](fd546cf)) * **assetsGroups:** disable checkRights on impersonate ([#325](#325)) ([eaa3edc](eaa3edc)) * **assets:** migrate tenant only if user is admin ([c24f942](c24f942)) * **assetsMigrateTenant:** add type for frontend and fix admin only ([e249f13](e249f13)) * **ci:** correct github token in ci ([a5a6b69](a5a6b69)) * copy-version script ([72b5e49](72b5e49)) * correct KuzzleRequest for pipes ([ef932d1](ef932d1)) * **export:** correct scroll in getNamedMeasures ([05fe4fd](05fe4fd)) * improve get internal devices documents ([03ccb29](03ccb29)) * **migrateTenant:** clear groups of newly created assets ([79f6a5a](79f6a5a)) * **npm:** fix repository url ([514dc59](514dc59)) * update semantic-release ([380c8b3](380c8b3)) ### Features * **asset:** add upsert method ([#323](#323)) ([5f87787](5f87787)) * **assets:** assets migrate tenant ([5f05101](5f05101)) * digitaltwin exports ([#315](#315)) ([aab65af](aab65af)) * **digitalTwin:** allow to modify mapping from application ([#322](#322)) ([89d375a](89d375a)) * **digitalTwin:** implement generic document pipes triggers ([#321](#321)) ([60a21fc](60a21fc)) * improve export ([#316](#316)) ([51db84a](51db84a)) * **measures:** add lastMeasuredAt on DigitalTwin ([#314](#314)) ([05b8a53](05b8a53))
github-actions bot
pushed a commit
that referenced
this pull request
Jan 11, 2024
# [2.4.0-temp-ci-debug.1](v2.3.2...v2.4.0-temp-ci-debug.1) (2024-01-11) ### Bug Fixes * **asset_migration:** fixes multiple assets migration and optimize ([fd546cf](fd546cf)) * **assetsGroups:** disable checkRights on impersonate ([#325](#325)) ([eaa3edc](eaa3edc)) * **assets:** migrate tenant only if user is admin ([c24f942](c24f942)) * **assetsMigrateTenant:** add type for frontend and fix admin only ([e249f13](e249f13)) * **ci:** correct github token in ci ([a5a6b69](a5a6b69)) * copy-version script ([72b5e49](72b5e49)) * correct KuzzleRequest for pipes ([ef932d1](ef932d1)) * **export:** correct scroll in getNamedMeasures ([05fe4fd](05fe4fd)) * improve get internal devices documents ([03ccb29](03ccb29)) * **migrateTenant:** clear groups of newly created assets ([79f6a5a](79f6a5a)) * **npm:** fix repository url ([514dc59](514dc59)) * update semantic-release ([380c8b3](380c8b3)) ### Features * **asset:** add upsert method ([#323](#323)) ([5f87787](5f87787)) * **assets:** assets migrate tenant ([5f05101](5f05101)) * digitaltwin exports ([#315](#315)) ([aab65af](aab65af)) * **digitalTwin:** allow to modify mapping from application ([#322](#322)) ([89d375a](89d375a)) * **digitalTwin:** implement generic document pipes triggers ([#321](#321)) ([60a21fc](60a21fc)) * improve export ([#316](#316)) ([51db84a](51db84a)) * **measures:** add lastMeasuredAt on DigitalTwin ([#314](#314)) ([05b8a53](05b8a53))
github-actions bot
pushed a commit
that referenced
this pull request
Jan 11, 2024
# [2.4.0-temp-ci-debug.1](v2.3.2...v2.4.0-temp-ci-debug.1) (2024-01-11) ### Bug Fixes * **asset_migration:** fixes multiple assets migration and optimize ([fd546cf](fd546cf)) * **assetsGroups:** disable checkRights on impersonate ([#325](#325)) ([eaa3edc](eaa3edc)) * **assets:** migrate tenant only if user is admin ([c24f942](c24f942)) * **assetsMigrateTenant:** add type for frontend and fix admin only ([e249f13](e249f13)) * **ci:** correct github token in ci ([a5a6b69](a5a6b69)) * copy-version script ([72b5e49](72b5e49)) * correct KuzzleRequest for pipes ([ef932d1](ef932d1)) * **export:** correct scroll in getNamedMeasures ([05fe4fd](05fe4fd)) * improve get internal devices documents ([03ccb29](03ccb29)) * **migrateTenant:** clear groups of newly created assets ([79f6a5a](79f6a5a)) * **npm:** fix repository url ([514dc59](514dc59)) * update semantic-release ([380c8b3](380c8b3)) ### Features * **asset:** add upsert method ([#323](#323)) ([5f87787](5f87787)) * **assets:** assets migrate tenant ([5f05101](5f05101)) * digitaltwin exports ([#315](#315)) ([aab65af](aab65af)) * **digitalTwin:** allow to modify mapping from application ([#322](#322)) ([89d375a](89d375a)) * **digitalTwin:** implement generic document pipes triggers ([#321](#321)) ([60a21fc](60a21fc)) * improve export ([#316](#316)) ([51db84a](51db84a)) * **measures:** add lastMeasuredAt on DigitalTwin ([#314](#314)) ([05b8a53](05b8a53))
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do ?