[5.x] [esArchiver] combine elasticDump and ScenarioManager (#10359) #11170
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* As a part of bringing functional testing to plugins, esArchiver gives these plugins a way to capture and reload es indexes without needing to write a bunch of custom code. It works similarly to the elasticDump and ScenarioManager tools that it replaces. Differences: - Streaming implementation allows for much larger archives - CLI for creating and using archives - Configurable archive location - Stores the data in gzipped files (better for source control, searching, large archives) - Automatically identifies and upgrades Kibana config documents Methods: - `#load(name)`: import an archive - `#loadIfNeeded(name)`: import an archive, but skip the documents what belong to any existing index - `#unload(name)`: delete the indexes stored in an archive CLI operations: - `./bin/es_archiver save <name> [index patterns...]`: save the mapping and documents in one or more indexes that match the wild-card patterns into an the `<name>` archive - `./bin/es_archiver load <name>`: load the mapping and documents from the `<name>` archive * [functional_tests/common/nagivate] check for statusPage * [es_archiver] move bins into new scripts dir * [functional_tests/apps/context] use esArchiver * [esArchiver] general improvements after showing to a few folks - remove auto-upgrading config doc logic (until we have better access to kibana version info) - export unload command - remove preemptive checks in favor of reacting to errors - use type "doc" vs "hit" for doc records (consistency) - wrote a bunch of pending tests to think though and plan * [esArchiver] make log a stream that writes to itself * [esArchiver] fill in stats and archive format tests * [esArchiver] splitup action logic * [esArchiver/cli] fix cli --help output and comment * [esArchiver] remove type-based param coercion * [esArchiver/log] use strings for log levels * [esArchvier] remove unused var * [esArchiver/indexDocRecordsStream] add tests * [esArchive] fill in remaining tests * [esArchiver] fix dem tests * [eslint] remove unused vars * [esArchiver/loadIfNeeded] fix call to load() * [esArchiver] remove loadDumpData helpers
|
Superseded by #11171 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport of #10359
Differences:
Methods:
#load(name): import an archive#loadIfNeeded(name): import an archive, but skip the documents what belong to any existing index#unload(name): delete the indexes stored in an archiveCLI operations:
./bin/es_archiver save <name> [index patterns...]: save the mapping and documents in one or more indexes that match the wild-card patterns into an the<name>archive./bin/es_archiver load <name>: load the mapping and documents from the<name>archive[functional_tests/common/nagivate] check for statusPage
[es_archiver] move bins into new scripts dir
[functional_tests/apps/context] use esArchiver
[esArchiver] general improvements after showing to a few folks
[esArchiver] make log a stream that writes to itself
[esArchiver] fill in stats and archive format tests
[esArchiver] splitup action logic
[esArchiver/cli] fix cli --help output and comment
[esArchiver] remove type-based param coercion
[esArchiver/log] use strings for log levels
[esArchvier] remove unused var
[esArchiver/indexDocRecordsStream] add tests
[esArchive] fill in remaining tests
[esArchiver] fix dem tests
[eslint] remove unused vars
[esArchiver/loadIfNeeded] fix call to load()
[esArchiver] remove loadDumpData helpers