Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up migrating backend storage by multithreading #8074

Open
philipsu522 opened this issue Dec 26, 2019 · 0 comments
Open

Speed up migrating backend storage by multithreading #8074

philipsu522 opened this issue Dec 26, 2019 · 0 comments

Comments

@philipsu522
Copy link

Is your feature request related to a problem? Please describe.
While using the operator migrate command from Consul to Azure, I'm noticing that I have a lot of certificates under the path 2019-12-23T17:57:04.553Z [INFO] copied key: path=logical/294bba60-c16b-2f47-1682-b7bf37d84d1d/certs/03-6f-20-cf-ac-42-7a-bd-15-c9-b6-08-be-a1-a3-82-aa-d4-9a-f3. This results in the migration possibly taking days to finish.

Describe the solution you'd like
I'd like to speed up the operation by parallelizing the copying of data. Goroutines can be added to from.Get() and to.Put() to increase the number of concurrent migrations at a time.

Describe alternatives you've considered
N/A

Explain any additional use-cases
If there are any use-cases that would help us understand the use/need/value please share them as they can help us decide on acceptance and prioritization.
This operation would speed up backend migration. Since migration should be done offline, we'd like this operation to finish as quickly as possible. Additionally, we're using this specifically to copy data and test functionality between different versions of Vault before moving our production system onto 1.3.

Additional context
N/A

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants