Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

migrate tool boltdb-shipper support #4425

Closed
robertlestak opened this issue Oct 7, 2021 · 1 comment
Closed

migrate tool boltdb-shipper support #4425

robertlestak opened this issue Oct 7, 2021 · 1 comment
Labels
stale A stale issue or PR that will automatically be closed.

Comments

@robertlestak
Copy link

robertlestak commented Oct 7, 2021

Describe the bug
When using the migrate tool, if schema_config.configs[*].store is boltdb-shipper, the tool fails with the error:

Failed to create destination store: error creating index client: Unrecognized storage client boltdb-shipper, choose one of: aws, cassandra, inmemory, gcp, bigtable, bigtable-hashed

This appears to come from an attempted NewIndexClient within the chunk_storage.NewStore, where boltdb-shipper is not in the Supported storage clients.

To Reproduce
Steps to reproduce the behavior:

  1. use the migrate tool with the destination (and presumably the source as well - but untested on my side) storage type set to boltdb-shipper per latest recommendation
  2. After a few seconds, the tool will fail with the error stating the storage client is unrecognized.

Expected behavior
The migrate tool should be able to support migrations to the recommended boltdb-shipper store.

Environment:

  • Infrastructure: In production, k8s. just testing the migrate tool, in the container built from make migrate-image.
  • Deployment tool: make migrate-image && docker run --entrypoint sh ....

Screenshots, Promtail config, or terminal output

➜ docker run \
        --rm -it \
        --env-file .env \
        --entrypoint sh \
        -v $PWD/loki-convert:/data \
        docker.io/grafana/loki-migrate:main-69d81df-WIP
/ # migrate \
> -source.config.file=/data/source-config.yaml \
> -dest.config.file=/data/dest-dev-config.yaml \
> -source.tenant=awsa1-hybrid-eks-dev \
> -dest.tenant=awsa1-hybrid-eks-dev \
> -from=2020-08-01T00:00:00-00:00 \
> -to=2020-10-01T00:00:00-00:00
2021-10-07 03:28:49.246308 I | proto: duplicate proto type registered: purgeplan.DeletePlan
2021-10-07 03:28:49.246396 I | proto: duplicate proto type registered: purgeplan.ChunksGroup
2021-10-07 03:28:49.246420 I | proto: duplicate proto type registered: purgeplan.ChunkDetails
2021-10-07 03:28:49.246443 I | proto: duplicate proto type registered: purgeplan.Interval
2021-10-07 03:28:49.278028 I | proto: duplicate proto type registered: grpc.PutChunksRequest
2021-10-07 03:28:49.278084 I | proto: duplicate proto type registered: grpc.GetChunksRequest
2021-10-07 03:28:49.278109 I | proto: duplicate proto type registered: grpc.GetChunksResponse
2021-10-07 03:28:49.278151 I | proto: duplicate proto type registered: grpc.Chunk
2021-10-07 03:28:49.278191 I | proto: duplicate proto type registered: grpc.ChunkID
2021-10-07 03:28:49.278228 I | proto: duplicate proto type registered: grpc.DeleteTableRequest
2021-10-07 03:28:49.278269 I | proto: duplicate proto type registered: grpc.DescribeTableRequest
2021-10-07 03:28:49.278306 I | proto: duplicate proto type registered: grpc.WriteBatch
2021-10-07 03:28:49.278329 I | proto: duplicate proto type registered: grpc.WriteIndexRequest
2021-10-07 03:28:49.278353 I | proto: duplicate proto type registered: grpc.DeleteIndexRequest
2021-10-07 03:28:49.278387 I | proto: duplicate proto type registered: grpc.QueryIndexResponse
2021-10-07 03:28:49.278420 I | proto: duplicate proto type registered: grpc.Row
2021-10-07 03:28:49.278449 I | proto: duplicate proto type registered: grpc.IndexEntry
2021-10-07 03:28:49.278509 I | proto: duplicate proto type registered: grpc.QueryIndexRequest
2021-10-07 03:28:49.278534 I | proto: duplicate proto type registered: grpc.UpdateTableRequest
2021-10-07 03:28:49.278559 I | proto: duplicate proto type registered: grpc.DescribeTableResponse
2021-10-07 03:28:49.278599 I | proto: duplicate proto type registered: grpc.CreateTableRequest
2021-10-07 03:28:49.278627 I | proto: duplicate proto type registered: grpc.TableDesc
2021-10-07 03:28:49.278691 I | proto: duplicate proto type registered: grpc.ListTablesResponse
2021-10-07 03:28:49.278723 I | proto: duplicate proto type registered: grpc.Labels
2021-10-07 03:28:49.279577 I | proto: duplicate proto type registered: storage.Entry
2021-10-07 03:28:49.279609 I | proto: duplicate proto type registered: storage.ReadBatch
2021-10-07 03:29:19.296582 I | Failed to create destination store: error creating index client: Unrecognized storage client boltdb-shipper, choose one of: aws, cassandra, inmemory, gcp, bigtable, bigtable-hashed
/ #
@stale
Copy link

stale bot commented Mar 3, 2022

Hi! This issue has been automatically marked as stale because it has not had any
activity in the past 30 days.

We use a stalebot among other tools to help manage the state of issues in this project.
A stalebot can be very useful in closing issues in a number of cases; the most common
is closing issues or PRs where the original reporter has not responded.

Stalebots are also emotionless and cruel and can close issues which are still very relevant.

If this issue is important to you, please add a comment to keep it open. More importantly, please add a thumbs-up to the original issue entry.

We regularly sort for closed issues which have a stale label sorted by thumbs up.

We may also:

  • Mark issues as revivable if we think it's a valid issue but isn't something we are likely
    to prioritize in the future (the issue will still remain closed).
  • Add a keepalive label to silence the stalebot if the issue is very common/popular/important.

We are doing our best to respond, organize, and prioritize all issues but it can be a challenging task,
our sincere apologies if you find yourself at the mercy of the stalebot.

@stale stale bot added the stale A stale issue or PR that will automatically be closed. label Mar 3, 2022
@stale stale bot closed this as completed Apr 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale A stale issue or PR that will automatically be closed.
Projects
None yet
Development

No branches or pull requests

1 participant