Loki: Internal Server Error. 500. QueryPages error: table=loki_: ResourceNotFoundException: Requested resource not found

[k-szortyka]

Describe the bug
I'm trying to store chunks in AWS S3, index in AWS DynamoDB and I get error:
Loki: Internal Server Error. 500. QueryPages error: table=loki_: ResourceNotFoundException: Requested resource not found

To Reproduce
Steps to reproduce the behavior:

1. Started Loki (SHA or version)
2. Started Promtail (SHA or version) to tail '...'
3. Query: {} term

Expected behavior
Create table in DynamoDB.

Environment:

• loki 2.4.0
• permissions: AmazonS3FullAccess, AmazonDynamoDBFullAccess

Screenshots, Promtail config, or terminal output
loki-config.yaml

auth_enabled: false

server:
  http_listen_port: 3100

common:
  path_prefix: /loki
  storage:
    filesystem:
      chunks_directory: /loki/chunks
      rules_directory: /loki/rules
  replication_factor: 1
  ring:
    instance_addr: 127.0.0.1
    kvstore:
      store: inmemory

schema_config:
  configs:
  - from: 2020-05-15
    store: aws
    object_store: s3
    schema: v11
    index:
      prefix: loki_
storage_config:
  aws:
   s3: s3://AKIATL2SxxxVH2AEK:lw4iIFxxxxxZCKmo9@eu-west-1/grafana-loki
   s3forcepathstyle: true
   dynamodb:
      dynamodb_url: dynamodb://AKIATL2SDHD6CxxxEK:lw4iIFDbE2NOHdWxxxxxxxxxAaU55k2AxZCKmo9@eu-west-1

ruler:
  alertmanager_url: http://localhost:9093

docker-compose

version: "3"

networks:
  loki:

services:
  loki:
    image: grafana/loki:2.4.0
    volumes:
      - /opt/loki/local-config.yaml:/etc/loki/local-config.yaml
    ports:
      - "3100:3100"
    command: -config.file=/etc/loki/local-config.yaml
    networks:
      - loki

  promtail:
    image: grafana/promtail:2.4.1
    volumes:
      - /opt/promtail/promtail-config.yaml:/etc/promtail/promtail-config.yaml
      - /var/log:/var/log
    command: -config.file=/etc/promtail/promtail-config.yaml
    networks:
      - loki

[yongzhang]

same issue, looks like loki didn't create dynamodb tables automatically, but didn't see error logs

[sbkg0002]

How did you resolve it @yongzhang ? I just hit the same problem in AWS.
I use the following IAM priv, but it seems to be hit-or-miss if it works - which is weird.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Action": "dynamodb:*",
            "Resource": "*"
        },
        {
            "Sid": "",
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::lokibucket-loki/*"
        },
        {
            "Sid": "",
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::lokibucket-loki"
        }
    ]
}

[yongzhang]

How did you resolve it @yongzhang ? I just hit the same problem in AWS. I use the following IAM priv, but it seems to be hit-or-miss if it works - which is weird.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Action": "dynamodb:*",
            "Resource": "*"
        },
        {
            "Sid": "",
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::lokibucket-loki/*"
        },
        {
            "Sid": "",
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::lokibucket-loki"
        }
    ]
}

If you're using v2.4.x, you may hit #5070.

[stale]

Hi! This issue has been automatically marked as stale because it has not had any
activity in the past 30 days.

We use a stalebot among other tools to help manage the state of issues in this project.
A stalebot can be very useful in closing issues in a number of cases; the most common
is closing issues or PRs where the original reporter has not responded.

Stalebots are also emotionless and cruel and can close issues which are still very relevant.

If this issue is important to you, please add a comment to keep it open. More importantly, please add a thumbs-up to the original issue entry.

We regularly sort for closed issues which have a stale label sorted by thumbs up.

We may also:

• Mark issues as revivable if we think it's a valid issue but isn't something we are likely
  to prioritize in the future (the issue will still remain closed).
• Add a keepalive label to silence the stalebot if the issue is very common/popular/important.

We are doing our best to respond, organize, and prioritize all issues but it can be a challenging task,
our sincere apologies if you find yourself at the mercy of the stalebot.