Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]Create MV fails setting up correctly #140

Open
YANG-DB opened this issue Nov 7, 2023 · 1 comment
Open

[BUG]Create MV fails setting up correctly #140

YANG-DB opened this issue Nov 7, 2023 · 1 comment
Labels
bug Something isn't working question Further information is requested

Comments

@YANG-DB
Copy link
Member

YANG-DB commented Nov 7, 2023

What is the bug?
Creation of a MV fails:

CREATE MATERIALIZED VIEW mys3.default.http_countip_distinct_views
AS
  SELECT
    window.start AS `start.time`,
    COUNT(distinct clientip) AS count_hourly_failed_distinct_ip
  FROM mys3.default.http_logs_plain
  WHERE status != 200
  GROUP BY TUMBLE(`@timestamp`, '60 Minutes')
WITH (
  auto_refresh = true,
  refresh_interval = '10 Minutes',
  checkpoint_location = 's3://flint-data-dp-eu-west-1-beta/data/http_log/checkpoint_http_count_view',
  watermark_delay = '10 Minutes'
);

How can one reproduce the bug?
Steps to reproduce the behavior:

Running the next MV creation

CREATE MATERIALIZED VIEW mys3.default.http_countip_distinct_views
AS
  SELECT
    window.start AS `start.time`,
    COUNT(distinct clientip) AS count_hourly_failed_distinct_ip
  FROM mys3.default.http_logs_plain
  WHERE status != 200
  GROUP BY TUMBLE(`@timestamp`, '60 Minutes')
WITH (
  auto_refresh = true,
  refresh_interval = '10 Minutes',
  checkpoint_location = 's3://flint-data-dp-eu-west-1-beta/data/http_log/checkpoint_http_count_view',
  watermark_delay = '10 Minutes'
);

Result in the next error:

{
    "data": {
        "ok": true,
        "resp": {
            "status": "FAILED",
            "error": "Fail to write result, cause: Failed to create Flint index"
        }
    }
}

What is the expected behavior?
Successful create the index and the ETL proccess

Do you have any screenshots?
Screenshot 2023-11-07 at 2 28 10 PM

Do you have any additional context?
Add any other context about the problem.

@YANG-DB YANG-DB added bug Something isn't working untriaged labels Nov 7, 2023
@dai-chen
Copy link
Collaborator

dai-chen commented Nov 8, 2023

@YANG-DB it would be helpful if you can share more info, such as any error stack in Spark log. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants