Parse error when running on latest version #31

Aka-shi · 2022-06-09T13:30:51Z

Don't understand the error I am facing

Action.yml

name: Lint Check

on:
  pull_request:
    branches:
      - master
env:
  DBT_PROFILES_DIR: ${{ github.workspace }}/.dbt/
  TRINO_DEV_URL: trino-dev-coordinator-int.de.razorpay.com
  DBT_USER: github-actions-dbt
jobs:
  Lint_Check:
    name: Lint_Check
    runs-on: [self-hosted]

    steps:
      - name: Check out
        uses: actions/checkout@v2
      - uses: yu-iskw/action-sqlfluff@v3
        name: sql-lint-check
        with:
          github_token: ${{ secrets.github_token }}
          reporter: github-pr-review
          config: "${{ github.workspace }}/.sqlfluff"
          paths: "${{ github.workspace }}/models"
          extra_requirements_txt: "${{ github.workspace }}/requirements.txt"

sqlfluff config

[sqlfluff]
templater = dbt
dialect = ansi
sql_file_exts = .sql
exclude_rules = L027

[sqlfluff:rules]
max_line_length = 150

[tool.sqlfluff.templater.jinja]
apply_dbt_builtins = True

[sqlfluff:templater:dbt]
project_dir = .
profiles_dir = ./.dbt
profile = trino
target = trino-dev

ERROR screenshots

The text was updated successfully, but these errors were encountered:

Aka-shi · 2022-06-09T14:24:41Z

@yu-iskw

yu-iskw · 2022-06-10T00:01:23Z

@Aka-shi Thank you for the feedback. I haven't seen the error. Can you share the dbt model which have the violations so that I reproduce it, if you don't mind?

yu-iskw · 2022-06-10T01:22:26Z

I am guessing the issue is caused by any breaking change of github action. I will look into it.

yu-iskw · 2022-06-23T08:29:07Z

@Aka-shi After all, I don't understand the root cause.

Can you tell me how to reproduce it in detail? I am not familiar with trio with dbt.
Can you share the logs?

yu-iskw · 2022-07-20T21:33:04Z

If you still have the issue, please feel free to re-open.

kieronellis · 2022-07-21T14:22:14Z

I'm getting the same issue but only with "lint" mode.

test_sqlfluff.sql

select blaa1, blaa2
from blaa

test.yaml

name: sqlfluff with reviewdog
on:
  pull_request:
jobs:
  test-check:
    name: runner / sqlfluff (github-check)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Decode service account credentials
        env:
          # The base64 encoded credentials.
          GOOGLE_APPLICATION_CREDENTIALS_BASE64: "${{ secrets.GOOGLE_APPLICATION_CREDENTIALS_BASE64 }}"
        run: |
          echo "...Decoding GOOGLE_APPLICATION_CREDENTIALS_BASE64..."
          python decode_base64.py || exit 1
          echo "...and validating json output..."
          python -m json.tool gcloud.json > /dev/null || exit 1

      - uses: yu-iskw/action-sqlfluff@v3.3.3
        env:
          DBT_PROFILES_DIR: ${{ github.workspace }}/.github/sqlfluff/.dbt/
        id: lint_sql
        with:
          github_token: ${{ secrets.TOKEN_GITHUB }}
          reporter: github-pr-review
          sqlfluff_version: "1.2.1"
          sqlfluff_command: "lint" # "fix" or "lint"
          config: "${{ github.workspace }}/.sqlfluff"
          paths: '${{ github.workspace }}/models'
          fail_on_error: true
          dialect: bigquery
          extra_requirements_txt: "${{ github.workspace }}/requirements.txt"
          processes: 10

.sqlfluff

# https://docs.sqlfluff.com/
[sqlfluff]
templater = dbt
dialect = bigquery
# Exclude: 
# L008 Space after comma.
# L029 Keywords should not be used as identifiers.
# L032 Prefer specifying join keys instead of using USING.
# L034 Select wildcards then simple targets before calculations and aggregates.
# L048 Quoted literals should be surrounded by a single whitespace.
# L054 Inconsistent column references in GROUP BY/ORDER BY clauses.
exclude_rules = L008, L029, L032, L034, L048, L054
# Number of passes to run before admitting defeat, default = 10
# runaway_limit = 100

[sqlfluff:rules:L003]
# Indentation not consistent with previous lines.
tab_space_size = 2

[sqlfluff:rules:L010]
# Keywords
capitalisation_policy = lower

[sqlfluff:rules:L014]
# Unquoted identifiers
extended_capitalisation_policy = lower

[sqlfluff:rules:L016]
max_line_length = 120
ignore_comment_clauses = True

[sqlfluff:rules:L019]
comma_style = leading

[sqlfluff:rules:L030]
# Function names
extended_capitalisation_policy = lower

[sqlfluff:rules:L040]
# Null & Boolean Literals
capitalisation_policy = lower

[sqlfluff:rules:L063]
# Data Types
extended_capitalisation_policy = lower

[sqlfluff:rules:L064]
# Consistent usage of preferred quotes for quoted literals
preferred_quoted_literal_style = single_quotes

output of local run sqlfluff, version 1.2.1

sqlfluff lint models/test_sqlfluff.sql
=== [dbt templater] Sorting Nodes...
=== [dbt templater] Compiling dbt project...
=== [dbt templater] Project Compiled.
== [models/test_sqlfluff.sql] FAIL                                                 
L:   1 | P:   1 | L036 | Select targets should be on a new line unless there is
                       | only one select target.
L:   2 | P:  10 | L009 | Files must end with a single trailing newline.
All Finished 📜 🎉!

output of github action in "lint" mode

Running sqlfluff 🐶 ...
  14:15:28  Partial parse save file not found. Starting full parse.
  [{"filepath": "models/test_sqlfluff.sql", "violations": [{"line_no": 1, "line_pos": 1, "code": "L036", "description": "Select targets should be on a new line unless there is only one select target."}, {"line_no": 2, "line_pos": 10, "code": "L009", "description": "Files must end with a single trailing newline."}]}]
  parse error: Expected string key before ':' at line 1, column 3
 Running reviewdog 🐶 ...
  parse error: Expected string key before ':' at line 1, column 3
  reviewdog: parse error: failed to unmarshal rdjson (DiagnosticResult): proto: syntax error (line 1:1): unexpected token

Everything the same, but changing the test.yaml to "fix" mode, everything works as expected

name: sqlfluff with reviewdog
on:
  pull_request:
jobs:
  test-check:
    name: runner / sqlfluff (github-check)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Decode service account credentials
        env:
          # The base64 encoded credentials.
          GOOGLE_APPLICATION_CREDENTIALS_BASE64: "${{ secrets.GOOGLE_APPLICATION_CREDENTIALS_BASE64 }}"
        run: |
          echo "...Decoding GOOGLE_APPLICATION_CREDENTIALS_BASE64..."
          python decode_base64.py || exit 1
          echo "...and validating json output..."
          python -m json.tool gcloud.json > /dev/null || exit 1

      - uses: yu-iskw/action-sqlfluff@v3.3.3
        env:
          DBT_PROFILES_DIR: ${{ github.workspace }}/.github/sqlfluff/.dbt/
        id: lint_sql
        with:
          github_token: ${{ secrets.TOKEN_GITHUB }}
          reporter: github-pr-review
          sqlfluff_version: "1.2.1"
          sqlfluff_command: "fix" # "fix" or "lint"
          config: "${{ github.workspace }}/.sqlfluff"
          paths: '${{ github.workspace }}/models'
          fail_on_error: true
          dialect: bigquery
          extra_requirements_txt: "${{ github.workspace }}/requirements.txt"
          processes: 10

Github action output in "fix" mode

 Running sqlfluff 🐶 ...
  ==== finding fixable violations ====
  === [dbt templater] Sorting Nodes...
  14:20:24  Partial parse save file not found. Starting full parse.
  === [dbt templater] Compiling dbt project...
  === [dbt templater] Project Compiled.
  WARNING    One fix for L003 not applied, it would re-cause the same error. 
  == [models/test_sqlfluff.sql] FAIL
  L:   1 | P:   1 | L036 | Select targets should be on a new line unless there is
                         | only one select target.
  L:   2 | P:  10 | L009 | Files must end with a single trailing newline.
  ==== fixing violations ====
  2 fixable linting violations found
  FORCE MODE: Attempting fixes...
  Persisting Changes...
  == [models/test_sqlfluff.sql] PASS
  Done. Please check your files to confirm.
 Running reviewdog 🐶 ...
  diff --git a/models/test_sqlfluff.sql b/models/test_sqlfluff.sql
  index f6625431..fffca06a 100644
  --- a/models/test_sqlfluff.sql
  +++ b/models/test_sqlfluff.sql
  @@ -1,2 +1,4 @@
  -select blaa1, blaa2
  -from blaa
  \ No newline at end of file
  +select
  +  blaa1
  +, blaa2
  +from blaa
  Saved working directory and index state WIP on (no branch): 0907bae1 Merge 989b0596cddaa8f8b40d45ec4c8a4e3dc7b37552 into e6ef8ca8816c60a38cd686ed9cf8497348f73111
  models/test_sqlfluff.sql:1:-select blaa1, blaa2
  models/test_sqlfluff.sql:2:-from blaa
  models/test_sqlfluff.sql:1:+select
  models/test_sqlfluff.sql:2:+  blaa1
  models/test_sqlfluff.sql:3:+, blaa2
  models/test_sqlfluff.sql:4:+from blaa
  reviewdog: input data has violations
  Dropped refs/stash@{0} (b01474aac5a1f5dcffae1ba610c3254d1bb7b1db)

kieronellis · 2022-08-03T10:24:35Z

@yu-iskw could you reopen this issue or should I create a new one

When piping `sqlfluff lint` into `tee`, exit code becomes 0 instead of 1. To fix it, we persist the lint output using `--write-output` CLI flag from sqlfluff instead of using `tee`. fix yu-iskw#31 yu-iskw#43

yu-iskw closed this as completed Jul 20, 2022

kieronellis mentioned this issue Aug 29, 2022

SQLFluff parse error in lint mode #43

Closed

louisguitton mentioned this issue Jan 11, 2023

Fix parse error: Invalid numeric literal #55

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Parse error when running on latest version #31

Parse error when running on latest version #31

Aka-shi commented Jun 9, 2022

Aka-shi commented Jun 9, 2022

yu-iskw commented Jun 10, 2022

yu-iskw commented Jun 10, 2022

yu-iskw commented Jun 23, 2022

yu-iskw commented Jul 20, 2022

kieronellis commented Jul 21, 2022

kieronellis commented Aug 3, 2022

Parse error when running on latest version #31

Parse error when running on latest version #31

Comments

Aka-shi commented Jun 9, 2022

Aka-shi commented Jun 9, 2022

yu-iskw commented Jun 10, 2022

yu-iskw commented Jun 10, 2022

yu-iskw commented Jun 23, 2022

yu-iskw commented Jul 20, 2022

kieronellis commented Jul 21, 2022

kieronellis commented Aug 3, 2022