sqlfluff lint: Add "github-annotation" format #1198

barrywhart · 2021-07-10T19:25:24Z

Resolves #1190

codecov · 2021-07-10T19:43:17Z

Codecov Report

Merging #1198 (966c681) into master (d3ec79d) will increase coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1198      +/-   ##
==========================================
+ Coverage   96.07%   96.10%   +0.02%     
==========================================
  Files         124      124              
  Lines        8469     8477       +8     
==========================================
+ Hits         8137     8147      +10     
+ Misses        332      330       -2

Flag	Coverage Δ
dbt018-py38	`61.40% <11.11%> (-0.05%)`	⬇️
dbt019-py38	`61.40% <11.11%> (-0.05%)`	⬇️
py36	`94.91% <100.00%> (+<0.01%)`	⬆️
py37	`94.94% <100.00%> (+<0.01%)`	⬆️
py38	`94.95% <100.00%> (+0.02%)`	⬆️
py39	`94.93% <100.00%> (+<0.01%)`	⬆️
win-py37	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/sqlfluff/cli/commands.py	`88.21% <100.00%> (+0.32%)`	⬆️
src/sqlfluff/core/linter/runner.py	`94.68% <0.00%> (+2.12%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d3ec79d...966c681. Read the comment docs.

barrywhart · 2021-07-11T01:10:26Z

@GClunies: Would it make sense to update the existing SQLFluff GitHub Action to leverage this? Are any changes required in order for this to work?

NiallRees · 2021-07-11T11:48:02Z

test/cli/commands_test.py

    elif serialize == "yaml":
        result = yaml.safe_load(result.output)
+        assert len(result) == 2
+    elif serialize == "github-annotation":
+        result = json.loads(result.output)


Let's add a unit test that asserts the full response?

NiallRees · 2021-07-11T11:48:35Z

Nice!

…rmat

src/sqlfluff/cli/commands.py

GClunies · 2021-07-13T15:27:11Z

Sorry for the late catch up here!

Just read through the original slack thread and GitHub Issue, I really like the idea proposed by @emily-hawkins and looks like this PR is well on its way to a solution @barrywhart!

From my brief review it seems like we would just need to modify the end of the GitHub workflow to be something like...

- name: Lint dbt models
        if: steps.get_files_to_lint.outputs.lintees != ''
        shell: bash -l {0}
        run: |
          sqlfluff lint --format github-annotation --annotation-level failure ${{ steps.get_files_to_lint.outputs.lintees }}
          
- name: Annotate PR
  uses: yuzutech/annotations-action@v0.1.0
  with:
    repo-token: "${{ secrets.GITHUB_TOKEN }}"
    title: 'SQLFluff'
    input: './annotations.json'

@emily-hawkins, have you had a chance to test this out on your project?

emily-hawkins · 2021-07-13T15:37:28Z

Sorry for the late catch up here!

Just read through the original slack thread and GitHub Issue, I really like the idea proposed by @emily-hawkins and looks like this PR is well on its way to a solution @barrywhart!

From my brief review it seems like we would just need to modify the end of the GitHub workflow to be something like...
- name: Lint dbt models
        if: steps.get_files_to_lint.outputs.lintees != ''
        shell: bash -l {0}
        run: |
          sqlfluff lint --format github-annotation --annotation-level failure ${{ steps.get_files_to_lint.outputs.lintees }}
          
- name: Annotate PR
  uses: yuzutech/annotations-action@v0.1.0
  with:
    repo-token: "${{ secrets.GITHUB_TOKEN }}"
    title: 'SQLFluff'
    input: './annotations.json'
@emily-hawkins, have you had a chance to test this out on your project?

Not yet! I was going to test it out once the params for annotation-level are added in

barrywhart · 2021-07-13T22:04:57Z

@emily-hawkins: Sorry, I didn't have time to work on this today. Will try and get to it tomorrow.

…rmat

barrywhart · 2021-07-13T22:27:24Z

@emily-hawkins: I added the --annotation-level just now.

barrywhart · 2021-07-14T02:07:20Z

@GClunies: A question about your suggested steps for the GitHub Action: is this going to redirect sqlfluff lint output to the file annotations.json? I don't see that it does, but maybe that's part of the string steps.get_files_to_lint.outputs.lintees?

Note that without redirection, running sqlfluff lint --format github-annotation will write the JSON linting output to stdout.

emily-hawkins · 2021-07-14T13:29:03Z

@emily-hawkins: I added the --annotation-level just now.

thank you!! i will test this out today

emily-hawkins · 2021-07-14T17:32:16Z

i got the sqlfluff parts to run on my end, but getting an issue now on the annotate action, which i put in an issue for yuzutech/annotations-action#36
hard to tell what could be causing it/if its just an issue with permissions or something.

emily-hawkins · 2021-07-14T20:14:38Z

i got the sqlfluff parts to run on my end, but getting an issue now on the annotate action, which i put in an issue for yuzutech/annotations-action#36
hard to tell what could be causing it/if its just an issue with permissions or something.

I got this to work!! there is something off in the newest version so downgrading the annotation action to 0.2.1 fixed it!

NiallRees · 2021-07-14T20:21:17Z

i got the sqlfluff parts to run on my end, but getting an issue now on the annotate action, which i put in an issue for yuzutech/annotations-action#36
hard to tell what could be causing it/if its just an issue with permissions or something.

I got this to work!! there is something off in the newest version so downgrading the annotation action to 0.2.1 fixed it!

This is so exciting! Would love to see a write-up if you can find the time!

emily-hawkins · 2021-07-14T20:41:10Z

for documentation, i can show what my workflows file looks like for 1) testing this PR and 2) how it will look when this is in production

for this PR specifically, there is some extra steps that need to happen so i can install sqlfluff with these changes.
first, im checking out this branch, installing python, dependencies, and building the distribution. then i can package that as an artifact and upload it. the next steps in lint-models are about downloading that artifact and then installing with pip. Then i follow the same steps as seen in https://github.com/sqlfluff/sqlfluff-github-actions/tree/main/menu_of_workflows/surfline to get changed files. I then need to save my sqlfluff lint output to a json file. The last step is running the annotations action using that file as an input. as i mentioned, the latest version of this action seems to be broken, so version 0.2.1 is a safest bet at this point. in production i wont need all the extra sqlfluff steps since i'll just be able to pip install sqlfluff like normal, so the workflow will be much more simple. (will post that below this first section)

name: SQLFluff lint dbt models

on:
  - pull_request

jobs:
  sqlfluff:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout sqlfluff branch
        uses: actions/checkout@v2
        with:
          repository: barrywhart/sqlfluff
          ref: bhart-issue_1190_lint_github_annotation_format
      - name: Install Python
        uses: "actions/setup-python@v2"
        with:
          python-version: "3.7"
      - name: Install Dependencies
        run: |
          pip install --upgrade pip
          pip install twine wheel --upgrade
      - name: Build Distribution
        run: "python setup.py sdist bdist_wheel"
      - name: Archive production artifacts
        uses: actions/upload-artifact@v2
        with:
          name: sqlfluff-dist
          path: dist/sqlfluff-0.6.0.tar.gz

  lint-models:
    runs-on: ubuntu-latest
    needs: sqlfluff
    steps:
      - name: checkout
        uses: actions/checkout@v2
      - name: Install Python
        uses: "actions/setup-python@v2"
        with:
          python-version: "3.7"
      - name: Download sqlfluff artifact
        uses: actions/download-artifact@v2
        with:
          name: sqlfluff-dist
      - name: install sqlfluff
        run: python3 -m pip install ./sqlfluff-0.6.0.tar.gz
      - name: Get changed files
        id: get_file_changes
        uses: trilom/file-changes-action@v1.2.3
        with:
          output: ' '

      - name: Get changed .sql files in /models to lint
        id: get_files_to_lint
        shell: bash -l {0}
        run: |
          # Set the command in the $() brackets as an output to use in later steps
          echo "::set-output name=lintees::$(
          # Issue where grep regular expressions don't work as expected on the
          # Github Actions shell, check dbt/models/ folder
          echo \
          $(echo ${{ steps.get_file_changes.outputs.files }} |
          tr -s ' ' '\n' |
          grep -E '^dbt/models.*[.]sql$' |
          tr -s '\n' ' ')
          )"
      - name: Lint dbt models
        id: sqlfluff_json
        if: steps.get_files_to_lint.outputs.lintees != ''
        shell: bash -l {0}
        run: sqlfluff lint --format github-annotation --annotation-level failure --nofail ${{ steps.get_files_to_lint.outputs.lintees }} > annotations.json
      - name: Annotate
        uses: yuzutech/annotations-action@v0.2.1
        with:
          repo-token: "${{ secrets.GITHUB_TOKEN }}"
          title: "SQLFluff Lint"
          input: "./annotations.json"

how the production version will look

name: SQLFluff lint dbt models

on:
  - pull_request

jobs:
  lint-models:
    runs-on: ubuntu-latest
    steps:
      - name: checkout
        uses: actions/checkout@v2
      - name: Install Python
        uses: "actions/setup-python@v2"
        with:
          python-version: "3.7"
      - name: install sqlfluff
        run: "pip install sqlfluff"
      - name: Get changed files
        id: get_file_changes
        uses: trilom/file-changes-action@v1.2.3
        with:
          output: ' '

      - name: Get changed .sql files in /models to lint
        id: get_files_to_lint
        shell: bash -l {0}
        run: |
          # Set the command in the $() brackets as an output to use in later steps
          echo "::set-output name=lintees::$(
          # Issue where grep regular expressions don't work as expected on the
          # Github Actions shell, check dbt/models/ folder
          echo \
          $(echo ${{ steps.get_file_changes.outputs.files }} |
          tr -s ' ' '\n' |
          grep -E '^dbt/models.*[.]sql$' |
          tr -s '\n' ' ')
          )"
      - name: Lint dbt models
        id: sqlfluff_json
        if: steps.get_files_to_lint.outputs.lintees != ''
        shell: bash -l {0}
        run: sqlfluff lint --format github-annotation --annotation-level failure --nofail ${{ steps.get_files_to_lint.outputs.lintees }} > annotations.json
      - name: Annotate
        uses: yuzutech/annotations-action@v0.2.1
        with:
          repo-token: "${{ secrets.GITHUB_TOKEN }}"
          title: "SQLFluff Lint"
          input: "./annotations.json"

GClunies · 2021-07-14T21:00:21Z

This is awesome @emily-hawkins and @barrywhart!

emily-hawkins · 2021-07-14T21:11:55Z

the person who owns the annotation action got back to me about why it might not be working in v 0.3.0
yuzutech/annotations-action#36 (comment)
maybe we can try his suggestion to see if thats all it is?

I think it does not work because you didn't set both the start_column and end_column fields. In other words, you cannot specify just one value, you need to provide both. Could you please try to add "end_column": 1 with version 0.3.0?

barrywhart · 2021-07-15T11:35:39Z

@emily-hawkins: I updated the output format to include the end_column field. Would you mind testing it again with annotation-action 0.3.0?

barrywhart · 2021-07-15T11:39:42Z

@emily-hawkins, @GClunies: I initially thought about documenting GitHub Actions and annotations in this PR, but I realize now it would be better to include it in https://github.com/sqlfluff/sqlfluff-github-actions. What do y'all think?

emily-hawkins · 2021-07-15T14:44:05Z

@emily-hawkins: I updated the output format to include the end_column field. Would you mind testing it again with annotation-action 0.3.0?

yes! ill test this out today

emily-hawkins · 2021-07-15T15:05:12Z

@emily-hawkins: I updated the output format to include the end_column field. Would you mind testing it again with annotation-action 0.3.0?

yes! ill test this out today

just tried it with v0.3.0 with these changes and it's working 🎉

barrywhart · 2021-07-15T15:36:09Z

Thanks so much, @emily-hawkins! I'm going to go ahead and merge this. It will be cool to get this documented in https://github.com/sqlfluff/sqlfluff-github-actions at some point.

@GClunies

GClunies · 2021-07-16T22:07:38Z

Thanks so much, @emily-hawkins! I'm going to go ahead and merge this. It will be cool to get this documented in https://github.com/sqlfluff/sqlfluff-github-actions at some point.

@barrywhart, it seems as simple as just modifying the workflow here so that the end is:

        - name: Lint dbt models
          id: sqlfluff_json
          if: steps.get_files_to_lint.outputs.lintees != ''
          shell: bash -l {0}
          run: |
            sqlfluff lint --format github-annotation --annotation-level failure --nofail ${{ steps.get_files_to_lint.outputs.lintees }} > annotations.json
      
        - name: Annotate SQLFluff Linting in PR
          uses: yuzutech/annotations-action@v0.2.1
          with:
            repo-token: "${{ secrets.GITHUB_TOKEN }}"
            title: "SQLFluff Lint"
            input: "./annotations.json"

Not sure much else needs to be done in terms of docs. This will "just work" from what I see.

Just about to go on PTO, so classic Friday rush to tie up loose end before I leave 😅 . Will PR on the plane!

matt-winkler · 2022-12-09T17:39:10Z

Hi team, I'm seeing what appears to be a related issue to this when setting up my own project. The issue I'm having is that the annotations step is blocked by the ==== readout . . . line at the top of the output file from the linting step. I was able to move forward by adding this very ugly grep to the process to keep only the valid json in annotations.json

sqlfluff lint --dialect snowflake --format github-annotation --annotation-level failure --nofail ${{ steps.get_files_to_lint.outputs.lintees }} | grep -o '\[{.*' > annotations.json

Am I missing a simpler way to do this?

sqlfluff lint: Add "github-annotation" format

07a4e6b

barrywhart marked this pull request as draft July 10, 2021 19:25

barrywhart mentioned this pull request Jul 10, 2021

Edit/add new JSON schema to work with github actions #1190

Closed

Black, add test

c9e62d9

barrywhart requested review from a team, alanmcruickshank and dmateusp and removed request for a team July 10, 2021 21:18

CHANGELOG.md and documentation

c46c64c

Fix doclint errors

b8d2e67

barrywhart marked this pull request as ready for review July 11, 2021 01:11

NiallRees reviewed Jul 11, 2021

View reviewed changes

Barry Hart added 2 commits July 12, 2021 08:27

Add a detailed test of github-annotation output format

b80e1ad

Merge branch 'master' into bhart-issue_1190_lint_github_annotation_fo…

200a8c1

…rmat

NiallRees approved these changes Jul 12, 2021

View reviewed changes

emily-hawkins reviewed Jul 12, 2021

View reviewed changes

src/sqlfluff/cli/commands.py Outdated Show resolved Hide resolved

Barry Hart added 2 commits July 13, 2021 18:25

Add "--annotation-level" lint option

11e023a

Merge branch 'master' into bhart-issue_1190_lint_github_annotation_fo…

59a3891

…rmat

Barry Hart added 2 commits July 13, 2021 21:57

Run Black, update test

70d9662

Run Black

a846d51

Add "end_column" field to annotations, tweak docs

6acd7f1

Doclint

a94de6e

Barry Hart added 3 commits July 15, 2021 07:45

Doc tweaks

b258f56

Doclint

f9c4bd8

Doclint

966c681

barrywhart merged commit 5b05bd6 into sqlfluff:master Jul 15, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sqlfluff lint: Add "github-annotation" format #1198

sqlfluff lint: Add "github-annotation" format #1198

barrywhart commented Jul 10, 2021 •

edited

Loading

codecov bot commented Jul 10, 2021 •

edited

Loading

barrywhart commented Jul 11, 2021

NiallRees Jul 11, 2021

barrywhart Jul 12, 2021

NiallRees commented Jul 11, 2021

GClunies commented Jul 13, 2021 •

edited

Loading

emily-hawkins commented Jul 13, 2021

barrywhart commented Jul 13, 2021

barrywhart commented Jul 13, 2021

barrywhart commented Jul 14, 2021

emily-hawkins commented Jul 14, 2021

emily-hawkins commented Jul 14, 2021

emily-hawkins commented Jul 14, 2021

NiallRees commented Jul 14, 2021

emily-hawkins commented Jul 14, 2021

GClunies commented Jul 14, 2021

emily-hawkins commented Jul 14, 2021

barrywhart commented Jul 15, 2021

barrywhart commented Jul 15, 2021

emily-hawkins commented Jul 15, 2021

emily-hawkins commented Jul 15, 2021

barrywhart commented Jul 15, 2021

GClunies commented Jul 16, 2021

matt-winkler commented Dec 9, 2022

sqlfluff lint: Add "github-annotation" format #1198

sqlfluff lint: Add "github-annotation" format #1198

Conversation

barrywhart commented Jul 10, 2021 • edited Loading

codecov bot commented Jul 10, 2021 • edited Loading

Codecov Report

barrywhart commented Jul 11, 2021

NiallRees Jul 11, 2021

Choose a reason for hiding this comment

barrywhart Jul 12, 2021

Choose a reason for hiding this comment

NiallRees commented Jul 11, 2021

GClunies commented Jul 13, 2021 • edited Loading

emily-hawkins commented Jul 13, 2021

barrywhart commented Jul 13, 2021

barrywhart commented Jul 13, 2021

barrywhart commented Jul 14, 2021

emily-hawkins commented Jul 14, 2021

emily-hawkins commented Jul 14, 2021

emily-hawkins commented Jul 14, 2021

NiallRees commented Jul 14, 2021

emily-hawkins commented Jul 14, 2021

GClunies commented Jul 14, 2021

emily-hawkins commented Jul 14, 2021

barrywhart commented Jul 15, 2021

barrywhart commented Jul 15, 2021

emily-hawkins commented Jul 15, 2021

emily-hawkins commented Jul 15, 2021

barrywhart commented Jul 15, 2021

GClunies commented Jul 16, 2021

matt-winkler commented Dec 9, 2022

barrywhart commented Jul 10, 2021 •

edited

Loading

codecov bot commented Jul 10, 2021 •

edited

Loading

GClunies commented Jul 13, 2021 •

edited

Loading