Handle internal errors elegantly and report the triggering sql file

[NiallRees]

At present, if an attempt to lint a file fails due to an unhandled internal error, the linting process quits and doesn't report which file is causing the issue. This makes it hard to identify what file has surfaced the internal error, which you need to do in order to .sqlfluffignore the file and work around it. This PR prints the exception as a warning with the file that has surfaced the issue.

Outcome:

WARNING    Unable to lint /Users/niallwoodward/dev/.../models/reporting/query.sql due to an internal error. Please report this as an issue with your query's contents and stacktrace below!
To hide this warning, add the failing file to .sqlfluffignore
Traceback (most recent call last):
  File "/Users/niallwoodward/dev/pull_requests/sqlfluff/src/sqlfluff/core/linter.py", line 1254, in lint_path
    target_file.read(), fname=fname, fix=fix, config=config
  File "/Users/niallwoodward/dev/pull_requests/sqlfluff/src/sqlfluff/core/linter.py", line 1044, in lint_string
    parsed = self.parse_string(in_str=in_str, fname=fname, config=config)
  File "/Users/niallwoodward/dev/pull_requests/sqlfluff/src/sqlfluff/core/linter.py", line 816, in parse_string
    tokens, lex_vs = lexer.lex(templated_file)
  File "/Users/niallwoodward/dev/pull_requests/sqlfluff/src/sqlfluff/core/parser/lexer.py", line 343, in lex
    return self.enrich_segments(segment_buff, raw), violations
  File "/Users/niallwoodward/dev/pull_requests/sqlfluff/src/sqlfluff/core/parser/lexer.py", line 373, in enrich_segments
    templated_slice
  File "/Users/niallwoodward/dev/pull_requests/sqlfluff/src/sqlfluff/core/templaters/base.py", line 289, in templated_slice_to_source_slice
    if stop_slices[-1][0] == "literal":
IndexError: list index out of range

WARNING    Indent balance test failed for '/Users/niallwoodward/dev/.../models/reporting/query2.sql'. Template indents will not be linted for this file.
WARNING    Indent balance test failed for '/Users/niallwoodward/dev/.../models/reporting/query3.sql'. Template indents will not be linted for this file.
==== summary ====
violations:        0 status:         PASS
______________________________________________________________________________________________________________ summary _______________________________________________________________________________________________________________
  sqlfluff: commands succeeded
  congratulations :)

[pwildenhain]

Never had seen the traceback module before, cool stuff 😎 .

Would you be able to at a test cast for a file that will cause an internal error, and make sure it gets the nice error message?

[NiallRees]

Never had seen the traceback module before, cool stuff 😎 .

Would you be able to at a test cast for a file that will cause an internal error, and make sure it gets the nice error message?

Had a lot of fun working out how to write a test case for this. Good call and thanks!

[dmateusp]

@NiallRees I'm not sure exactly how to test that PR haha, but just wanted to double check that we know which file in a directory caused the internal issue in case it's raised?

I'm seeing your test is on a specific file like sqlfluff lint my_project/b/x.sql and not sqlfluff lint my_project/ for example

[NiallRees]

@NiallRees I'm not sure exactly how to test that PR haha, but just wanted to double check that we know which file in a directory caused the internal issue in case it's raised?

I'm seeing your test is on a specific file like sqlfluff lint my_project/b/x.sql and not sqlfluff lint my_project/ for example

I did this as a follow up to #600, where it took ages to work out what sql file was surfacing the error. Does that answer your question? Not sure I quite understand.

[pwildenhain]

I'll sleep better knowing this test is here 😴

[dmateusp]

Ah, I got confused, thanks for linking the issue