sqlfluff fix keeps adding then removing whitespace

[CyberShadow]

Expected Behaviour

After a finite number of iterations (ideally 1 🙂), sqlfluff fix should eventually converge on an output that it is happy with.

Observed Behaviour

sqlfluff fix keeps adding whitespace before some for / endfor Jinja tags, and then removing it again upon a successive invocation (which makes it challenging to use as, e.g., a pre-commit hook).

Steps to Reproduce

$ echo '{% set elements = "foo" %}

SELECT
    {% for elem in elements %}
    {{ elem }},
    {% endfor %}

    {% for elem in elements %}
    {{ elem }}+
    {% endfor %}
    0
' > test_0.sql

$ for n in $(seq 5) ; do cp test_$((n-1)).sql test_$n.sql ; sqlfluff fix --force test_$n.sql ; done
(sqlfluff output omitted)
$ diff -u test_4.sql test_5.sql
--- test_4.sql	2021-09-23 16:36:30.696291287 +0000
+++ test_5.sql	2021-09-23 16:36:31.232985009 +0000
@@ -3,10 +3,10 @@
 SELECT
     {% for elem in elements %}
     {{ elem }},
-    {% endfor %}
+{% endfor %}
 
-        {% for elem in elements %}
+    {% for elem in elements %}
     {{ elem }} +
-    {% endfor %}
+{% endfor %}
     0

Dialect

None specified

Version

SQLFluff a50454c, Python 3.9.7

Configuration

None

[barrywhart]

I have a PR which addresses #1425. This seems like a different issue. Most likely, there are two different lint rules with contradictory "opinions" about good linting.

Until this is fixed, a possible workaround (and helpful info for fixing!) would be to run the fixer in verbose mode (-vvvv) and see which rules are firing each time. I think you'll see an alternating pair of rules from one run to the next. If you disable one of these rules, you should get stable lint output.

[CyberShadow]

Yep, the minimal set of rules needed to reproduce this seem to be --rules L001,L003,L039.

The full verbose output is quite a lot (doesn't fit in a comment):
https://dump.cy.md/a8156741fe241e88d5d67cb32d172923/06%3A49%3A42-upload.txt

[barrywhart]

I have a fix that makes L001 smarter -- it no longer removes "trailing whitespace" if there's template code following the whitespace. Does this fix the original issue, @CyberShadow?

[CyberShadow]

Does this fix the original issue, @CyberShadow?

Sorry for the late reply - yes, as far as I can tell, it does, thank you!