Refactor JinjaTracer: Split into two classes, break up "_slice_templa…

…te()" function (#2870)

* Minor cleanup/refactor of JinjaTracer

* More refactoring

* More refactoring

* More refactoring

* Add more type annotations

* Extract a function track_block_end() out of update_next_slice_indices()

* Make some functions static

* Split up _slice_template() some more

* More types and tidying

* RawSliceInfo: Provide default value for next_slice_indices

* Use RawSliceInfo instance instead of alt_id/alt_code variables

* Split JinjaTracer into two classes: JinjaAnalyzer and JinjaTracer

* Make idx a field, rename as idx_raw

* Tidying

* PR review

* PR review

* Add explanatory comment about str_buff and str_parts

Co-authored-by: Barry Hart <barry.hart@mailchimp.com>

requirements_dev.txt

@@ -22,6 +22,7 @@ types-chardet
 types-appdirs
 types-colorama
 types-pyyaml
+types-Jinja2
 types-regex
 # Requests is required for the util script
 requests

src/sqlfluff/core/templaters/jinja.py

@@ -24,7 +24,7 @@
     TemplatedFileSlice,
 )
 from sqlfluff.core.templaters.python import PythonTemplater
-from sqlfluff.core.templaters.slicers.tracer import JinjaTracer
+from sqlfluff.core.templaters.slicers.tracer import JinjaAnalyzer
 
 
 # Instantiate the templater logger
@@ -413,6 +413,7 @@ def slice_file(
         templater_logger.info("Slicing File Template")
         templater_logger.debug("    Raw String: %r", raw_str)
         templater_logger.debug("    Templated String: %r", templated_str)
-        tracer = JinjaTracer(raw_str, self._get_jinja_env(), make_template)
+        analyzer = JinjaAnalyzer(raw_str, self._get_jinja_env())
+        tracer = analyzer.analyze(make_template)
         trace = tracer.trace()
         return trace.raw_sliced, trace.sliced_file, trace.templated_str