Syntax Highlighting Breaks In Multi-language Buffers

[Swivelgames]

This issued is based on the comment by @Swivelgames in #2790

Anyone still experiencing syntax highlighting issues with large files should comment in the previous ticket or make a new one. This issue is specifically related to multi-language buffers.

Syntax Highlighting Breaks In Multi-language Buffers

Reproducible in VIM 8.2 on multiple platforms

VIM - Vi IMproved 8.2 (2019 Dec 12, compiled May 13 2022 20:56:28)
macOS version - x86_64
Included patches: 1-4950
Compiled by Homebrew
Huge version without GUI.  Features included (+) or not (-):
+acl               +file_in_path      +mouse_urxvt       -tag_any_white
+arabic            +find_in_path      +mouse_xterm       -tcl
+autocmd           +float             +multi_byte        +termguicolors
+autochdir         +folding           +multi_lang        +terminal
-autoservername    -footer            -mzscheme          +terminfo
-balloon_eval      +fork()            +netbeans_intg     +termresponse
+balloon_eval_term +gettext           +num64             +textobjects
-browse            -hangul_input      +packages          +textprop
++builtin_terms    +iconv             +path_extra        +timers
+byte_offset       +insert_expand     +perl              +title
+channel           +ipv6              +persistent_undo   -toolbar
+cindent           +job               +popupwin          +user_commands
-clientserver      +jumplist          +postscript        +vartabs
+clipboard         +keymap            +printer           +vertsplit
+cmdline_compl     +lambda            +profile           +vim9script
+cmdline_hist      +langmap           -python            +viminfo
+cmdline_info      +libcall           +python3           +virtualedit
+comments          +linebreak         +quickfix          +visual
+conceal           +lispindent        +reltime           +visualextra
+cryptv            +listcmds          +rightleft         +vreplace
+cscope            +localmap          +ruby              +wildignore
+cursorbind        +lua               +scrollbind        +wildmenu
+cursorshape       +menu              +signs             +windows
+dialog_con        +mksession         +smartindent       +writebackup
+diff              +modify_fname      -sodium            -X11
+digraphs          +mouse             -sound             -xfontset
-dnd               -mouseshape        +spell             -xim
-ebcdic            +mouse_dec         +startuptime       -xpm
+emacs_tags        -mouse_gpm         +statusline        -xsmp
+eval              -mouse_jsbterm     -sun_workshop      -xterm_clipboard
+ex_extra          +mouse_netterm     +syntax            -xterm_save
+extra_search      +mouse_sgr         +tag_binary        
-farsi             -mouse_sysmouse    -tag_old_static    
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
  fall-back for $VIM: "/usr/local/share/vim"
Compilation: clang -c -I. -Iproto -DHAVE_CONFIG_H -DMACOS_X -DMACOS_X_DARWIN -g -O2 -D_REENTRANT -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 
Linking: clang -L. -fstack-protector-strong -L/usr/local/lib -L/usr/local/opt/libyaml/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/readline/lib -L/usr/local/lib -o vim -lm -lncurses -liconv -lintl -framework AppKit -L/usr/local/opt/lua/lib -llua5.4 -mmacosx-version-min=12.0 -fstack-protector-strong -L/usr/local/lib -L/usr/local/Cellar/perl/5.34.0/lib/perl5/5.34.0/darwin-thread-multi-2level/CORE -lperl -L/usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/config-3.10-darwin -lpython3.10 -framework CoreFoundation -lruby.3.1 -L/usr/local/Cellar/ruby/3.1.2/lib 

When editing a file that contains more than one syntax-highlighted language, certain scenarios cause VIM to revert to highlighting only the primary language.

@Tao-Quixote had a great example of this, where a file that is using HTML/Jade, CSS/SASS/Stylus, and JavaScript/TypeScript/CoffeeScript appears to only highlight the first parent syntax language after the breakage occurs.

So far, the only thing that works is manually running :syntax sync fromstart whenever this happens.

Reproduction Steps

The easiest way to reproduce the issue, that I have found, is to:

• Open VIM (# vim)
• Edit the problematic file (:e my-file) (In my case, it was a ~1000 line Vue.js file)
• Jump to the bottom of the problematic file (G)
• Open a new Buffer in the same window (:e some-other-file)
• Switch back to the problematic buffer (:bp)

Expected

• Syntax highlighting will be applied appropriately for all languages present in the file

Actual

• Syntax highlighting is only performed for primary language; Remaining languages are either highlighted with that syntax, or not at all.

Example File

Example derived from the comment by @Tao-Quixote in #2790

<template>
  <div class="cntr">
    <!-- ... -->
  </div>
</template>

<style>
  .cntr {
    color: red;
    /* ... */
  }
</style>

<script>
  export default {
    props: {
      ...
    },
    methods: {
      function foo () {}
    }
  }
</script>

Blah, blah, blah. Proper name. Place name. Backstory stuff...

EDIT: This section used to be collapsed. I removed the collapsible portion to make sure this section does not get overlooked, since it already addresses the first response to this ticket.

This issue is based on the original issue outlined in #2790. However, upon further inspection, I believe there are actually two separate problems being discussed in that story:

• Syntax Highlighting fails for large files (this is most likely due to redrawtime)
• Syntax Highlighting inconsistently applies syntax when multiple syntaxes are present in a file

The former may be fixed sometimes with one of these two configuration changes, because syntax highlighting seems to halt when it takes longer to execute than the threshold defined by redrawtime:

set redrawtime=10000 " If something breaks, you may have to wait 10 seconds for VIM to unfreeze
set re=0 " Force VIM to use the new RegEx engine, because it's faster/more-efficient/etc

The latter issue mentioned above is fixed during each occurrence by running :syntax sync fromstart.

These two solutions are mutally exclusive to their respective issues, highlighting the separation (no pun intended).

Upon inspection, both of these issues are valid and continue to garner attention despite the issue remaining closed. Neither of these should be broken out of the box, of course, as these are major usability issues. And, of course, anything should still be considered a bug if the solution is to introduce extraneous configuration in order to make a particular feature work consistently; Especially if that configuration is not immediately obvious, requires excessive values, and requires the user to dig around through closed issues in order to find the workaround. It may be worth it to explore whether or not the former issue is still valid.

However, again, discussions related to issues with syntax highlighting for large files should be limited to the previous ticket or a new one. This issue is specifically related to multi-language buffers.

[chrisbra]

Note sure why you opened a new ticket, but please note, including several other syntax rules inside one single file, may be the cause of it. It may happen, that some syntax rules do not play together very well (and may not even expect to find parts of the language that are clearly not allowed for that language) when combined and it may cause already heavy syntax highlighting to become even more slow, and syncing may be off.

You already seemed to have found syntax sync fromstart and I think this is the proper solution to your problem. It may make sense to check how fast the syntax engine works by running the syntime command. Also it may make sense to check the particular syntax files in place to see if they have some syn sync rules already applied which may not fit for your document.

[Swivelgames]

Note sure why you opened a new ticket, [...]

I acknowledge that I initially fell into the same trap when I wrote my comment on the previous ticket! 😅

The Backstory section of the issue above outlines the justification for creating this new ticket. These are separate issues. I believe you're still conflating the two issues as well. As I pointed out, the previous ticket was related to Large Files, not to multi-language buffers. Not only that, the ticket was closed.

After it was closed, a few people (including myself) with the Multi-language issue contributed to the thread, under the assumption that the two issues were identical. After investigating both, I discovered that, while they're both within the same problem space/domain (syntax highlighting), they're two very separate issues.

I'm sure you're busy with a number of other things. Out of respect for everyone's time, I took the time to debug, reproduce, and demonstrate the differences between both issues. Not only was this done in my original post, but I will also elaborate below. 🙂

[Swivelgames]

[...] but please note, including several other syntax rules inside one single file, may be the cause of it. [...]

Correct! That is the scope of this issue ticket. 😄 In the end, I'm pretty sure this comes down to areas of responsibility.

[...] It may happen, that some syntax rules do not play together very well (and may not even expect to find parts of the language that are clearly not allowed for that language) when combined and it may cause already heavy syntax highlighting to become even more slow, and syncing may be off. [...] It may make sense to check how fast the syntax engine works by running the syntime command. Also it may make sense to check the particular syntax files in place to see if they have some syn sync rules already applied which may not fit for your document.

As previously mentioned, the Multi-language Highlighting issue is not due to slowness

This is, again, another example of conflating the two issues mentioned before. The Large File issue was related to slowness. This is another issue entirely. This is underscored by the following:

1. syntime report does not reveal exorbitant timings
2. This is reinforced by the fact that set redrawtime workaround has no effect
3. set re workaround has no effect
4. Lastly, syntax highlighting actually completes to the end of the file, but it only highlights first language.

I even applied your changes to 8.2.5009, set set redrawtime=5, and witnessed a message. But when I exited vim, and reproduced the issue (as outlined below), no message appeared, underscoring the fact that this issue isn't due to timing or slowness.

This issue was reproduced after isolating it to a fresh compilation of vim master

More importantly, this issue was isolated and reproduced using a fresh compilation of vim from source using version 8.2.5009

Reproduction steps:

1. Run vim --clean -u NONE ./victimizedFile
2. Type :syntax on
3. Type G

Notice the file's syntax is highlighted, but sub-languages are not. In fact, Steps 2 and 3 can be done in any order.

Reproduction Examples

Results of syntime report

# added by me for emphasis to mark lines of significance

  TOTAL      COUNT  MATCH   SLOWEST     AVERAGE   NAME               PATTERN
  0.000380   100    62      0.000037    0.000004  htmlTagError       [^>]<
  0.000373   70     40      0.000041    0.000005  htmlArg            \<\(http-equiv\|href\|title\)=
  0.000332   56     21      0.000093    0.000006  htmlTagName        \<\(b\|i\|u\|h[1-6]\|em\|strong\|head\|body\|title\)\>
  0.000244   36     8       0.000066    0.000007  htmlArg            \<\(accept-charset\|label\)\>
  0.000133   92     77      0.000032    0.000001  htmlError          [<>&]
  0.000127   75     22      0.000009    0.000002  htmlLink           <a\>\_[^>]*\<href\>
  0.000119   111    73      0.000022    0.000001  htmlTagN           </\s*[-a-zA-Z0-9]\+
  0.000097   90     52      0.000018    0.000001  htmlTagN           <\s*[-a-zA-Z0-9]\+
  0.000058   95     54      0.000002    0.000001  htmlTag            <[^/]
  0.000056   54     1       0.000024    0.000001  htmlH2             <h2\>
  0.000049   53     0       0.000018    0.000001  htmlItalic         <i\>
  0.000049   30     0       0.000006    0.000002  htmlArg            \<aria-\%(activedescendant\|atomic\|autocomplete\|busy\|checked\|
  0.000048   53     0       0.000017    0.000001  htmlTitle          <title\>
# 0.000046   53     0       0.000003    0.000001  javaScript         <script \_[^>]*language *=\_[^>]*vbscript\_[^>]*>
  0.000045   53     0       0.000020    0.000001  htmlStrike         <del\>
  0.000043   53     0       0.000019    0.000001  htmlHead           <head\>
  0.000037   53     0       0.000017    0.000001  htmlStrike         <strike\>
# 0.000031   53     0       0.000003    0.000001  javaScript         <script\_[^>]*>
  0.000031   53     0       0.000002    0.000001  htmlBold           <b\>
  0.000030   57     4       0.000003    0.000001  htmlItalic         <em\>
  0.000029   53     0       0.000003    0.000001  htmlH6             <h6\>
  0.000029   53     0       0.000003    0.000001  htmlComment        <!--\%(-\?>\)\@!
  0.000028   53     0       0.000003    0.000001  htmlH4             <h4\>
  0.000027   53     0       0.000002    0.000001  htmlUnderline      <u\>
  0.000027   54     1       0.000002    0.000001  htmlH3             <h3\>
  0.000025   55     2       0.000003    0.000000  htmlBold           <strong\>
  0.000024   53     0       0.000003    0.000000  htmlH5             <h5\>
  0.000023   54     1       0.000002    0.000000  htmlH1             <h1\>
  0.000022   72     72      0.000017    0.000000  htmlEndTag         >
  0.000021   95     68      0.000001    0.000000  htmlEndTag         </
  0.000020   53     0       0.000003    0.000000  htmlCssDefinition  style="
  0.000020   30     0       0.000005    0.000001  htmlEvent          \<on\a\+\s*=[\t ]*'
  0.000020   30     0       0.000003    0.000001  htmlEvent          \<on\a\+\s*=[\t ]*"
  0.000019   130    130     0.000001    0.000000  htmlTag            >
  0.000015   30     0       0.000002    0.000001  htmlValue          =[\t ]*[^'" \t>][^ \t>]*
  0.000014   53     0       0.000002    0.000000  htmlComment        <!DOCTYPE
  0.000013   12     12      0.000002    0.000001  htmlLink           </a\_s*>
# 0.000010   53     0       0.000002    0.000000  cssStyle           <style
  0.000009   84     54      0.000001    0.000000  htmlString         "
  0.000008   53     0       0.000002    0.000000  htmlSpecialChar    &#\=[0-9A-Za-z]\{1,8};
  0.000007   53     0       0.000001    0.000000  htmlComment        <!
  0.000006   53     0       0.000001    0.000000  htmlPreProc        <!--#
  0.000005   30     0       0.000002    0.000000  htmlArg            \<z-index\>
  0.000005   30     0       0.000001    0.000000  htmlString         '
  0.000005   24     24      0.000001    0.000000  htmlString         "
  0.000003   1      0       0.000003    0.000003  htmlBoldItalic     <i\>
  0.000002   2      2       0.000001    0.000001  htmlItalic         </em\_s*>
  0.000002   2      2       0.000001    0.000001  htmlH2             </h2\_s*>
  0.000002   2      2       0.000001    0.000001  htmlH1             </h1\_s*>
  0.000002   2      2       0.000001    0.000001  htmlBold           </strong\_s*>
  0.000002   1      0       0.000002    0.000002  htmlBoldUnderline  <u\>
  0.000002   2      2       0.000001    0.000001  htmlBoldItalic     <em\>
  0.000002   2      2       0.000001    0.000001  htmlH3             </h3\_s*>
# 0.000001   12     0       0.000001    0.000000  javaScriptExpression &{
  0.000000   6      0       0.000000    0.000000  htmlLeadingSpace   ^\s\+

  0.002777   2635

Source code for ./victimizedFile.html example

<template>
	<p>Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Vestibulum tortor quam, feugiat vitae, ultricies eget, tempor sit amet, ante. Donec eu libero sit amet quam egestas semper. Aenean ultricies mi vitae est. Mauris placerat eleifend leo.</p>

	<form action="#" method="post">
		<div>
			<label for="textarea">Textarea:</label>
			<textarea cols="40" rows="8" name="textarea" id="textarea"></textarea>
		</div>

		<div>
			<label for="checkbox">Checkbox:</label>
			<input type="checkbox" name="checkbox" id="checkbox" />
		</div>

		<div>
			<input type="submit" value="Submit" />
		</div>
	</form>

	<nav>
		<ul>
			<li><a href="#nowhere" title="Lorum ipsum dolor sit amet">Lorem</a></li>
			<li><a href="#nowhere" title="Aliquam tincidunt mauris eu risus">Aliquam</a></li>
			<li><a href="#nowhere" title="Morbi in sem quis dui placerat ornare">Morbi</a></li>
			<li><a href="#nowhere" title="Praesent dapibus, neque id cursus faucibus">Praesent</a></li>
			<li><a href="#nowhere" title="Pellentesque fermentum dolor">Pellentesque</a></li>
		</ul>
	</nav>

	<h1>HTML Ipsum Presents</h1>

	<p><strong>Pellentesque habitant morbi tristique</strong> senectus et netus et malesuada fames ac turpis egestas. Vestibulum tortor quam, feugiat vitae, ultricies eget, tempor sit amet, ante. Donec eu libero sit amet quam egestas semper. <em>Aenean ultricies mi vitae est.</em> Mauris placerat eleifend leo. Quisque sit amet est et sapien ullamcorper pharetra. Vestibulum erat wisi, condimentum sed, <code>commodo vitae</code>, ornare sit amet, wisi. Aenean fermentum, elit eget tincidunt condimentum, eros ipsum rutrum orci, sagittis tempus lacus enim ac dui. <a href="#">Donec non enim</a> in turpis pulvinar facilisis. Ut felis.</p>

	<h2>Header Level 2</h2>

	<ol>
	   <li>Lorem ipsum dolor sit amet, consectetuer adipiscing elit.</li>
	   <li>Aliquam tincidunt mauris eu risus.</li>
	</ol>

	<blockquote><p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vivamus magna. Cras in mi at felis aliquet congue. Ut a est eget ligula molestie gravida. Curabitur massa. Donec eleifend, libero at sagittis mollis, tellus est malesuada tellus, at luctus turpis elit sit amet quam. Vivamus pretium ornare est.</p></blockquote>

	<h3>Header Level 3</h3>

	<ul>
	   <li>Lorem ipsum dolor sit amet, consectetuer adipiscing elit.</li>
	   <li>Aliquam tincidunt mauris eu risus.</li>
	</ul>

	<pre><code>
	#header h1 a {
	  display: block;
	  width: 300px;
	  height: 80px;
	}
	</code></pre>
</template>

<script type="text/javascript">
/**
 * @fileOverview Generates "Lorem ipsum" style text.
 * @author rviscomi@gmail.com Rick Viscomi,
 * 		tinsley@tinsology.net Mathew Tinsley
 * @version 1.0
 */

/**
 *	Copyright (c) 2009, Mathew Tinsley (tinsley@tinsology.net)
 *	All rights reserved.
 *
 *	Redistribution and use in source and binary forms, with or without
 *	modification, are permitted provided that the following conditions are met:
 *		* Redistributions of source code must retain the above copyright
 *		  notice, this list of conditions and the following disclaimer.
 *		* Redistributions in binary form must reproduce the above copyright
 *		  notice, this list of conditions and the following disclaimer in the
 *		  documentation and/or other materials provided with the distribution.
 *		* Neither the name of the organization nor the
 *		  names of its contributors may be used to endorse or promote products
 *		  derived from this software without specific prior written permission.
 *
 *	THIS SOFTWARE IS PROVIDED BY MATHEW TINSLEY ''AS IS'' AND ANY
 *	EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 *	WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 *	DISCLAIMED. IN NO EVENT SHALL <copyright holder> BE LIABLE FOR ANY
 *	DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 *	(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 *	LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 *	ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 *	(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 *	SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */

/**
 * @class Jibborish generator.
 */
var LoremIpsum = function () {
};

/**
 * Average number of words per sentence.
 * @constant {number}
 */
LoremIpsum.WORDS_PER_SENTENCE_AVG = 24.460;

/**
 * Standard deviation of the number of words per sentence.
 * @constant {number}
 */
LoremIpsum.WORDS_PER_SENTENCE_STD = 5.080;

/**
 * List of possible words.
 * @constant {Array.string}
 */
LoremIpsum.WORDS = [
		'lorem', 'ipsum', 'dolor', 'sit', 'amet', 'consectetur',
		'adipiscing', 'elit', 'curabitur', 'vel', 'hendrerit', 'libero',
		'eleifend', 'blandit', 'nunc', 'ornare', 'odio', 'ut',
		'orci', 'gravida', 'imperdiet', 'nullam', 'purus', 'lacinia',
		'a', 'pretium', 'quis', 'congue', 'praesent', 'sagittis', 
		'laoreet', 'auctor', 'mauris', 'non', 'velit', 'eros',
		'dictum', 'proin', 'accumsan', 'sapien', 'nec', 'massa',
		'volutpat', 'venenatis', 'sed', 'eu', 'molestie', 'lacus',
		'quisque', 'porttitor', 'ligula', 'dui', 'mollis', 'tempus',
		'at', 'magna', 'vestibulum', 'turpis', 'ac', 'diam',
		'tincidunt', 'id', 'condimentum', 'enim', 'sodales', 'in',
		'hac', 'habitasse', 'platea', 'dictumst', 'aenean', 'neque',
		'fusce', 'augue', 'leo', 'eget', 'semper', 'mattis', 
		'tortor', 'scelerisque', 'nulla', 'interdum', 'tellus', 'malesuada',
		'rhoncus', 'porta', 'sem', 'aliquet', 'et', 'nam',
		'suspendisse', 'potenti', 'vivamus', 'luctus', 'fringilla', 'erat',
		'donec', 'justo', 'vehicula', 'ultricies', 'varius', 'ante',
		'primis', 'faucibus', 'ultrices', 'posuere', 'cubilia', 'curae',
		'etiam', 'cursus', 'aliquam', 'quam', 'dapibus', 'nisl',
		'feugiat', 'egestas', 'class', 'aptent', 'taciti', 'sociosqu',
		'ad', 'litora', 'torquent', 'per', 'conubia', 'nostra',
		'inceptos', 'himenaeos', 'phasellus', 'nibh', 'pulvinar', 'vitae',
		'urna', 'iaculis', 'lobortis', 'nisi', 'viverra', 'arcu',
		'morbi', 'pellentesque', 'metus', 'commodo', 'ut', 'facilisis',
		'felis', 'tristique', 'ullamcorper', 'placerat', 'aenean', 'convallis',
		'sollicitudin', 'integer', 'rutrum', 'duis', 'est', 'etiam',
		'bibendum', 'donec', 'pharetra', 'vulputate', 'maecenas', 'mi',
		'fermentum', 'consequat', 'suscipit', 'aliquam', 'habitant', 'senectus',
		'netus', 'fames', 'quisque', 'euismod', 'curabitur', 'lectus',
		'elementum', 'tempor', 'risus', 'cras'
];

/**
 * Generate "Lorem ipsum" style words.
 * @param num_words {number} Number of words to generate.
 * @return {string} "Lorem ipsum..."
 */
LoremIpsum.prototype.generate = function (num_words) {
	var words, ii, position, word, current, sentences, sentence_length, sentence;
	
	/**
	 * @default 100
	 */
	num_words = num_words || 100;
	
	words = [LoremIpsum.WORDS[0], LoremIpsum.WORDS[1]];
	num_words -= 2;
	
	for (ii = 0; ii < num_words; ii++) {
		position = Math.floor(Math.random() * LoremIpsum.WORDS.length);
		word = LoremIpsum.WORDS[position];
		
		if (ii > 0 && words[ii - 1] === word) {
			ii -= 1;
			
		} else {
			words[ii] = word;
		}
	}
	
	sentences = [];
	current = 0;
	
	while (num_words > 0) {
		sentence_length = this.getRandomSentenceLength();
		
		if (num_words - sentence_length < 4) {
			sentence_length = num_words;
		}
		
		num_words -= sentence_length;
		
		sentence = [];
		
		for (ii = current; ii < (current + sentence_length); ii++) {
			sentence.push(words[ii]);
		}
		
		sentence = this.punctuate(sentence);
		current += sentence_length;
		sentences.push(sentence.join(' '));
	}
	
	return sentences.join(' ');
};

/**
 * Insert commas and periods in the given sentence.
 * @param {Array.string} sentence List of words in the sentence.
 * @return {Array.string} Sentence with punctuation added.
 */
LoremIpsum.prototype.punctuate = function (sentence) {
	var word_length, num_commas, ii, position;
	
	word_length = sentence.length;
	
	/* End the sentence with a period. */
	sentence[word_length - 1] += '.';
	
	if (word_length < 4) {
		return sentence;
	}
	
	num_commas = this.getRandomCommaCount(word_length);
	
	for (ii = 0; ii <= num_commas; ii++) {
		position = Math.round(ii * word_length / (num_commas + 1));
		
		if (position < (word_length - 1) && position > 0) {
			/* Add the comma. */
			sentence[position] += ',';
		}
	}
	
	/* Capitalize the first word in the sentence. */
	sentence[0] = sentence[0].charAt(0).toUpperCase() + sentence[0].slice(1);
	
	return sentence;
};

/**
 * Produces a random number of commas.
 * @param {number} word_length Number of words in the sentence.
 * @return {number} Random number of commas
 */
LoremIpsum.prototype.getRandomCommaCount = function (word_length) {
	var base, average, standard_deviation;
	
	/* Arbitrary. */
	base = 6;
	
	average = Math.log(word_length) / Math.log(base);
	standard_deviation = average / base;
	
	return Math.round(this.gaussMS(average, standard_deviation));
};

/**
 * Produces a random sentence length based on the average word length
 * of an English sentence.
 * @return {number} Random sentence length
 */
LoremIpsum.prototype.getRandomSentenceLength = function () {
	return Math.round(
			this.gaussMS(
					LoremIpsum.WORDS_PER_SENTENCE_AVG,
					LoremIpsum.WORDS_PER_SENTENCE_STD
			)
	);
};

/**
 * Produces a random number.
 * @return {number} Random number
 */
LoremIpsum.prototype.gauss = function () {
	return (Math.random() * 2 - 1) +
			(Math.random() * 2 - 1) +
			(Math.random() * 2 - 1);
};

/**
 * Produces a random number with Gaussian distribution.
 * @param {number} mean
 * @param {number} standard_deviation
 * @return {number} Random number
 */
LoremIpsum.prototype.gaussMS = function (mean, standard_deviation) {
	return Math.round(this.gauss() * standard_deviation + mean);
};
</script>

Parts of this code were sourced from webfx.com and rviscomi/lorem-ipsum.js

[Swivelgames]

Additionally, it appears that the following remedies the issue, which may point to the source of the problem:

1. Jump to the top of the file: g
2. Scroll down (holding j, or by using Lzt, C-d, C-f, or other means)

Notice how the portion of the file that is visible within the buffer is thoroughly highlighted as you scroll, regardless of whether or not the code present on the screen is a sub-language.

Interestingly enough, if you jump to the last line (G) before making it to the end of the file, you will notice that the highlighting for the sub-language is not complete.

---

(Note that, while the above is, the below isn't consistently reproducible.)

Slowly scrolling up from there, you'll eventually land on a spot in the buffer where half of the code is highlighted, and the other half is not. Expanding on that, the bottom of the highlighted portion of code from the sub-language was the last visible line in the buffer when we scrolled down from the top.

[Swivelgames]

Here are all the reproduction steps outlined above in action:

./vim --version

VIM - Vi IMproved 8.2 (2019 Dec 12, compiled May 23 2022 14:22:12)
macOS version - x86_64
Included patches: 1-5009
Compiled by joseph.dalrymple@ALK-C02C96B8MD6R
Huge version without GUI.  Features included (+) or not (-):
+acl               +file_in_path      +mouse_urxvt       -tag_any_white
+arabic            +find_in_path      +mouse_xterm       -tcl
+autocmd           +float             +multi_byte        +termguicolors
+autochdir         +folding           +multi_lang        +terminal
-autoservername    -footer            -mzscheme          +terminfo
-balloon_eval      +fork()            +netbeans_intg     +termresponse
+balloon_eval_term +gettext           +num64             +textobjects
-browse            -hangul_input      +packages          +textprop
++builtin_terms    +iconv             +path_extra        +timers
+byte_offset       +insert_expand     -perl              +title
+channel           +ipv6              +persistent_undo   -toolbar
+cindent           +job               +popupwin          +user_commands
-clientserver      +jumplist          +postscript        +vartabs
+clipboard         +keymap            +printer           +vertsplit
+cmdline_compl     +lambda            +profile           +vim9script
+cmdline_hist      +langmap           -python            +viminfo
+cmdline_info      +libcall           -python3           +virtualedit
+comments          +linebreak         +quickfix          +visual
+conceal           +lispindent        +reltime           +visualextra
+cryptv            +listcmds          +rightleft         +vreplace
-cscope            +localmap          -ruby              +wildignore
+cursorbind        -lua               +scrollbind        +wildmenu
+cursorshape       +menu              +signs             +windows
+dialog_con        +mksession         +smartindent       +writebackup
+diff              +modify_fname      -sodium            -X11
+digraphs          +mouse             -sound             -xfontset
-dnd               -mouseshape        +spell             -xim
-ebcdic            +mouse_dec         +startuptime       -xpm
+emacs_tags        -mouse_gpm         +statusline        -xsmp
+eval              -mouse_jsbterm     -sun_workshop      -xterm_clipboard
+ex_extra          +mouse_netterm     +syntax            -xterm_save
+extra_search      +mouse_sgr         +tag_binary        
-farsi             -mouse_sysmouse    -tag_old_static    
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
  fall-back for $VIM: "/usr/local/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -DMACOS_X -DMACOS_X_DARWIN -g -O2 -D_REENTRANT -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 
Linking: 
gcc -L/usr/local/lib -o vim -lm -lncurses -liconv -lintl -framework AppKit 

https://asciinema.org/a/w2VwAbNeGSVSE4St7DwehJjuP

[Swivelgames]

@chrisbra Didn't mean to overload you with too much info, but does any of this help? My last comment is probably the most useful, I imagine.

[zachjensz]

I am experiencing this exact issue as well! Do we still not have a fix?

Installing treesitter fixes this issue however introduces inferior syntax highlighting (class/id selectors have same colour etc.) so I would rather use the built in syntax highlighting

[Swivelgames]

Not that I'm aware of. This is still broken for me.

Can you post your vim --version?

[chrisbra]

as mentioned before, this is highly dependent on the syntax script involved as well as the buffer content. I think each syntax script need to be addressed specifically (for which you may need to get into contact with the syntact script author, as mentioned in the file).

[Swivelgames]

That appears to be @dkearns, but I'm not sure.

@dkearns Any chance you have insight into this?

[cecamp]

Mostly this sounds like a synchronization problem, especially where "syn sync fromstart" is the fix. Vim provides several choices for this; see :help syn-sync . One may specify how many lines vim should start back at to begin synchronization. There are pros and cons to this: picking syn sync minlines too small will still exhibit problems and the bigger it is the slower vim'll be. I think this is really a user settings issue not a syntax/html.vim issue.

I think the other problem involves the choice of "region"s that delineate the different language zone. Its just a vim syntax region -- so if the start and end patterns are somehow ambiguous with the included language there'll be problems. See syntax/vim.vim for examples with including lua, perl, ruby, python, tcl, and mzscheme.

[dkearns]

@Swivelgames I only quickly skimmed your very thorough report but I'll have a better look soon.

It appears to be a syncing problem caused by the long JavaScript block comment. The HTML syntax file sets a low sync minlines value of 10. If you increase that to something like 500 it should work better.

[Hritik14]

I'm facing a similar issue when using https://github.com/preservim/vim-markdown with code fencing. The fenced code block loses syntax highlighting when I jump to end of file by G. It works on manually trigging :syntax sync fromstart