Vim folding ignores commentstring for HTML (foldmethod=marker)

[habamax]

Steps to reproduce:

1. open vim --clean
2. add paragraph:

Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Maecenas feugiat fermentum pretium. Cras eu dolor imperdiet justo mattis
pulvinar. Cras nec lectus ligula. Proin elementum luctus elit, a tincidunt quam
facilisis non. Nunc quis mauris non turpis finibus luctus. Maecenas ante
sapien, sagittis quis accumsan in, feugiat quis sem. Praesent et auctor libero.

3. set ft=html
4. set foldmethod=marker
5. set foldmarker={,}
6. put cursor inside paragraph and press zfip

expected: first row and last row should have <!--{--> and <!--}--> at the ends.

actual: first row and last row have { and }.

Demo

vim version:

VIM - Vi IMproved 8.1 (2018 May 18, compiled Dec  5 2018 23:02:23)
MS-Windows 64-bit GUI version with OLE support
Included patches: 1-565
Compiled by appveyor@APPVYR-WIN
Huge version with GUI.  Features included (+) or not (-):
+acl                +comments           +file_in_path       +listcmds           +persistent_undo    +tag_binary         +visualextra
+arabic             +conceal            +find_in_path       +localmap           -postscript         +tag_old_static     +viminfo
+autocmd            +cryptv             +float              +lua/dyn            +printer            -tag_any_white      +vreplace
+autochdir          +cscope             +folding            +menu               +profile            +tcl/dyn            -vtp
+autoservername     +cursorbind         -footer             +mksession          +python/dyn         -termguicolors      +wildignore
+balloon_eval       +cursorshape        +gettext/dyn        +modify_fname       +python3/dyn        +terminal           +wildmenu
-balloon_eval_term  +dialog_con_gui     -hangul_input       +mouse              +quickfix           -tgetent            +windows
+browse             +diff               +iconv/dyn          +mouseshape         +reltime            -termresponse       +writebackup
++builtin_terms     +digraphs           +insert_expand      +multi_byte_ime/dyn +rightleft          +textobjects        -xfontset
+byte_offset        +directx            +job                +multi_lang         +ruby/dyn           +timers             -xim
+channel            -dnd                +jumplist           +mzscheme/dyn       +scrollbind         +title              +xpm_w32
+cindent            -ebcdic             +keymap             +netbeans_intg      +signs              +toolbar            -xterm_save
+clientserver       +emacs_tags         +lambda             +num64              +smartindent        +user_commands      
+clipboard          +eval               +langmap            +ole                +startuptime        +vartabs            
+cmdline_compl      +ex_extra           +libcall            +packages           +statusline         +vertsplit          
+cmdline_hist       +extra_search       +linebreak          +path_extra         -sun_workshop       +virtualedit        
+cmdline_info       +farsi              +lispindent         +perl/dyn           +syntax             +visual             
   system vimrc file: "$VIM\vimrc"
     user vimrc file: "$HOME\_vimrc"
 2nd user vimrc file: "$HOME\vimfiles\vimrc"
 3rd user vimrc file: "$VIM\_vimrc"
      user exrc file: "$HOME\_exrc"
  2nd user exrc file: "$VIM\_exrc"
  system gvimrc file: "$VIM\gvimrc"
    user gvimrc file: "$HOME\_gvimrc"
2nd user gvimrc file: "$HOME\vimfiles\gvimrc"
3rd user gvimrc file: "$VIM\_gvimrc"
       defaults file: "$VIMRUNTIME\defaults.vim"
    system menu file: "$VIMRUNTIME\menu.vim"
Compilation: cl -c /W3 /nologo  -I. -Iproto -DHAVE_PATHDEF -DWIN32  -DFEAT_CSCOPE -DFEAT_TERMINAL -DFEAT_NETBEANS_INTG -DFEAT_JOB_CHANNEL   -DFEAT_XPM_W32   -DWINVER=0x0501 -D_WIN32_WINNT=0x0501 /MP -DHAVE_STDINT_H /Ox /GL -DNDEBUG  /Zl /MT -DFEAT_OLE -DFEAT_MBYTE_IME -DDYNAMIC_IME -DFEAT_MBYTE -DFEAT_GUI_W32 -DFEAT_DIRECTX -DDYNAMIC_DIRECTX -DFEAT_DIRECTX_COLOR_EMOJI -DDYNAMIC_ICONV -DDYNAMIC_GETTEXT -DFEAT_TCL -DDYNAMIC_TCL -DDYNAMIC_TCL_DLL=\"tcl86t.dll\" -DDYNAMIC_TCL_VER=\"8.6\" -DFEAT_LUA -DDYNAMIC_LUA -DDYNAMIC_LUA_DLL=\"lua53.dll\" -DFEAT_PYTHON -DDYNAMIC_PYTHON -DDYNAMIC_PYTHON_DLL=\"python27.dll\" -DFEAT_PYTHON3 -DDYNAMIC_PYTHON3 -DDYNAMIC_PYTHON3_DLL=\"python37.dll\" -DFEAT_MZSCHEME -I "C:\Program Files\Racket\include" -DMZ_PRECISE_GC -DDYNAMIC_MZSCHEME -DDYNAMIC_MZSCH_DLL=\"libracket3m_a36fs8.dll\" -DDYNAMIC_MZGC_DLL=\"libracket3m_a36fs8.dll\" -DFEAT_PERL -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DDYNAMIC_PERL -DDYNAMIC_PERL_DLL=\"perl528.dll\" -DFEAT_RUBY -DDYNAMIC_RUBY -DDYNAMIC_RUBY_VER=24 -DDYNAMIC_RUBY_DLL=\"x64-msvcrt-ruby240.dll\" -DFEAT_HUGE /Fd.\ObjGXOULYHTRZAMD64/ /Zi
Linking: link  /nologo /subsystem:windows,5.02 /opt:ref /LTCG:STATUS /HIGHENTROPYVA:NO oldnames.lib kernel32.lib advapi32.lib shell32.lib gdi32.lib  comdlg32.lib ole32.lib netapi32.lib uuid.lib /machine:AMD64 gdi32.lib version.lib   winspool.lib comctl32.lib advapi32.lib shell32.lib netapi32.lib  /machine:AMD64  libcmt.lib oleaut32.lib user32.lib  /nodefaultlib:lua53.lib  /STACK:8388608  /nodefaultlib:python27.lib /nodefaultlib:python37.lib   "C:\Tcl\lib\tclstub86.lib" WSock32.lib xpm\x64\lib-vc14\libXpm.lib /PDB:gvim.pdb -debug

[habamax]

If you setlocal comments= then it works.

@chrisbra believes it is v8.0.0453 to cause the issue.

The origin of the issue: https://vi.stackexchange.com/questions/18186/why-does-vim-ignore-commentstring-when-folding-foldmethod-marker

[chrisbra]

On a quick test, it seems #1549 (025a6b7) is causing this. I believe skip_comment() thinks the line is in a comment.

[chrisbra]

I have debugged that a bit. The problem comes from get_last_leader_offset() called from skip_comment() function. It sees a space in the current line as part of the middle part comment and therefore skip_comment thinks we are inside a comment.

Here is a patch, that checks specifically for middle part comments, if they are consist of only whitespace, that the whole line before the current position also consists of only whitespace.

[chrisbra]

chrisbra@d269fde

[habamax]

@chrisbra I am not sure I will be able to test it -- if it is required to build vim first.

[brammool]

Christian wrote:

I have debugged that a bit. The problem comes from `get_last_leader_offset()` called from `skip_comment()` function. It sees a space in the current line as part of the middle part comment and therefore `skip_comment` thinks we are inside a comment. Here is a patch, that checks specifically for middle part comments, if they are consist of only whitespace, that the whole line before the current position also consists of only whitespace.

Thanks. The test didn't pass, it expects a space to be inserted. I don't see how that would happen, thus I'll assume the test was wrong.

…

-- Mrs Abbott: I'm a paediatrician. Basil: Feet? Mrs Abbott: Children. Sybil: Oh, Basil! Basil: Well, children have feet, don't they? That's how they move around, my dear. You must take a look next time, it's most interesting. (Fawlty Towers) /// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \\\ /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ an exciting new programming language -- http://www.Zimbu.org /// \\\ help me help AIDS victims -- http://ICCF-Holland.org ///