Improve testsuites compatibility on the systems with 1s granularity timestamp

[LZhaoM]

Is your feature request about something that is currently impossible or hard to do? Please describe the problem.
When running make test in an OS with only 1s granularity timestamp, some tests are impossible to pass as the time leaving for timestamp changes is not long enough. For instance:

vim/src/testdir/test_buffer.vim

Lines 550 to 560 in 44c1c04

	call writefile(['one'], 'XallocFail5', 'D')
	if has('unix')
	edit XallocFail5
	" sleep for some time to make sure the timestamp is different
	sleep 200m
	call writefile(['two'], 'XallocFail5')
	set autoread
	call test_alloc_fail(GetAllocId('newbuf_bvars'), 0, 0)
	call assert_fails('checktime', 'E342:')
	set autoread&
	bw!

Sleep for a longer time like 2 seconds will be enough to pass the test.

In the test:

vim/src/testdir/test_diffmode.vim

Line 2097 in 44c1c04

func Test_diff_overlapped_diff_blocks_will_be_merged()

In those two functions:

vim/src/testdir/test_diffmode.vim

Lines 840 to 855 in 44c1c04

	func WriteDiffFiles(buf, list1, list2)
	call writefile(a:list1, 'Xdifile1')
	call writefile(a:list2, 'Xdifile2')
	if a:buf
	call term_sendkeys(a:buf, ":checktime\<CR>")
	endif
	endfunc
	func WriteDiffFiles3(buf, list1, list2, list3)
	call writefile(a:list1, 'Xdifile1')
	call writefile(a:list2, 'Xdifile2')
	call writefile(a:list3, 'Xdifile3')
	if a:buf
	call term_sendkeys(a:buf, ":checktime\<CR>")
	endif
	endfunc

I think some sleep time is needed for timestamp to change before timecheck. The reason why it's fine I think it's because of the high-precision timestamp on modern OSes.

Describe the solution you'd like
Investigate more tests like the cases above and add a if-else sleep statement like the following:

vim/src/testdir/test_filechanged.vim

Lines 14 to 19 in 44c1c04

	" Need to wait until the timestamp would change.
	if has('nanotime')
	sleep 10m
	else
	sleep 2
	endif

Additional context
Not sure if nanotime is the condition should be used in this case, currently has(nanotime) will be 1 when the library on the OS supports s.ST_MTIM_NSEC:

vim/src/configure.ac

Lines 3972 to 3995 in 44c1c04

	dnl nanoseconds field of struct stat
	AC_CACHE_CHECK([for nanoseconds field of struct stat],
	ac_cv_struct_st_mtim_nsec,
	[ac_save_CPPFLAGS="$CPPFLAGS"
	ac_cv_struct_st_mtim_nsec=no
	# st_mtim.tv_nsec -- the usual case
	# st_mtim._tv_nsec -- Solaris 2.6, if
	# (defined _XOPEN_SOURCE && _XOPEN_SOURCE_EXTENDED == 1
	# && !defined __EXTENSIONS__)
	# st_mtim.st__tim.tv_nsec -- UnixWare 2.1.2
	# st_mtime_n -- AIX 5.2 and above
	# st_mtimespec.tv_nsec -- Darwin (Mac OSX)
	for ac_val in st_mtim.tv_nsec st_mtim._tv_nsec st_mtim.st__tim.tv_nsec st_mtime_n st_mtimespec.tv_nsec; do
	CPPFLAGS="$ac_save_CPPFLAGS -DST_MTIM_NSEC=$ac_val"
	AC_COMPILE_IFELSE([AC_LANG_PROGRAM([#include <sys/types.h>
	#include <sys/stat.h>], [struct stat s; s.ST_MTIM_NSEC;])],
	[ac_cv_struct_st_mtim_nsec=$ac_val; break])
	done
	CPPFLAGS="$ac_save_CPPFLAGS"
	])
	if test $ac_cv_struct_st_mtim_nsec != no; then
	AC_DEFINE_UNQUOTED([ST_MTIM_NSEC], [$ac_cv_struct_st_mtim_nsec],
	[Define if struct stat contains a nanoseconds field])
	fi

However, it's possible that the library indeed supports s.ST_MTIM_NSEC but the filesystem driver it uses does not support sub-second timestamp, like ext2 without the extended timestamp. Therefore, something like --disable-nanotime in ./configure may be useful.

Not sure if the vim maintainers are willing to support this feature, but I can open a PR and work on this if desired :).

[chrisbra]

Thanks, yes I think the nanotime feature check doesn't really help here, as we are limited by whatever the underlying filesystem supports (like on old vfat). Since we don't really know on what filesystem the tests are run (and I think Vim shouldn't need to have this knowledge) I guess adding a :2sleep should be fine, together with a small comment why this is needed.

Thanks for working on this.

[LZhaoM]

Do you mean adding a 2sleep before checktime in WriteDiffFiles3 and WriteDiffFiles directly? If yes, I don't think it's a good idea. It will significantly increase the make test duration. You can see how many WriteDiffFiles there are in Test_diff_overlapped_diff_blocks_will_be_merged:

vim/src/testdir/test_diffmode.vim

Lines 2097 to 2300 in d6e3c90

	func Test_diff_overlapped_diff_blocks_will_be_merged()
	CheckScreendump
	let lines =<< trim END
	func DiffExprStub()
	let txt_in = readfile(v:fname_in)
	let txt_new = readfile(v:fname_new)
	if txt_in == ["line1"] && txt_new == ["line2"]
	call writefile(["1c1"], v:fname_out)
	elseif txt_in == readfile("Xdiin1") && txt_new == readfile("Xdinew1")
	call writefile(readfile("Xdiout1"), v:fname_out)
	elseif txt_in == readfile("Xdiin2") && txt_new == readfile("Xdinew2")
	call writefile(readfile("Xdiout2"), v:fname_out)
	endif
	endfunc
	END
	call writefile(lines, 'XdiffSetup', 'D')
	call WriteDiffFiles(0, [], [])
	let buf = RunVimInTerminal('-d -S XdiffSetup Xdifile1 Xdifile2', {})
	call term_sendkeys(buf, ":set autoread\<CR>\<c-w>w:set autoread\<CR>\<c-w>w")
	call WriteDiffFiles(buf, ["a", "b"], ["x", "x"])
	call writefile(["a", "b"], "Xdiin1")
	call writefile(["x", "x"], "Xdinew1")
	call writefile(["1c1", "2c2"], "Xdiout1")
	call term_sendkeys(buf, ":set diffexpr=DiffExprStub()\<CR>:")
	call VerifyBoth(buf, "Test_diff_overlapped_2.01", "")
	call term_sendkeys(buf, ":set diffexpr&\<CR>:")
	call WriteDiffFiles(buf, ["a", "b", "c"], ["x", "c"])
	call writefile(["a", "b", "c"], "Xdiin1")
	call writefile(["x", "c"], "Xdinew1")
	call writefile(["1c1", "2d1"], "Xdiout1")
	call term_sendkeys(buf, ":set diffexpr=DiffExprStub()\<CR>:")
	call VerifyBoth(buf, "Test_diff_overlapped_2.02", "")
	call term_sendkeys(buf, ":set diffexpr&\<CR>:")
	call WriteDiffFiles(buf, ["a", "c"], ["x", "x", "c"])
	call writefile(["a", "c"], "Xdiin1")
	call writefile(["x", "x", "c"], "Xdinew1")
	call writefile(["1c1", "1a2"], "Xdiout1")
	call term_sendkeys(buf, ":set diffexpr=DiffExprStub()\<CR>:")
	call VerifyBoth(buf, "Test_diff_overlapped_2.03", "")
	call term_sendkeys(buf, ":set diffexpr&\<CR>:")
	call StopVimInTerminal(buf)
	wincmd c
	call WriteDiffFiles3(0, [], [], [])
	let buf = RunVimInTerminal('-d -S XdiffSetup Xdifile1 Xdifile2 Xdifile3', {})
	call term_sendkeys(buf, ":set autoread\<CR>\<c-w>w:set autoread\<CR>\<c-w>w:set autoread\<CR>\<c-w>w")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["a", "x", "c"], ["y", "b", "c"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.01", "")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["a", "x", "c"], ["a", "y", "c"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.02", "")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["a", "x", "c"], ["a", "b", "y"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.03", "")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["a", "x", "c"], ["y", "y", "c"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.04", "")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["a", "x", "c"], ["a", "y", "y"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.05", "")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["a", "x", "c"], ["y", "y", "y"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.06", "")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["a", "x", "x"], ["y", "y", "c"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.07", "")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["x", "x", "c"], ["a", "y", "y"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.08", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["y", "y", "y", "d", "e"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.09", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["y", "y", "y", "y", "e"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.10", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["y", "y", "y", "y", "y"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.11", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["a", "y", "y", "d", "e"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.12", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["a", "y", "y", "y", "e"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.13", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["a", "y", "y", "y", "y"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.14", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["a", "b", "y", "d", "e"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.15", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["a", "b", "y", "y", "e"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.16", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["a", "b", "y", "y", "y"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.17", "")
	call WriteDiffFiles3(buf, ["a", "b"], ["x", "b"], ["y", "y"])
	call writefile(["a", "b"], "Xdiin1")
	call writefile(["x", "b"], "Xdinew1")
	call writefile(["1c1"], "Xdiout1")
	call writefile(["a", "b"], "Xdiin2")
	call writefile(["y", "y"], "Xdinew2")
	call writefile(["1c1", "2c2"], "Xdiout2")
	call term_sendkeys(buf, ":set diffexpr=DiffExprStub()\<CR>:")
	call VerifyInternal(buf, "Test_diff_overlapped_3.18", "")
	call term_sendkeys(buf, ":set diffexpr&\<CR>:")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d"], ["x", "b", "x", "d"], ["y", "y", "c", "d"])
	call writefile(["a", "b", "c", "d"], "Xdiin1")
	call writefile(["x", "b", "x", "d"], "Xdinew1")
	call writefile(["1c1", "3c3"], "Xdiout1")
	call writefile(["a", "b", "c", "d"], "Xdiin2")
	call writefile(["y", "y", "c", "d"], "Xdinew2")
	call writefile(["1c1", "2c2"], "Xdiout2")
	call term_sendkeys(buf, ":set diffexpr=DiffExprStub()\<CR>:")
	call VerifyInternal(buf, "Test_diff_overlapped_3.19", "")
	call term_sendkeys(buf, ":set diffexpr&\<CR>:")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d"], ["x", "b", "x", "d"], ["y", "y", "y", "d"])
	call writefile(["a", "b", "c", "d"], "Xdiin1")
	call writefile(["x", "b", "x", "d"], "Xdinew1")
	call writefile(["1c1", "3c3"], "Xdiout1")
	call writefile(["a", "b", "c", "d"], "Xdiin2")
	call writefile(["y", "y", "y", "d"], "Xdinew2")
	call writefile(["1c1", "2,3c2,3"], "Xdiout2")
	call term_sendkeys(buf, ":set diffexpr=DiffExprStub()\<CR>:")
	call VerifyInternal(buf, "Test_diff_overlapped_3.20", "")
	call term_sendkeys(buf, ":set diffexpr&\<CR>:")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d"], ["x", "b", "x", "d"], ["y", "y", "y", "y"])
	call writefile(["a", "b", "c", "d"], "Xdiin1")
	call writefile(["x", "b", "x", "d"], "Xdinew1")
	call writefile(["1c1", "3c3"], "Xdiout1")
	call writefile(["a", "b", "c", "d"], "Xdiin2")
	call writefile(["y", "y", "y", "y"], "Xdinew2")
	call writefile(["1c1", "2,4c2,4"], "Xdiout2")
	call term_sendkeys(buf, ":set diffexpr=DiffExprStub()\<CR>:")
	call VerifyInternal(buf, "Test_diff_overlapped_3.21", "")
	call term_sendkeys(buf, ":set diffexpr&\<CR>:")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["a", "x", "c"], ["b", "c"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.22", "")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["a", "x", "c"], ["c"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.23", "")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["a", "x", "c"], [])
	call VerifyBoth(buf, "Test_diff_overlapped_3.24", "")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["a", "x", "c"], ["a", "c"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.25", "")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["a", "x", "c"], ["a"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.26", "")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["a", "x", "c"], ["b"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.27", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["d", "e"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.28", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["e"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.29", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["a", "d", "e"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.30", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["a", "e"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.31", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["a"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.32", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["a", "b", "d", "e"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.33", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["a", "b", "e"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.34", "")
	call WriteDiffFiles3(buf, ["a", "b", "c", "d", "e"], ["a", "x", "c", "x", "e"], ["a", "b"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.35", "")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["a", "x", "c"], ["a", "y", "b", "c"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.36", "")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["a", "x", "c"], ["a", "b", "y", "c"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.37", "")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["d", "e"], ["b", "f"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.38", "")
	call WriteDiffFiles3(buf, ["a", "b", "c"], ["d", "e"], ["b"])
	call VerifyBoth(buf, "Test_diff_overlapped_3.39", "")
	call StopVimInTerminal(buf)
	endfunc

I tried adding 2sleep before checktime. This test will timeout:

vim/src/testdir/runtest.vim

Lines 246 to 252 in d6e3c90

	if has('timers')
	" No test should take longer than 45 seconds. If it takes longer we
	" assume we are stuck and need to break out.
	let test_timeout_timer =
	\ timer_start(RunningWithValgrind() ? 90000 : 45000, 'TestTimeout')
	let g:timeout_start = localtime()
	endif

And to be honest, now I'm quite unsure about if it's worth to support running make test on the old filesystems which don't support sub-second timestamp.

[LZhaoM]

BTW the solution I found of solving the Test_diff_overlapped_diff_blocks_will_be_merged timeout is splitting the test into three parts. There are some other tests that may require longer sleep time. I'm still investigating.

[chrisbra]

okay, thanks for further investigating. BTW: on what filesystem are you experiencing this issue? Perhaps we should indeed ignore this problem if this only affects "legacy" or rather unusual filesystems?

[LZhaoM]

ext2fs