Unstable TestRDocGeneratorJsonIndex#test_generate test

[junaruga]

I am seeing the following failure that happens randomly on Travis Ubuntu jammy ppc64le environment in ruby/ruby. The latest failure is at the relatively recent commit ruby/ruby@1d51e4c .

https://app.travis-ci.com/github/ruby/ruby/jobs/612435804#L2936

  1) Failure:
TestRDocGeneratorJsonIndex#test_generate [/home/travis/build/ruby/ruby/test/rdoc/test_rdoc_generator_json_index.rb:108]:
.js files should be the same timestamp of original

The test is built and executed with the following steps.

+ export NPROC="$(nproc)"
+ export JOBS=-j$((1+${NPROC}))
+ export RUBY_TESTOPTS="$JOBS -q --tty=no"
+ export optflags=-O1
+ export debugflags=-ggdb3

+ export CC=gcc-11
+ gcc-11 --version
gcc-11 (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ ../tool/travis_wait.sh $SETARCH make -s test-all RUBYOPT="-w"
...
Run options: 
  --seed=2200
  "--ruby=./miniruby -I../lib -I. -I.ext/common  ../tool/runruby.rb --extout=.ext  -- --disable-gems"
  --excludes-dir=../test/.excludes
  --name=!/memory_leak/
  -j3
  -q
  --tty=no

[junaruga]

Another error log after removing the optflags=-O1 in .travis.yml. So, the optflags=-O1 is not criteria to make this issue happen.
https://app.travis-ci.com/github/ruby/ruby/jobs/612440144

[junaruga]

I was able to capture the failing test, and compared values.

https://app.travis-ci.com/github/ruby/ruby/jobs/612446128#L2936

  1) Failure:
TestRDocGeneratorJsonIndex#test_generate [/home/travis/build/ruby/ruby/test/rdoc/test_rdoc_generator_json_index.rb:108]:
.js files should be the same timestamp of original.
<"2023-10-30 17:17:00.869180353 +0000"> expected but was
<"2023-10-30 17:17:00 +0000">.

[junaruga]

I just tested the following script on RubyCI ppc64le server.

$ cat test.rb 
require 'pathname'

f = Pathname('/tmp/a.txt')
p f.mtime.inspect

$ touch /tmp/a.txt
$ ruby test.rb 
"2023-10-30 18:30:57.305517101 +0000"

[junaruga]

I was able to capture the failing test, and compared values.

https://app.travis-ci.com/github/ruby/ruby/jobs/612446128#L2936

  1) Failure:
TestRDocGeneratorJsonIndex#test_generate [/home/travis/build/ruby/ruby/test/rdoc/test_rdoc_generator_json_index.rb:108]:
.js files should be the same timestamp of original.
<"2023-10-30 17:17:00.869180353 +0000"> expected but was
<"2023-10-30 17:17:00 +0000">.

Here is just another failing case with the values.
https://app.travis-ci.com/github/ruby/ruby/jobs/612583543#L2936

  1) Failure:
TestRDocGeneratorJsonIndex#test_generate [/home/travis/build/ruby/ruby/test/rdoc/test_rdoc_generator_json_index.rb:108]:
.js files should be the same timestamp of original.
<"2023-11-01 15:38:55.065902574 +0000"> expected but was
<"2023-11-01 15:38:55 +0000">.

[junaruga]

I tried to reproduce this issue on RubyCI ppc64le Ubuntu jammy server for the Ruby ruby/ruby@ad4f973. However, I was not able to reproduce the issue.

test_repeat.sh

#!/bin/bash

set -eux

for i in $(seq 100); do
  echo "=== ${i}"
  RUBY_TESTOPTS="-j8 -q" make test-all TESTS="test/rdoc/test_rdoc_generator_json_index.rb"
done

$ nproc
8

$ nohup /path/to/test_repeat.sh >& test.log &