[JS][CI] Timeout on macos in GitHub Actions

[abetomo]

Describe the bug, including details regarding any error messages, version, and platform.

https://github.com/apache/arrow/actions/runs/5097749959/jobs/9164462790

macos in name: NodeJS times out.

AMD64 macOS 11 NodeJS 18
The job running on runner GitHub Actions 25 has exceeded the maximum execution time of 60 minutes.

Component(s)

JavaScript

[abetomo]

Should we speed up the task?
Increase timeout values?

[kou]

Should we speed up the task?

It's better.

@domoritz Do you have any idea to speed up gulp test?

[domoritz]

Not really. It’s would need significant work to make it faster.

[abetomo]

take

[abetomo]

I will try to speed up the task.
If you can think of any ideas on how to speed up tasks, please let me know.

[abetomo]

There did not seem to be any particularly slow tests.
How about a way to change the way the test is run?

1. Use --shard option of jest
   • https://jestjs.io/docs/next/cli#--shard
   • For example: GH-36154: [JS][CI]Split and run tests in jest's --shard #36313
2. Use @swc/jest instead of ts-jest
   • https://github.com/swc-project/jest
   • It could be about 1.5 times faster when there is no cache of jest
3. Keep a cache of jest in CI

[kou]

For example: #36313

Oh, each sharded job takes about 40 minutes...

https://github.com/abetomo/arrow/actions/runs/5378070925

(Non sharded jobs for Linux/Windows takes about 40 minutes.)

Keep a cache of jest in CI

Is it easy to implement?

[abetomo]

Setting up a cache is easy and I would try it.

[kou]

I tried running tests on arm64 macOS but yarn build (especially node node_modules/gulp-cli/bin/gulp.js compile:esnext:umd) was failed:

[10:10:34] Using gulpfile ~/work/cpp/arrow/js/gulpfile.js
[10:10:34] Starting 'compile:esnext:umd'...
[10:10:38] 'compile:esnext:umd' errored after 3.77 s
[10:10:38] Error: Error: spawn Unknown system error -86
    at ChildProcess.spawn (node:internal/child_process:420:11)
    at spawn (node:child_process:757:9)
    at Compiler.run (/Users/kou/work/cpp/arrow/js/node_modules/google-closure-compiler/lib/node/closure-compiler.js:76:26)
    at CompilationStream._flush (/Users/kou/work/cpp/arrow/js/node_modules/google-closure-compiler/lib/gulp/index.js:163:40)
    at CompilationStream.final [as _final] (node:internal/streams/transform:132:10)
    at callFinal (node:internal/streams/writable:707:12)
    at prefinish (node:internal/streams/writable:719:7)
    at finishMaybe (node:internal/streams/writable:729:5)
    at Writable.end (node:internal/streams/writable:641:5)
    at DestroyableTransform.endFn (node:internal/streams/pipeline:424:11) {
  errno: -86,
  code: 'Unknown system error -86',
  syscall: 'spawn'
}

    at formatError (/Users/kou/work/cpp/arrow/js/node_modules/gulp-cli/lib/versioned/^4.0.0/format-error.js:21:10)
    at Gulp.<anonymous> (/Users/kou/work/cpp/arrow/js/node_modules/gulp-cli/lib/versioned/^4.0.0/log/events.js:33:15)
    at Gulp.emit (node:events:523:35)
    at Gulp.emit (node:domain:489:12)
    at Object.error (/Users/kou/work/cpp/arrow/js/node_modules/undertaker/lib/helpers/createExtensions.js:61:10)
    at handler (/Users/kou/work/cpp/arrow/js/node_modules/now-and-later/lib/map.js:50:14)
    at f (/Users/kou/work/cpp/arrow/js/node_modules/once/once.js:25:25)
    at f (/Users/kou/work/cpp/arrow/js/node_modules/once/once.js:25:25)
    at tryCatch (/Users/kou/work/cpp/arrow/js/node_modules/bach/node_modules/async-done/index.js:24:15)
    at done (/Users/kou/work/cpp/arrow/js/node_modules/bach/node_modules/async-done/index.js:40:12)

It seems that node_modules/google-closure-compiler-osx/compiler is only for x86_64:

$ file node_modules/google-closure-compiler-osx/compiler      
node_modules/google-closure-compiler-osx/compiler: Mach-O 64-bit executable x86_64

How did you run yarn build? (Are you using arm64 macOS?)

[abetomo]

Store cache of jest in CI.
abetomo@d5cd02d

• I increased timeout-minutes in macos because it takes longer with no cache.
• key is worth considering.

Time taken to test.

• No cache
  • https://github.com/abetomo/arrow/actions/runs/5395795864/jobs/9798660784
  • macos Test: 58m
• Cached 1
  • https://github.com/abetomo/arrow/actions/runs/5395795864/jobs/9799811556
  • macos Test: 41m
• Cached 2
  • https://github.com/abetomo/arrow/actions/runs/5395795864/jobs/9800640566
  • macos Test: 39m
• Cached 3
  • https://github.com/abetomo/arrow/actions/runs/5395795864/jobs/9801397138
  • macos Test: 47m

[abetomo]

How did you run yarn build? (Are you using arm64 macOS?)

I was running on an x86_64 Mac and yarn build completed successfully.

[kou]

It seems working.
Could you open a pull request for this?

• Could you use js-jest-... for key?
• Do we need restore-keys: js-jest-${{ runner.os }}-${{ matrix.node-version }}- too?

[abetomo]

Thanks for the comment.
I have opened a PR.
I set it up because it is better to use restore-keys.