[Question] Is buck caching the test results?

[acecilia]

Hello:

I am using buck and also bazel to run the same set of tests (in here). When running them with bazel, the output is as follows:

INFO: Analyzed 31 targets (0 packages loaded, 0 targets configured).
INFO: Found 25 targets and 6 test targets...
INFO: Elapsed time: 0.151s, Critical Path: 0.01s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
//Libraries/ObjcModule:ObjcModuleTests                          (cached) PASSED in 21.2s
//Libraries/SwiftApp:SwiftAppAppTests                           (cached) PASSED in 26.7s
//Libraries/SwiftApp:SwiftAppTests                              (cached) PASSED in 16.7s
//Libraries/SwiftModule:SwiftModuleAppTests                     (cached) PASSED in 27.9s
//Libraries/SwiftModule:SwiftModuleTests                        (cached) PASSED in 21.1s
//Libraries/SwiftModuleWithPrebuiltDependency:SwiftModuleWithPrebuiltDependencyTests (cached) PASSED in 18.2s

As you can see, it clearly indicates which of the test results were cached and which of them did run.

With buck I get the following output:

Not using buckd because watchman isn't installed.
Parsing buck files: finished in 0.5 sec
Building: finished in 0.8 sec (100%) 40/40 jobs, 0 updated
  Total time: 1.5 sec
Testing: finished in 12.2 sec (8 PASS/0 FAIL)
RESULTS FOR //Libraries/ObjcModule:ObjcModuleObjcTests //Libraries/ObjcModule:ObjcModuleTests //Libraries/SwiftApp:SwiftAppAppTests //Libraries/SwiftApp:SwiftAppTests //Libraries/SwiftModule:SwiftModuleAppTests //Libraries/SwiftModule:SwiftModuleTests //Libraries/SwiftModuleWithPrebuiltDependency:SwiftModuleWithPrebuiltDependencyTests
PASS    <100ms  1 Passed   0 Skipped   0 Failed   Tests
PASS    <100ms  2 Passed   0 Skipped   0 Failed   Tests
PASS    <100ms  2 Passed   0 Skipped   0 Failed   Tests
PASS    <100ms  1 Passed   0 Skipped   0 Failed   Tests
PASS    <100ms  1 Passed   0 Skipped   0 Failed   Tests
PASS    <100ms  1 Passed   0 Skipped   0 Failed   Tests
TESTS PASSED

Is buck caching the test results? If yes, how can I see which of those test results were cached, and which of them did actually run?

I see here that buck used to have some options related to test result cache back in 2016, but they were removed in 2017 here. I am not sure why, as there is no changelog neither release notes that I can refer to. Any idea?

Thanks!

[gnuthor]

Buck's test caching was removed in 2017 just as you mentioned, allegedly because (quote) "People wrote really bad tests that weren't deterministic, and didn't realize that their tests weren't actually running. afaik we removed it because of this, and we should run all tests every time".|

See also here

There's the possibility that specific tests could implement their own caching of course.

[acecilia]

Thanks @gnuthor, the link is very helpful.

Such a pity, I also think as commented in 0fc321f#commitcomment-22227138 that test result caching is one of the main reasons to use a build system like buck/bazel.