[phobos] make unittest should not rerun tests unnecessarily

[dlangBugzillaToGithub]

nick (@ntrel) reported this on 2024-06-03T16:09:58Z

Transfered from https://issues.dlang.org/show_bug.cgi?id=24584

CC List

• kinke

Description

Currently it seems to run every test, regardless of whether relevant Phobos dependencies have changed. This makes it a very slow process to find & fix all errors when introducing a dmd error.

Alternatively, having some way of running all tests despite any failing tests would be useful.

[dlangBugzillaToGithub]

nick (@ntrel) commented on 2024-06-12T15:55:12Z

> running all tests despite any failing tests would be useful

Looks like Gnu make has a -k option for that:
https://www.gnu.org/software/make/manual/html_node/Options-Summary.html#index-_002d_002dkeep_002dgoing-2

Not tested. But regardless it would be useful if there was a way to run an individual module's unittests rather than have to rerun the ones that already succeeded.

[dlangBugzillaToGithub]

kinke commented on 2024-06-14T08:58:22Z

(In reply to Nick Treleaven from comment #1)
> But regardless it would be useful if there was a way to run an
> individual module's unittests rather than have to rerun the ones that
> already succeeded.

There are multiple existing options for running the unittests of a single module or package, all based on https://github.com/dlang/phobos/blob/ad9f87d1b816783bd8c13f461490e68f7fcae20a/Makefile#L431-L448:

```
$ make -j$(nproc) std/algorithm/mutation.test # single module
$ make -j$(nproc) std/algorithm.test # whole package
$ make -j$(nproc) unittest/std/algorithm/mutation.run # single module
```

That said, *running* (not building) all Phobos unittests on my laptop takes about 6 (release) / 7.5 (debug) seconds, using current master.

[dlangBugzillaToGithub]

nick (@ntrel) commented on 2024-06-14T12:09:23Z

> There are multiple existing options for running the unittests of a single module or package

Oh great, thanks. I completely missed that but I see now that .test is mentioned at the top of the Makefile.

> *running* (not building) all Phobos unittests on my laptop takes about 6 (release) / 7.5 (debug) seconds

I have an old PC (I don't use -j as I only have 2 cores and 1 I keep for other things):

time make unittest
...
make[1]: Leaving directory '/home/nick/git/phobos'

real	16m57.602s
user	15m21.911s
sys	1m0.822s

[dlangBugzillaToGithub]

nick (@ntrel) commented on 2024-06-14T12:30:37Z

(In reply to kinke from comment #2)
> ```
> $ make -j$(nproc) std/algorithm/mutation.test # single module
> $ make -j$(nproc) std/algorithm.test # whole package
> $ make -j$(nproc) unittest/std/algorithm/mutation.run # single module
> ```

Pull to update Makefile comment:
https://github.com/dlang/phobos/pull/9014

[dlangBugzillaToGithub]

kinke commented on 2024-06-14T13:27:58Z

(In reply to Nick Treleaven from comment #3)
> I have an old PC (I don't use -j as I only have 2 cores and 1 I keep for
> other things):
> 
> time make unittest
> ...
> make[1]: Leaving directory '/home/nick/git/phobos'
> 
> real	16m57.602s
> user	15m21.911s
> sys	1m0.822s

Oh wow. Then the issue most likely isn't about *running* a subset of unittests only, but improving incremental builds of the unittest runners. Currently, changing a single Phobos source module leads to a complete rebuild (recompiling all Phobos unittest object files): https://github.com/dlang/phobos/blob/ad9f87d1b816783bd8c13f461490e68f7fcae20a/Makefile#L400-L401

The Makefiles don't exploit the compiler `-makedeps` switch yet. That would allow only recompiling object files which are dirty (directly or indirectly importing a changed Phobos module).

[thewilsonator]

@ntrel #9014 was merged. is this issue still affecting you?