Fix Issue 18419 - make all Phobos unittests version(StdUnittest) #6159
Conversation
Thanks for your pull request, @n8sh! We are looking forward to reviewing it, and you should be hearing from a maintainer soon. Some tips to help speed things up:
Bear in mind that large or tricky changes may require multiple rounds of review and revision. Please see CONTRIBUTING.md for more information. Bugzilla references
|
12f139f
to
aea34e6
IMHO, if we need to do this, there is something very wrong about how D handles unit tests. |
Indeed. This change would fix a very real problem, but ... perhaps this is better fixed elsewhere. |
This won't interfere with someone making unittest-related improvements to other parts of the D toolchain. It may also take some of the pressure off of whoever is working on that, so they can take their time to get the design right instead of feeling like they have to rush something out the door. |
For reference, the discussion is here: #6142
Ok, alternatives:
Better ideas? @FeepingCreature - any chance we can add a unittest for your |
so now when you build the docs with dmd, you need -version=StdUnittest yes? but i say don't let the perfect solution be the enemy of a workable stopgap. we can always fix root cause later and revert this at that time |
edit: The important part: Yes, this pr fixes it, and another issue beside.
When you say "your project", I think you still underestimate the scale of the problem. This is what it takes on master, to cause a failing build:
Basically every phobos module is currently broken with A feasible testcase would just consist of a single file that imports every phobos module, built with -unittest -deps. Though be warned: this may consume a lot of memory to try, if it doesn't error out immediately. If you have less than 12GB free, hold your fingers on ctrl-C and keep a watchful eye on edit:
I like regex! (Too bad importing std.regex increases compiletime by 12 seconds in release builds...) Edit: tried that command line with this pr, and it seems to fix the issue, though it results in a possibly unrelated linker error. |
I ran your script in the master branch, except without
|
You need to use the freshly compiled Phobos library when doing anything with master - so yes the "error" is unrelated. |
Man. I was just forced to do this sort of thing with dxml, because folks where getting weird linker errors when they built the unit tests for their own projects which used dxml. For some reason, not all of the symbols used by dxml's unit tests were available - though dub seems to take this a step stupider than dmd does on its own. dub seems to build the actual unittest target for all dependencies not just have problems with the imports being affected by the -unittest flag. This whole situation is a mess. |
Agreed and it looks like we need to make a decision pretty soon (the master branch off will happen is supposed to happen in two days). |
Let's get an executive decision on this from @andralex and move on. |
As a simple consideration, any change that adds an adornment to 4000+ lines of code in a 300 KLOC project should be approached with suspicion that there probably are 10 wrong lines of code someplace else. It's nice to see that other commenters noticed that as well. Neither the bug nor the PR lists the problem that this change is addressing, so I'll rely on the info in #6142 and links from there. Far as I understand:
There'd be a long discussion, but in short:
|
That wasn't what I understood the main problem to be. My understanding was that Phobos unittests participate in building user code when To demonstrate: $ cat test.d
import std.regex;
$ time dmd -c test.d
dmd -c test.d 0.21s user 0.01s system 98% cpu 0.226 total
$ time dmd -c -unittest test.d
dmd -c -unittest test.d 0.76s user 0.10s system 99% cpu 0.863 total |
A benefit of using |
On the whole, I agree that Please let's not confuse this issue with template unittests, even though the two are related. Your argument that template unittests being run in conjunction with user types is a good thing, is actually misrepresenting the real situation. Yes, in the cases where the unittest actually tests the template with the given user types, this is something beneficial. However, the majority of unittests in Phobos are of the kind:
Most of the time, such unittests are put inside the template because of the ddoc'd unittest feature: it automatically turns into a code example attached to the template's docs, which is a wonderful thing, because that ensures that code examples in the docs are actually tested by the autotester and verified to be runnable (and have correct results). However, since this is inside a template, that means the same unittest, testing the same type of test data, is instantiated n times, where n is the number of different types user code might call it with. That's And this isn't even the issue we're trying to address here. The issue here is that some Phobos unittests are very expensive to run, like those in And this goes beyond just Phobos alone. The way unittests are currently implemented, the same behaviour happens for all third party libraries that user code may import. If some library author A ships a D library L that contains unittests, then every single time some downstream user U imports L and compiles his code with tl;dr: there needs to be a way for upstream library authors (including Phobos) to make it so that users compiling with |
Specifically, the problem I encountered is that Phobos unittests participate in user code when Imports in unittests in Phobos would still run into the |
@FeepingCreature can the ddoc unittests be versioned with |
@CyberShadow I agree the increase of build time in unittests is unpleasant. It seems the right long-term solution to that is making the |
I agree. I think we need to experiment with making |
Actually, I already have an idea that's quite simple (barring unforeseen side-effects), that should address most, if not all, of the concerns raised here: Basically, change the compiler so that Presumably, any unittests that user code might be concerned about would be contained in a module that eventually would be passed to the compiler on the command-line, so they will end up being instantiated somewhere. Third-party libraries (or Phobos itself) would be separately compiled, and presumably distributed as binaries, so presumably they won't end up on the command-line (at least the .d files won't), so library-specific unittests will be automatically left out. Perhaps I should reserve some time to write up a DIP about this. |
@quickfur Nice idea, though it won't work for e.g. |
I think the best option is to revert StdUnittest. I don't think adding in this cruft/workaround is a viable alternative to fixing the compiler in this area. I'm going to go head and close this, and open a new PR on stable that does that. I think this conversation has been stalled for too long, and we need to fix this before we ship code with StdUnittest in it. |
This is the purpose for which
version(StdUnittest)
was added. The pain is one-time, the benefit is forever. It is also a good practice for other library authors to copy.