Bug 5547: assertPred (the druntime part)

[kennytm]

See dlang/dmd#263.

Important: The pull request dlang/dmd#725 should be merged before pulling this.

Just provides the function __unittest_toString, which basically forwards to std.conv.to!string or tango.utils.Convert.to!string. If there is a third standard library (gasp) the corresponding code have to be added manually.

(BTW, is there a good way to decouple Phobos/Tango without rewriting to!string in druntime?)

requires: dmd git://github.com/kennytm/dmd.git bug7399-import-too-fatal

[MartinNowak]

This won't work out. It creates a circular dependency between druntime and phobos.

[kennytm]

@dawgfoto The only way to break the dependency is to put to!string into Druntime.

[alexrp]

What is Tango code even doing here?

[kennytm]

@alexrp : Read the description of the pull request!

[alexrp]

This is the runtime for D2. The Tango port for D2 works alongside Phobos. I don't see why you want to special case anything for Tango here.

[kennytm]

@alexrp : The Tango port for D2 works alongside Phobos does not mean Phobos must exist when Tango is installed.

[alexrp]

By this logic, we would still have to special case every single standard library people come up with here. I don't like this. The dependency on Phobos is already very controversial IMO.

[kennytm]

@alexrp : Yes if the 3rd standard library appears we need to write code for that, otherwise it will just return T.stringof. This has been explicitly written in the pull request description, and I am not going to change it, because this is the best solution with minimal change. You could request the repo owner to revert the commit.

[MartinNowak]

@dawgfoto The only way to break the dependency is to put to!string into Druntime.

Right, AFAIK we don't have any string formatting code in druntime or the compiler.
You'd need to!string or something similar in druntime to implement the feature.
Whether or not such functionality should be part of the runtime is debateable,
you could start a discussion in the newsgroup.

In the long term a no-magic library implementation of assertPred is cleaner and extensible.
(see datetime.d@L33457 for a start).

[alexrp]

I agree with @dawgfoto that we should avoid a dependency on Phobos. I don't really care how it's done (moving string conversion functionality to druntime or whatever), but druntime just shouldn't depend on Phobos. This is the runtime library, and it should work independently of any other D library (be it the 'standard' library or not).

[kennytm]

@alexrp @dawgfoto : Please notice that this do compile independently of Phobos/Tango, only that it returns T.stringof if neither exists. So the worst case is that you'll see

AssertError: float == float is unexpected

---

If you think Druntime forwarding to Phobos/Tango is bad, please devise a solution. I know there are other solutions, but they are no better than the current situation.

1. Introduce std.xxxx.assertPred — Already banned, don't even think about it. Please read http://d.puremagic.com/issues/show_bug.cgi?id=5547 and http://forum.dlang.org/thread/iirnib$28ig$1@digitalmars.com.
2. Introduce std.xxxx.assertPred, and let DMD rewrite assert to std.xxxx.assertPred — Which means the D compilers need to depend even more on Phobos besides std.math.pow.
3. Move std.conv.to and all its dependency from Phobos to Druntime.
4. Duplicate a small subset of std.conv.to!string to Druntime (just like rt.util.utf vs std.utf now).

[alexrp]

Please notice that this do compile independently of Phobos/Tango, only that it returns T.stringof if neither exists.

I understand, but I think the problem is with bootstrapping. If you're compiling the DMD tool chain on a system with no existing DMD tool chain, you're going to have to first compile druntime, then phobos, and then druntime again to get the full functionality out of this. That's how I understand it, at least. It's not the end of the world, but it does complicate a proper build.

Introduce std.xxxx.assertPred, and let DMD rewrite assert to std.xxxx.assertPred — Which means the D compilers need to depend even more on Phobos besides std.math.pow.

Well, DMD has a lot of other special cases for std.* already (std.stdio, a bunch of std.math intrinsics, etc). A rewrite might not necessarily be that bad.

Move std.conv.to and all its dependency from Phobos to Druntime.

Probably overkill. Is all of it really needed?

Duplicate a small subset of std.conv.to!string to Druntime (just like rt.util.utf vs std.utf now).

This I can get behind, but I must confess to not knowing how much code would have to be moved.

[alexrp]

One thing that also worries me is that by using to!string(), use of the GC is basically forced. But on the other hand, implementing it without a GC would be tiresome and probably not worth the effort.

[kennytm]

@alexrp :

I understand, but I think the problem is with bootstrapping. If you're compiling the DMD tool chain on a system with no existing DMD tool chain, you're going to have to first compile druntime, then phobos, and then druntime again to get the full functionality out of this. That's how I understand it, at least. It's not the end of the world, but it does complicate a proper build.

No you don't need to do that. The "dependency" is only in the source level. __unittest_toString is inside the object.di file which will not be compiled to anywhere.

Well, DMD has a lot of other special cases for std.* already (std.stdio, a bunch of std.math intrinsics, etc). A rewrite might not necessarily be that bad.

Only std.math.pow is an essential dependency (to support a ^^ b). The rest are:

1. Optional, e.g. DMD can recognize std.math.cos as a built-in (before core.math was introduced), but it doesn't mean std.math.cos must be present in the library; my Bug 5547 (assertPred) dmd#263 recognizes feqrel (not even std.math.feqrel), but it doesn't require std.math.feqrel to exist in the library.
2. Informational, e.g. the hint of std.conv.octal, the "did you mean" hints.

I don't see any dependency on std.stdio aside from the test cases.

Move std.conv.to and all its dependency from Phobos to Druntime.

Probably overkill. Is all of it really needed?

I don't know, but it seems there are other use cases of it: http://forum.dlang.org/thread/20120323175428.GA28704@quickfur.ath.cx

[alexrp]

No you don't need to do that. The "dependency" is only in the source level. __unittest_toString is inside the object.di file which will not be compiled to anywhere.

Hmm, I haven't looked at the DMD pull request yet, but I take it DMD just instantiates it then? If this is the case, what are your thoughts, @dawgfoto?

I don't see any dependency on std.stdio aside from the test cases.

I meant the "did you mean" hint. I know how/why DMD 'depends' on some things in Phobos (directly or indirectly), but my point was that having this extra dependency probably isn't a big deal, since assertPred is something you opt into and not an essential part of the language.

[MartinNowak]

just like rt.util.utf vs std.utf now

And now they're out of sync and rt.util.utf is slow while std.utf is fast.

I agree that one could definitely expect more from a modern language.

Some ideas for formatting:

• basic types can all be handled by sprintf
• arrays, vectors and structs are composed of basic types (loops and recursion)
• Objects have a toString method (structs too sometimes)

This would only be slightly more code than __unittest_toString.

Some more ideads:

• Make sure there is general support from Walter for dmd.
• Try to avoid magic/special constructs as much as possible.
• Instead of putting all the logic into the compiler we should use a rewrite.
  The point here is to have a thin, loosely coupled and extensible interface.
• A simple rewrite would be assert(a == 6) to object.assertPred!"=="(a, 6).
  There is definitely something to learn from the op*ary methods.
• We should do all formating in the runtime (no "6" string literals please).

Bugzilla 5547 - Improve assert to give information...

[MartinNowak]

No you don't need to do that. The "dependency" is only in the source level. __unittest_toString is inside the object.di file which will not be compiled to anywhere.

It will get compiled into druntime's unittests and they could no longer be run without phobos.

Not a single dependency on a phobos symbol must end up in the druntime library or we can no longer link programs.

[kennytm]

@dawgfoto :

Some ideas for formatting:

• basic types can all be handled by sprintf
• arrays, vectors and structs are composed of basic types (loops and recursion)
• Objects have a toString method (structs too sometimes)

This would only be slightly more code than __unittest_toString.

That is equivalent to my solution number 4, without spelling out to!string.

(Besides, writing some completely new code to perform something already solved, just to get rid of a dependency inside the same product? Smells like a modularization failure.)

Instead of putting all the logic into the compiler we should use a rewrite.
The point here is to have a thin, loosely coupled and extensible interface.

A simple rewrite would be assert(a == 6) to object.assertPred!"=="(a, 6).
There is definitely something to learn from the op*ary methods.

The current implementation in DMD is already a rewrite, as described in https://github.com/D-Programming-Language/dmd/pull/263/files#L0R47.

We should do all formating in the runtime (no "6" string literals please).

I don't see any problem with it.

---

It will get compiled into druntime's unittests and they could no longer be run without phobos.

No it does not. Please read the whole commit. I have defined a druntime_unittest version which turns the function to simply return T.stringof.

[MartinNowak]

(Besides, writing some completely new code to perform something already solved, just to get rid of a dependency inside the same product? Smells like a modularization failure.)

Different product.
If you're proposing that enriched assert information needs language/compiler integration
then it belongs in the compiler support library.

Some smallish printf machinery would suffice.

The current implementation in DMD is already a rewrite, as described in https://github.com/D-Programming-Language/dmd/pull/263/files#L0R47.

I've seen that, it's quite nice but biased towards doing things in the compiler. You know the limitations of previous attempts (AA, Complex or Array).

The problem could be nicely solved with macros, but currently it requires a mixin and stringified expressions.

template Assert(string exp)
{
    enum Assert = "assert("~exp~", \""~exp~"\");";
}

void main()
{
    int i;
    mixin(Assert!"i == 0");
    mixin(Assert!"i != 0");
}

[kennytm]

@dawgfoto

I've seen that, it's quite nice but biased towards doing things in the compiler. You know the limitations of previous attempts (AA, Complex or Array).

The AA, complex and array type do not need compiler compiler support. The AA literal, complex literal, and array literal syntax do. The same in 5547 here: the assertion function do not need compiler support, but the assertion syntax do. All the assertpred.c does is to extract the variables from the AST which cannot be done outside of the compiler. Even your assert(i == 5) → assertPred!"=="(i, 5) requires AST transformation which cannot be done outside of the compiler.

The problem could be nicely solved with macros, but currently it requires a mixin and stringified expressions.

Your solution solved nothing. The point of the request is to make assert(a == b); to print the values of a and b when the assertion failed in the unit test. This requires a D parser in CTFE.

[kennytm]

@dawgfoto

And back to the formatting again,

Some smallish printf machinery would suffice.

I doubt that. It needs to support printing anything that may appear in the assertion expression, like

1. Basic types like int and float and char and wstring and typeof(null) and pointers
2. Composite types like array and AAs
3. Custom types (struct and class), and provide something useful even when they do not define .toString()
4. Enums
5. (Legacy types like cdouble.)

The effort would be like duplicating half of toImpl!string and recreating the std.utf/rt.util.utf situation again.

Someone could of course step up and write such sprintf function and replace this to use sprintf instead, but as I said before I'm not going to change this since it has been sufficient for me.

[MartinNowak]

Your solution solved nothing.

That was only loud thinking. This issue pops up regularly and that we cannot solve it within the language is a deficiency.

Adding tailor-made magic for assert breaks composability and language syntax rules.

void assertEqual(R1, R2)(R1 r1, R2 r2)
{
   for (;!r1.empty && !r2.empty; r1.popFront(), r2.popFront())
       if (r1.front != r2.front)
           throw new AssertError("r1 == r2");
}

What am I supposed to do here?
From where do I get the magic message?

We already made the bad decision to make assert a language construct.

// none of these work
auto passert = verboseTests ? &verboseAssert : &assert;
alias assert foo;

Having it behave differently in unittest blocks is fairly odd when it should behave differently during unittests. This means it won't work for testing contracts.

[MartinNowak]

And again, string mapping, concatenation and formatting shouldn't be hardwired in the compiler.
https://github.com/D-Programming-Language/dmd/pull/263/files#L0R251

It's simple enough to translate that information into a template-function call.

[kennytm]

@dawgfoto If you have comments on dlang/dmd#263, can you please comment over there? Or put an inline note (Click the "+" speech bubble on the left of the line)? This pull request is about Druntime and it has been closed, if we keep talking here, there will likely be almost no one noticing these opinions except us.

---

That was only loud thinking. This issue pops up regularly and that we cannot solve it within the language is a deficiency.

That's not loud thinking, that's a fact. Your solution in #41 (comment) solved nothing. Who cares about the message "AssertError: i == 5", we already know the file and line number and the stack trace. What is valuable is the value of i which your solution in that comment did not address. I did not say modifying the code to generate something like assertPred!"=="(i, 5) doesn't solve anything.

This issue pops up regularly and that we cannot solve it within the language is a deficiency.

Not sure what issue do you mean. If you mean AST transformation, I agree totally.

What am I supposed to do here?

Write the code properly using assert().

void assertEqual(R1, R2)(R1 r1, R2 r2)
{
   for (;!r1.empty && !r2.empty; r1.popFront(), r2.popFront())
       assert(r1.front == r2.front);
}

If you want to throw an AssertError you are on your own. (Just like when you use the GC function directly, or an ASM block... if you use low-level construct, don't expect the compiler to help you.)

This means it won't work for testing contracts.

It can be changed to work in testing contracts. (But apparently contracts is such a failure that no one adds this requirement in 5547 ^^)