Issue 5481 - Support deprecated("message")

[yebblies]

This patch adds two new forms of deprecation:
deprecated("message") and
deprecated("message", soft)

The first behaves the same as the old deprecated, but also prints a message after the standard error.
The second instead issues a warning.

This patch is backwards compatible with the current deprecation system.

http://d.puremagic.com/issues/show_bug.cgi?id=5481

@jmdavis I know this isn't exactly what you wanted. It can easily be changed if there's any sort of agreement. (@andralex ?)

[jmdavis]

That's essentially what I'm looking for but not quite. The main question is whether soft deprecation should be a warning or just always printed when the symbol is used. We've been treating it as "always printed," but some folks have found that to be annoying.

For anyone who ignores warnings, soft deprecation as done here doesn't add anything, and they still get hit with the hard deprecation out of the blue if they haven't read the documentation or changelog. And for anyone who always uses -w, they effectively get hit with hard deprecation immediately. It only works as "soft" when you compile with -wi. So, I'd be inclined to have "soft" deprecation always print the message when the symbol is used rather than making it a warning. Otherwise, the usefulness of the soft deprecation is limited.

Certainly, having a message with deprecated is a huge improvement, and soft deprecation as done here is better than nothing, but I fear that it's not much better than nothing, since it only works for the folks compiling with -wi. But it would make the folks who don't like the "scheduled for deprecation" messages happy. So, it's definitely debatable.

If you want more feedback on the changes though, you're going to have to post about it in the newsgroup. Only the dmd devs are likely to notice a pull request to dmd. I only noticed it because I was CCed on the enhancement request. Andrei probably won't notice it at all.

[yebblies]

I don't think it's acceptable to have soft deprecation message always print, with no way to disable them. When this was last brought up on the newsgroup, many people said the same thing.

Given that, there are a few ways to do it.

1. Make soft deprecation messages warnings
2. Make soft deprecation messages always print and add a switch to disable them
3. Make soft deprecation messages print when warnings are enabled

I think 1 is the best, 2 requires a new compiler switch and is probably overkill, and 3 introduces a new kind of 'non-fatal' warning which is rather against the spirit of D's warnings. Is there another way you can think of that can keep sides happy?

Thanks for the feedback. I'm asking here first, and you particularly, because phobos is the major user of deprecated and you seem to be in charge of managing deprecation in phobos. I do want to try and reach an agreement before taking this to the newsgroup again.

I was under the impression @ tagging github users would send them a message. I'll post a link on the phobos list.

[jmdavis]

I fully understand people not wanting to be flooded with messages, and if too many get printed, they get ignored anyway. However, the problem with making it a warning is that the only case that it really does anything is when you compile with -wi. With -w, it's the same has hard deprecation, and without either, you never see soft deprecation messages at all. Also, it potentially means that soft deprecation is stronger than hard deprecation in that there's no way to turn off the soft deprecation warnings if you compile with -wi or -w, whereas you can turn off the complaints about deprecation with -d. And overloading -d to do that with soft deprecation as well isn't necessarily a good idea, because then they'll be turning of the hard deprecation messages/errors as well.

The ideal situation probably be something like #2, since then you actually see the messages, but then if people get annoyed with them, they'll just use the new flag to disable them, and they won't do any good anyway. So, it's not entirely ideal. However, it does make soft deprecation act more or less like hard deprecation except that you only get a message instead of an error. So, that's good.

#3 goes against how warnings normally work with dmd, so we probably don't want to do that.

I think that the reality of the matter is that you're never going to make everyone happy, so the question is how best to get the message across that programmers are going to need to change their code and yet not annoy them in the process. #2 definitely seems like the best route for that to me, but it is further complicating things by adding yet another compiler flag. So, while #2 might be the best, we may want to go with #1 just to simplify things - especially since in the long run, deprecation in Phobos should be rare.

[mrmonday]

See also: #248 - I think this has some relevance to this discussion.

[CyberShadow]

Make soft deprecation messages always print and add a switch to disable them

2 requires a new compiler switch and is probably overkill

I'm surprised no one suggested a version identifier for this (e.g. -version=SilenceSoftDeprec).

[kennytm]

How about to have 3 versions, deprecated("...", error) (default), deprecated("...", warn) and deprecated("...", msg)?

[yebblies]

@CyberShadow: Isn't that basically the same as adding a new compiler switch, but by creating a new abuse of version syntax? ie. Causes the same complication to the compiler's interface as a new switch, except worse?

@kennytm: That would still require a way to silence deprecation messages, and I'm not sure adding an extra stage of deprecation is necessary.

@jmdavis:
I really think warnings are the way to go here. As for not being able to turn off the warnings when compiling with -wi, there is a way - don't use -wi. The way I see it, -wi basically means 'complain about things I may or may not care about'. I really don't think dmd should be constantly spitting out messages that essentially say 'One day your code won't compile without -d, then months later it won't compile at all'. I do see the need to notify people of impending deprecations, but I feel the proper place for this is in the changelog, not on my console every time I compile. Informational messages (and warnings) are great in IDEs, where they can be shown out of the way, but I find they mostly just get in the way when using the console.
I really think an opt-in solution is the best compromise we're going to get for scheduled-to-be-deprecated messages

[CyberShadow]

@CyberShadow: Isn't that basically the same as adding a new compiler switch, but by creating a new abuse of version syntax? ie. Causes the same complication to the compiler's interface as a new switch, except worse?

I meant in the context of implementing soft deprecation as a template. Currently, soft deprecation in templates unconditionally issues a pragma message. A -version to disable the message could resolve that complaint. I guess that direction is obsoleted by this pull request, though, so never mind.

[yebblies]

I meant in the context of implementing soft deprecation as a template. Currently, soft deprecation in templates unconditionally issues a pragma message. A -version to disable the message could resolve that complaint. I guess that direction is obsoleted by this pull request, though, so never mind.

Ah, ok. That might be worth doing if this pull request (or another solution) isn't accepted by the next release.

[jmdavis]

Okay. Assuming that when compiling with -w, soft deprecation is identical to hard deprecation, then this is probably the best way to go. It's not exactly ideal, but I don't think that we're really going to have a truly satisfactory solution here. I'd really like it to always give a message unless the programmer disables it, but there are a number of people who just find that to be too annoying. And since in the long run, deprecation in Phobos should be relatively rare, making deprecation much more complicated than this is probably overkill. Really, the only reason that we need soft deprecation on top of hard deprecation is because Phobos is used by pretty much everyone using D, so immediately hard deprecating something is too disruptive. Most smaller projects would be just fine with only hard deprecation (though the addition of the message is a huge improvement regardless).

So, IIUC, what this patch does is

1. Make it so that if you include a message with deprecated on a symbol, then that message is printed when the programmer gets a deprecation error for that symbol.
2. Soft deprecation is added on top of hard deprecation. It is identical to hard deprecation except for the fact that it is a warning instead of an error. So, it acts like all other warnings with regards to -wi and -w except that just like hard deprecation errors, soft deprecation warnings are disabled with -d.

Assuming that that's what it does, and I haven't misunderstood, then I can accept that solution. It's not quite what I'd like, but it works well enough considering that deprecation should be fairly rare.

[donc]

Soft deprecation is added on top of hard deprecation. It is identical to hard deprecation except for the fact that it is a warning instead of an error. So, it acts like all other warnings with regards to -wi and -w except that just like hard deprecation errors, soft deprecation warnings are disabled with -d.

I may have misunderstood something here, but it seems this whole approach is one step too soft. We currently have a big problem: when something has been deprecated and has been compiled with -d, then when it is removed, it just fails to compile -- and it doesn't give you a hint about what to do. This has been happening with things in the test suite, for example. We need a deprecation which is harder than normal: it must generate an error even when compiled with -d. (A 'your time is up' error). Note that once you start compiling with -d, if anything else you're using gets deprecated, you won't be informed at all. So, we've got these "scheduled for deprecation" messages, so that people don't have to do the dangerous activity of using -d.
I think the solution is to make the existing deprecation system work properly, by adding a HARD deprecation, rather than to formalize the workaround by adding a SOFT deprecation.

[jmdavis]

Well, the point of soft deprecation as generally discussed is to inform users that a symbol is going to be deprecated and that they're going to have to have to change their code while not breaking their code (which is why making it a warning instead of just a message isn't altogether ideal - anyone compiling with -w (as I'd argue everyone should be doing) has their code broken immediately). The point is to give them time to fix their code before they would have to build with -d or have their code not compile.

Hard deprecation as has been generally discussed is normal deprecation. And as that currently works, if you use -d, it stops warning you entirely up until the symbol is gone, and you're code won't compile. At that point, you also get no message as to what replaced the missing symbol, so it does become a problem (though normally code really shouldn't be being built with -d; Phobos is a bit of an exception given that it includes stuff which has been deprecated, but I don't know why the test suite would be using -d). However, if you were to add a harder level of deprecation, then the symbol couldn't even be used at all, so having it would be pointless beyond having the message. There might be some value in doing that, but I think that we need to be wary of making deprecated too complicated.

I do not think that the current situation is ideal, nor do I think that this patch is entirely ideal, but I really don't know how to improve it beyond this. Ultimately, you need something like -d so that you can compile code which uses deprecated symbols, but unless we complicate things considerably by allowing some way to target specific deprecated symbols with -d, I don't see how we can fix that.

We could just say "screw it," hard deprecate things immediately (that is, deprecate them with the current type of deprecation), and then have a harder level of deprecation where the symbol is unusable - even with -d but gives a message telling the programmer what to use instead, but then everyone would just compile with -d.

So, I don't know. I think that this pull request is an improvement, but I don't see a really great way to fix this problem in general. You either end up pestering the user with messages to fix their code and/or hitting them with breaking changes that they weren't expecting and force them to change their code immediately. We need a fairly simple and reasonable approach to this which at least mostly works, but hopefully in the long run deprecation is fairly rare so that it's not a big issue regardless.

[yebblies]

Walter doesn't want people to be forced to use -d with no warning. (source)

Adding harder deprecation seems to me like it's solving a different problem: giving useless error messages once something has been completely removed. It sounds useful, and this patch is about 6 lines away from giving you that, but it's not the same thing.

[donc]

You either end up pestering the user with messages to fix their code and/or hitting them with breaking changes that they weren't expecting and force them to change their code immediately.

If the "change" simply involves compiling with -d, I don't think that's at all unreasonable.That can be done in less than a minute, and doesn't require any thought. But at the moment, -d is unusable, it leaves your code vulnerable. Which means that at the moment, as soon as something is marked as deprecated, you are practically forced to change it.
Actually the two proposals are almost the same.
hard/normal:
scheduled (only in documentation): you might want to start changing your code now, but compiler won't tell you.
normal deprecated: if you don't want to change your code yet, you must use -d. Message tells you how to change it. No warning if already using -d.
hard deprecated: you must change your code now, even if using -d. Message tells you how to change it.
removed: no impact, code will already have been changed.

normal/soft:
soft deprecated: compiler tells you to start changing your code, but code still compiles. Message tells you how to change it. No notification if using -d.
normal deprecated: if you don't want to change your code yet, you must use -d. Message tells you how to change it. No notification if using -d.
removed: code will fail to compile. No message.

I think documentation/normal/hard is better than soft/normal/sudden death.
OTOH I think that deprecated("message") is far more important than introducing multiple categories. So anything that includes that is an improvement over what we currently have.

[jmdavis]

Well, I don't think that there's much question that adding a message to deprecated is a big improvement by itself. The question though is what to do with the rest of it. Right now, we've been putting messages in the documentation and pragmas in any functions which are templated when scheduling them for deprecation. The messages have been annoying some people (on top of not being specific enough, since they tell you which function not to use but not where you're using it), but without some sort of message mechanism by the compiler, it's not much better than deprecating symbols immediately, because odds are very high that the majority of people won't notice the note in the documentation. We could just add the message to deprecated and drop the pragmas, but if we do that, I'm not sure that there's much point in waiting very long before deprecating something - if not just deprecating it immediately - since it's doubtful that many people will notice the message in the documentation.

[donc]

Walter doesn't want people to be forced to use -d with no warning.

Then I really don't understand.
If adding -d to your makefile is too much of a burden, then I don't see the point of -d. You can still announce deprecation in documentation before you deprecate it, just as is done in other languages. C does soft deprecation perfectly well.

I saw the role of -d to be "I acknowledge that this functionality is deprecated, but I need more time".
From Walter's comment, it seems to be "I should have changed this by now, but I haven't, so I'll change my makefile instead. Stop annoying me. I'll change the code when it's actually removed." Which means it's poor practice to ever use it.

[jmdavis]

Well, I believe that Walter's stance on -d and refusing to even make people change their makefiles when we want to deprecate something is pretty much the entire reason that we started doing "scheduled for deprecation" instead of immediately marking stuff with deprecated.

Personally, I hate -d, because it shuts the compiler up completely, and you don't see what you need to fix anymore, and I do tend to think that it's poor practice to use it as anything other than a quick fix. And if you really have "scheduled for deprecation," then it's that much worse to use -d.

Maybe the better solution would be to make it so that we have soft deprecation as in this pull request (and -d probably makes the warning go away) but change hard deprecation such that -d just makes the error a warning instead of making it go away. Then, if you don't use -d, you get fair warning about the impending change and have time to change your code, and if use -d or don't change your code in time, then you get warning once the symbol has actually been deprecated if you compile with -d, or if you don't use -d, you have to immediately fix your code. So, the deprecation warnings/errors never go completely silent except if you use -d during soft deprecation, which is just delaying the message, since it always shows up during hard deprecation.

[jmdavis]

Actually, with that solution, you still wouldn't get a message with -d and hard deprecation unless you compiled with -wi (and -w would just make it an error in spite of -d). So, maybe that's not a good enough solution. It would be an improvement though IMHO. I do kind of wish though that -wi was the default and that we had a flag to turn off warnings rather than a flag to turn them on. That would make this less of an issue and would encourage people actually fix the warnings in their code.

[JesseKPhillips]

What I think should happen. If you use -d, don't show deprecation messages. If you don't use -d soft deprecation doesn't stop compilation, but hard does. Use -d with -w or -wi to see deprecation messages but don't stop compilation.

[WalterBright]

I get the feeling that this is being way over-engineered. I'd prefer just a simple optional message for the deprecated keyword, and leave the rest as is. We have warnings, and deprecations, and building more layers of complexity on top of this just does not seem worthwhile. The deprecation message should simply say what the user should do to correct the problem.

[jmdavis]

I definitely agree that this risks being overengineered. However, I also think that we need to make changes beyond just adding the ability to give a message to deprecated. Currently, if you mark something as deprecated, then people are forced to immediately change their code or use -d, and if they use -d, then all deprecation messages go away, so they then don't know what they're using which has been deprecated (unless they go and build without -d again) and effectively get no warning when the deprecated symbol is removed. At minimum, it would be far better if the messages were still printed with -d.

And because code is effectively immediately broken when something is deprecated, you wanted us to schedule stuff for deprecation first - which we've been doing. But the only way to give any message beyond putting it in the documentation and changelog (where it's probably not going to be seen by very many people) is to use pragmas, which only works in templated functions, and has proven to be overly annoying for many people.

So, if you stop using pragmas as we have been and we only put information on stuff being scheduled for deprecation in the changelog and documentation, then most people won't notice it, and they won't change their code. So, when we actually deprecate something, it's not much better than having immediately deprecated it. And if people don't want to change their code immediately, they then use -d, don't see any messages, and probably still don't change their code. So, it effectively breaks without warning when the item is actually removed. Such a situation is not exactly ideal and could at least use some minor tweaks.

In the long run, deprecation in Phobos should be fairly rare, so the deprecation mechanism won't be as a big a deal, and so having deprecated be complicated will definitely be overkill. So, we do not want to overengineer this. However, we do need to try and figure out some simple changes that can be made (such as making -d still print the messages, just not make it an error anymore) so that deprecated works better. Adding the messages to deprecated is pretty much a no-brainer, but I don't think that it's quite enough on its own.

[andralex]

One man's over-engineered feature is another man's differentiating feature that makes D attractive. We need to reckon that one way or another we need to be on top of replacing old mistakes with new mistakes :o).

I have a different idea that automates the deprecation process. Consider:

deprecated("Use bar() instead of foo().", "2012/02/14") void foo();

When the compiler encounters that, it compares the seconds string with a string built like yyyy/mm/dd from the current date. (It's a straight string comparison, nothing fancy.) If the string in the deprecation message is greater, allow use of the feature with -d, otherwise unconditionally disallow.

The error message can use all this info to be very nice, e.g.:

Symbol foo has been deprecated and will be removed as of 2012/02/14. Use bar() instead of foo().

and later on:

Symbol foo has been permanently deprecated as of 2012/02/14. Use bar() instead of foo().

[CyberShadow]

What's "current date"? Surely not the date from the system clock during compilation? Dates in deprecation messages are simply a way to refer to future DMD releases without having to use future version numbers. Some people may want to avoid upgrading specifically to avoid the hassle of updating their code base to work with the newer versions - breaking that would be a horrible idea.

[andralex]

Yes, current date is date at compilation time. I don't understand the rest of your argument.

[CyberShadow]

You are suggesting that compilation of programs with the same version of DMD/Phobos to break by itself over time, right?

[andralex]

That is the intent.

[CyberShadow]

...And you really don't see anything wrong with the idea that things break by themselves without the user doing anything?

[WalterBright]

In general, I like Andrei's idea, although dmd does not have any date parsing
code in it, and to add it would be a pile of bloat.

[CyberShadow]

Sorry to sound arrogant, I'm not sure if I'm the only one who sees the problem with this suggestion or simply the consensus being that it's a non-problem. Perhaps I should expand on my thoughts a bit.

I think that making any behavior depend on the system clock is a very bad idea.

Currently, user code breaks when the user performs an action which implies that things might break: update DMD/Phobos. The user usually performs this action at a point where they are ready to handle the consequences: be prepared to be annoyed by new pending-deprecation messages, edit their makefile, finally stop using features that have been pending deprecation, etc.

Making things stop working when the system clock rolls over some value is pretty much putting a time bomb in D. Even though we can expect that the majority of users will want to stay up-to-date with current compiler/library versions, it would be presumptuous to assume that about all users. For all you know, someone may be stuck using an old DMD version due to a very specific compiler regression that no one wants to fix, or their codebase depends on a specific quirk of that compiler or library version, or they simply don't have the resources to migrate their codebase away from the deprecated functionality.

This is half the trouble. In some circumstances (automated builds / build farms), pending-deprecation messages may be hidden from users. It's very uncommon for software which can (and, to some extent, is designed to be) used in an automated manner to stop working depending on the value of the system clock. I imagine this would be an unwelcome surprise to such users.

Given my above points, I don't see how you could consider this objectively a good idea, considering its advantages are relatively minor.

[jmdavis]

Much as I like Andrei's idea in concept, I'm going to have to agree with CyberShadow here. The problem with Andrei's proposal is that it's not at all version-based. If I have version 2.055 of dmd/druntime/phobos and my code compiles with it, I would expect that that code would still compile with that code 2 years from now. For any code that's not actively maintained, that's going to be a problem. Having it not compile with 2.070 would be fine, because it's provides a new version of the libraries (and the compiler itself could contain bug fixes or feature changes which change the behavior or render uncompilable the old code), but 2.055 should still be able to compile the same code.

Much as I like the idea of providing a way to make functions automatically fully deprecate themselves, I think that making it time-based as opposed to version-based is a problem. Simply making deprecated take a flag or bool instead of the date would work much better. Then that flag/bool indicates whether -d stops compilation or not. Sure, the messages get a bit more complicated, since you have to add any dates or version numbers to them yourself, but breaking code based on when it was built rather than with what version of code it was built with definitely seems to me like it would cause problems.

There have been times at work where we have had to compile code for an older product that no one has touched in months, if not years - code which has not been being maintained - and if that code had compiled the last time that it was worked on but doesn't now simply because the date changed, that would be a big problem. The environment (and thus the compiler version) would be the same, because we're strict about our build environments, but code for older products is generally not updated unless there are bug fixes that have to be made, and suddenly having to worry about porting code from older versions of functions or modules to newer functions (which may or may not be trivial) just to do a bug fix isn't really acceptable IMHO.

[WalterBright]

On 9/10/2011 2:27 PM, Vladimir Panteleev wrote:

I think that making any behavior depend on the system clock is a very bad idea.

You make a sound argument. I agree.

[andralex]

I agree with the "time bomb" argument though I'm not as sanguine about it as others. An alternative would be to compare the deprecation date against the compiler's build date. That means in the given example that a compiler released after 2012/02/14 will refuse to compile that code.

Walter, there's no need for date parsing. The beauty of the yyyy/mm/dd format is that you can compare dates just as strings. Databases actually do that to great effect.

[jmdavis]

Using the compiler build date would work quite well for Phobos (and would be quite cool actually), but it doesn't scale beyond that. Should when a function in one of my personal projects is fully deprecated depend on the version of the compiler or Phobos that you're using? I would think not. But with your proposed date scheme, it would.

[WalterBright]

On 9/10/2011 4:20 PM, Andrei Alexandrescu wrote:

Walter, there's no need for date parsing. The beauty of the yyyy/mm/dd format is that you can compare dates just as strings. Databases actually do that to great effect.

Sure, but then the user uses yyyy-mm-dd format, or mm-dd-yyyy format, or
dd.mm.yy, or any of innumerable others and it mysteriously fails. I can see all
the bug reports now. If you're going to do date strings, they have to be done
right. std.dateparse is about the amount of code needed to do it.

Databases might be able to get away with it because they are not using free
format user input strings, but perhaps using form based input that can be sanitized.

[ghost]

I agree with CyberShadow,, but I disagree with this:
"The user usually performs this action at a point where they are ready to handle the consequences: be prepared to be annoyed by new pending-deprecation messages, edit their makefile, finally stop using features that have been pending deprecation, etc."

Currently DMD still has plenty of bugs and a good reason to upgrade is to get the latest set of bugfixes, but you might not be prepared to upgrade to a newer version of Phobos yet. Unfortunately Phobos and DMD releases are tied together, so with new bugfixes you potentially get new deprecation warnings.

[JesseKPhillips]

I'm not sure what to do about the date/soft parameter, but I don't like the idea of -d not allowing deprecated code to compile. For one thing it makes documenting -d confusing. Without -d all deprecations hard/soft should show a message. With -d soft deprecation would not print a message but hard ones still would.

[shoo]

I don't agree to the time bomb.
At least, compiler must be able to compile same code in the same environment. It must not depend on the time.

I think that the soft deprecation is unnecessary, too.
Because it becomes able to compile it by using "-d".
I think that the further buffer mechanisms are unnecessary.
When a programmer uses "-d", compiler displays the message. And, if programmer want to quiet the message, using "-quiet".

[jmdavis]

@yebblies We can all agree that the ability to give deprecated a custom message is a definite improvement. So, I suggest that you close this pull request and create a new one where that improvement is included and only that improvement. If we can agree on other improvements later, then we can add them then, but we might as well get the improvement of the custom messages in so that we can start taking advantage of it.

[yebblies]

I'm going close this in favor of a patch that only adds an optional message to deprecated.

It seems like once this is done, phobos should move to a different deprecation process:

• Scheduled for deprecation - Documentation only, no messages or code changes
• Deprecated - Code tagged with deprecated and requires -d to compile
• Removed

One alteration might be adding a stage between Deprecated and Removed -

• Disabled (???) - Like Don's hard deprecation, but using @disable. @disable already uses the same mechanism as deprecated. Do we need a message for @disable?

I understand this isn't anybody's idea of perfect, but are there any huge problems with this idea?

[jmdavis]

Walter seems to be against pretty much anything which complicates deprecated other than adding the message, and using @disable like that is pretty much just adding a step to deprecated but putting it on @disable instead. So, I think that any idea along those lines needs to be dealt with and discussed separately. Please just add the message to deprecated first and after that's in, we can discuss further improvements. But at this point, the discussion of further improvements just delays us getting the one improvement that we're sure that we want added.

[yebblies]

Please just add the message to deprecated first and after that's in, we can discuss further improvements. But at this point, the discussion of further improvements just delays us getting the one improvement that we're sure that we want added.

Yeah, I did this yesterday: #463 Regardless of other issues, I haven't seen anyone arguing against adding the optional message to deprecated.

I like @donc's suggestion of having a stage where you must modify your code, but the old code is still available for copy-paste rescue. Rather than complicating deprecated to support this, @disable already does it.