Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change LDC version to 2.080.0 #2673

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Conversation

kinke
Copy link
Member

@kinke kinke commented Apr 29, 2018

No description provided.

@JohanEngelen
Copy link
Member

Is there a way to make it clearer that e.g. LDC 2.x.0 is using the frontend of DMD 2.x.3 ?
I ask, because I am against releasing an LDC with a frontend version a.b.0, because the frontend .0 releases are generally of low quality, and for LDC we should strive for higher quality standard (LDC is used in production), i.e. release only after, say, frontend .3 releases.

@kinke
Copy link
Member Author

kinke commented Apr 30, 2018

I thought about releasing a 2.080.0-beta1 as promptly as possible after the upstream release, and releasing the final 2.080.1 as soon as the first upstream patch is released and LDC-specific regressions reported in the (then longer) beta phase have been fixed. I doubt we'll see as many patch versions as for 2.078 (3) soon again, and expect 1 patch to be the regular case.

Copy link
Contributor

@joakim-noah joakim-noah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New release plan sounds good to me.

CMakeLists.txt Outdated
@@ -62,16 +62,12 @@ endfunction()
#

# Version information
set(LDC_VERSION "1.10.0") # May be overridden by git hash tag
set(LDC_VERSION "2.080.0") # May be overridden by git hash tag
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since these are now identical, maybe this can just be set to DMD_VERSION? I guess you want the flexibility for the two to vary?

@JohanEngelen
Copy link
Member

OK, so we number the LDC version the same as the frontend version it's based on? That sounds good.

@dnadlinger
Copy link
Member

-1 from my side, as I think this will only lead to confusion with patch releases and skipped versions – i.e. I think the version numbers should only be synced if the two compilers are released as part of a single release process. Since I haven't been managing the releases for a long time, my opinion probably shouldn't carry too much weight, though.

@JohanEngelen
Copy link
Member

this will only lead to confusion with patch releases

That's a very good point.

@kinke
Copy link
Member Author

kinke commented May 1, 2018

Yeah okay skipped versions are confusing indeed, so how about y in 2.080.y representing the LDC patch level for the 2.080 ABI/defaultlib interface (and LDC 2.080.0-beta1 likely being based on DMD 2.080.0 and LDC 2.080.0 final on DMD 2.080.1)? Can Johan live with that?

@JohanEngelen
Copy link
Member

OK, if going forward with this, let's not follow the DMD patch number.

Something related: do we want to release every two months, or do we just skip version numbers?
https://dlang.org/changelog/release-schedule.html
I don't expect any real testing happening at that pace. Certainly not at Weka. Updating the compiler at Weka is already a large effort because of regressions and language/stdlib changes. Fyi: Weka is currently at LDC 1.6 (with Weka modifications), and recently we were thinking about reverting to LDC 1.2 ...
The extremely high-paced release schedule of DMD is something I had planned to discuss at DConf already.

No need to rush this versioning change, let's let it simmer for a bit?

@kinke
Copy link
Member Author

kinke commented May 1, 2018

Something related: do we want to release every two months, or do we just skip version numbers?

The release overhead has gotten very small, except for the manual armhf and Android packages, so I don't see a reason not to try to keep up with DMD's pace as long as I don't run into a burnout or someone else takes stuff off my shoulders. The upstream changes accumulate anyway, so releasing every 2 months instead of, say, 6 months, makes sure stuff can be tested and fixed earlier. I leave it to the user to decide his migration pace.
I also guess the test coverage actually increases once we're almost even with DMD, as people keeping a manageably-sized code base up-to-date with latest D(MD) can use LDC for CI testing as well.

No need to rush this versioning change, let's let it simmer for a bit?

1.9 => 1.10 would provide a nice opportunity IMO, but I'm okay with delaying it.

@kinke
Copy link
Member Author

kinke commented May 1, 2018

I don't expect any real testing happening at that pace.

747 downloads for 1.9-beta1 in a single week (and 368 for 1.9 final in ~36 hours). Debian/Ubuntu ship with 1.8 (and build all D packages with it). So I'd say testing does happen, and I don't recall that many issues originating from Weka in the first place.

@joakim-noah
Copy link
Contributor

Can we go ahead with this? As @kinke said, the 1.10 numbering might require some adjustment, so no reason not to make the 2.080 adjustment instead. As for patch releases, his proposed solution of having a different patch version for LDC solves it: the next official release would be LDC 2.080.0 with the eventual DMD 2.080.1 frontend, as reported by the compiler. If we need to, we could always update that LDC patch version independent of the dmdfe patch version.

@kinke kinke changed the base branch from merge-2.080 to master May 12, 2018 20:36
@kinke kinke changed the title [2.080] Change LDC version to 2.080.0 Change LDC version to 2.080.0 May 12, 2018
@kinke
Copy link
Member Author

kinke commented May 12, 2018

I'm all for the change, as I somehow don't fancy 1.10.0 (and think the users will appreciate the streamlined versioning scheme & understand the different patch levels). I planned on tagging tomorrow, so if @klickverbot and @JohanEngelen don't agree by then, I'll go with 1.10.0.

@JohanEngelen
Copy link
Member

The amount of downloads doesn't mean much. I am talking about in-depth testing with a complex codebase, where minute semantic details matter, where performance matters, where ABI details matter, etc. Some evidence that such numbers do not mean much: DMD has been downloaded a lot since 2.040 but it has not been tested very well at all: only recently a bug was found with bad codegen for i > 0... https://issues.dlang.org/show_bug.cgi?id=18315 (extreme example, but complaints about bad optimized codegen in the forums are quite common).

The upstream changes accumulate anyway, so releasing every 2 months instead of, say, 6 months, makes sure stuff can be tested and fixed earlier. I leave it to the user to decide his migration pace.

Releasing final versions is not needed for testing; on the contrary, testing should happen before, not after, a release. If we keep a very high pace of releasing, there is not going to be a well-tested release; it'll just all be ok-ish releases that each have their own sets of new bugs, and the user only has the option to migrate to an ok-ish version. The release of 1.9.0 happened after one week of beta. To me, that means it is mainly untested (and even some of our own core CI systems are failing). We are currently moving closer to "hobby-like" style of development, and I believe we can do much better than that.

and I don't recall that many issues originating from Weka in the first place.

regarding Weka: I meant it as an example of a complex codebase where testing a new compiler takes much longer than a week (in part because thusfar, workarounds must be introduced to make the code compile at all with a newer compiler due to regressions or other issues). But plenty of very fundamental issues were found at Weka, in the DMD frontend, in Phobos, LDC, and LDC's druntime.

The extremely high-paced release schedule of DMD is something I had planned to discuss at DConf already.

But I didn't :( I wrote to the maillist instead:
https://forum.dlang.org/post/ingkexhebiaqdzknicvr@forum.dlang.org

@JohanEngelen
Copy link
Member

I planned on tagging tomorrow

Please announce the plans for a new release with, say, a week's notice, such that the other developers of LDC have time to consider.

@joakim-noah
Copy link
Contributor

@JohanEngelen, I agree with most of what you said: this release schedule is too rapid and doesn't give users time to shake out bugs.

However, switching our version numbering to match upstream is not related to that. Has Martin addressed your numbering concerns with this latest pull?

@kinke
Copy link
Member Author

kinke commented May 14, 2018

The release of 1.9.0 happened after one week of beta. To me, that means it is mainly untested

The one week was shorter than the usual 2 weeks, but the reasons were DConf & 2.080 being almost ready. Still, if there are no reported issues after 700 downloads, I take that as a sign that there are no obvious serious regressions.

Releasing final versions is not needed for testing; on the contrary, testing should happen before, not after, a release.

In an ideal world, yes, but we all know this largely isn't happening, at least not if people know the final's gonna be out 2 weeks after the beta. We don't really test LLVM betas ourselves and have only recently been able to test DMD betas (and yes, some of my fixes made it in time into 2.080.0). With CI builds being available all the time, there's no real blocker for users to test latest LDC, but yeah, hardly gonna happen.

Just out of curiousity, has there ever been an LDC release Weka was able to test thoroughly before it actually got released? I totally understand that for commercial users, non-0.17 'LTS' versions would be very convenient, but as such I see that as their (and other interested parties') responsibility to do something about it - and mainly regarding DMD/druntime/Phobos, as OTOH the only LDC-specific regression lately was the install_path shared lib issue on OSX.

We are currently moving closer to "hobby-like" style of development, and I believe we can do much better than that.

As you know, I didn't just keep up with DMD's pace, but was actually faster so that we caught up and are now as good as even, while fixing other things as well. It's clear that this wouldn't have been possible if I had to wait for weeks for the PRs to be finally reviewed & merged by someone else; I guess that's what you're trying to say. Anyway, I think the end result is definitely worth it, hardly any reports of new serious LDC issues and D enthusiasts should get used to a new LDC beta coming out a few weeks after the DMD release, so that I don't have to read 'can't use LDC coz it's lagging several versions behind' anymore. I firmly believe this uniformity is important for the D ecosystem, will increase LDC usage/test coverage (wrt. bandwidth - people exporting naked functions, using identical labels in different function overloads...) and reduce frustration (Mike: 'will have to wait for 4 months for 2.081 language change to land in LDC' - nope, will most likely be 2 months from now, still a lot, but half).

Please announce the plans for a new release with, say, a week's notice, such that the other developers of LDC have time to consider.

Will do; I hoped releasing a 2.080.0-beta1 as promptly as possible after the upstream release would be clear enough (the original plan was to release one week earlier, but I wanted to fix the Ubuntu 18.04 issues first).

@kalev
Copy link
Contributor

kalev commented May 15, 2018

Thanks for being so awesome with LDC, @kinke!

@JohanEngelen
Copy link
Member

However, switching our version numbering to match upstream is not related to that. Has Martin addressed your numbering concerns with this latest pull?

sorry for the thread hijack. I think it's not so bad to align the version numbers, but it does lock us to that versioning scheme. Our current version is quite easily converted to dlang version (+70), but it is indeed often annoying to say "LDC 1.9.0 (dlang 2.079)".
So I'm OK with going ahead with this.

@JohanEngelen
Copy link
Member

Just out of curiousity, has there ever been an LDC release Weka was able to test thoroughly before it actually got released?

Yes. 1.8.0 was (somewhat) tested before release, currently fully passing their test suite (not tested on full full full test suite yet). Little longer ago, if you remember the += evaluation order problem, that was caught at Weka before releasing LDC.
Weka has it's own fork of LDC + druntime, so testing is not very easy. This discussion is good motivation for prioritizing the work on fixing mainline LDC for Weka...

As you know, I didn't just keep up with DMD's pace, but was actually faster

Yeah, a lot of work! DMD's major release schedule is crazy.
Perhaps I'd rather have us just skip frontend versions as in the past?

It's clear that this wouldn't have been possible if I had to wait for weeks for the PRs to be finally reviewed & merged by someone else; I guess that's what you're trying to say.

Sometimes, merging without review is needed for LDC because of manpower (sometimes also review is not taken well, resulting in bad feelings). But we do have to keep up and improve testing and be even more wary of ones own changes. Aside: having 3 of our Travis jobs as "allow failure" is very unfortunate. :(

so that I don't have to read 'can't use LDC coz it's lagging several versions behind' anymore

I think we can just politely ignore such complaints as long as we are not years behind DMD. There will always be a delay, and probably those folks are just hard to satisfy anyway (I find such comments quite disrespectful of our time and work). There's also a group of people who want a stable language that we also shouldn't alienate.

@kinke
Copy link
Member Author

kinke commented May 15, 2018

I think it's not so bad to align the version numbers, but it does lock us to that versioning scheme. Our current version is quite easily converted to dlang version (+70)

Heh, my brain apparently used to replace the last digit with LDC's minor version (7x) and now needed to switch to 70+x. Anyway, when skipping a DMD version, that scheme would be broken badly; so even if we have to skip a version at some point, I'd always prefer skipped LDC versions (2.080, 2.081, 2.083, ...) over a continuous but compatibility-wise very cumbersome 1.10, 1.11, 1.12. [For major.minor that goes (as defining ABI & libs API); finalizing a 2.080.0-beta1 as 2.080.2 final would be very confusing indeed.]
David, you onboard too?

@kinke
Copy link
Member Author

kinke commented May 15, 2018

Weka has it's own fork of LDC + druntime, so testing is not very easy. This discussion is good motivation for prioritizing the work on fixing mainline LDC for Weka...

Sounds good, I hope it plays out.

But we do have to keep up and improve testing and be even more wary of ones own changes.

I know (and try to be); new tests in newer frontends (and defaultlib unittests) make me feel more and more comfortable with overall quality.

Aside: having 3 of our Travis jobs as "allow failure" is very unfortunate. :(

I actually hoped a user would hit it too and so lead to more information about it. It's more looking like a Travis strangeness now, as the single failing DMD test worked fine on both other Ubuntu 14.04 CI platforms (and in my then-16.04 VM). IIRC, Travis and SSH isn't straightforward (so that I chose Circle for that), so I went ahead and allowed the 2 BUILD_SHARED_LIBS=ON jobs to fail as quick remedy; removing the test file probably makes more sense now (as done for other CI strangeness too).
As to the SPIR-V job, I can't (easily) check it myself (single failure, crash in codegen/dcompute_cl_addrspaces.d).

@joakim-noah
Copy link
Contributor

@redstar, any input?

@joakim-noah
Copy link
Contributor

Should we go ahead with this? I think we've given every one time to weigh in.

@joakim-noah
Copy link
Contributor

Would be good to get this in before the second beta, so people know we're making this move. Now is a good time to make the shift, because going to 1.10 may cause similar version numbering problems for build scripts, so might as well make the bigger jump to 2.080, rather than doing it twice.

@JohanEngelen, please merge if you're okay with this, I don't see a reason not to.

@kinke
Copy link
Member Author

kinke commented May 31, 2018

I'm not sure changing the scheme now inbetween betas is a good idea; that's why I wanted the decision before publishing beta1.

going to 1.10 may cause similar version numbering problems for build scripts

Anything particular in mind? Two-digit minor versions wouldn't be something new (0.17).

@joakim-noah
Copy link
Contributor

I doubt between the betas matters, most will only probably package the final release.

As for numbering issues, packaging systems do all kinds of weird things, like thinking 1.10 is lower than 1.9, or the famous issue where Windows went from 8 to 10 supposedly because they were worried about software only checking the first digit and confusing 9 with 95.

Of course, these numbering issues may not really crop up that much for us, just a possibility. But we're going to hit this potential problem here after 1.9 regardless, I think we might as well make the bigger jump once.

@dnadlinger
Copy link
Member

We already packaged the beta.

Any such issues would occur just the same with DMD-style version numbers, so nothing to be gained here from rushing things.

Besides, large minor versions are awfully common in open source software (0.22, MediaWiki 1.30, …), so package managers can handle that just fine.

I'd still like to see a proper discussion of the proposed version switch – including packagers – and announcement on the forums before thinking of implementing something like that. There is little to be gained by switching, but the potential for considerable confusion.

@joakim-noah
Copy link
Contributor

We already packaged the beta.

We already released a manual download, which presents no issues for version numbering, I'm talking about downstream packagers.

Any such issues would occur just the same with DMD-style version numbers, so nothing to be gained here from rushing things.

Not once we switched to DMD version numbering because of its leading zero, the only potential problem would be from this initial jump.

Besides, large minor versions are awfully common in open source software (0.22, MediaWiki 1.30, …), so package managers can handle that just fine.

That's true, which is why I said it's a possible problem, not guaranteed.

I'd still like to see a proper discussion of the proposed version switch – including packagers – and announcement on the forums before thinking of implementing something like that.

If a forum discussion is so necessary, wish you'd mentioned it a month ago when you first chimed in, but it may be worth doing, so done.

There is little to be gained by switching, but the potential for considerable confusion.

On the contrary, I think this move would remove confusion that has been there for years about which ldc version corresponds to a dmd version.

@ximion
Copy link
Contributor

ximion commented Jun 3, 2018

As long as the version number increases and you are certain that you will not jump back to a lower version at some point, changing the version is fine for a Linux distributor.

However, I am a bit concerned over the version jumps being a bit confusing for users, and LDC-specific changes now being in the patch part of the version (in general, it feels like a pretty confusing change).
However, from a purely technical perspective, as long as the version number is increasing, there is no problem at all.

@JohanEngelen
Copy link
Member

Also note that DMD itself is considering a change in version numbering scheme.

@joakim-noah
Copy link
Contributor

Also note that DMD itself is considering a change in version numbering scheme.

To what? Would make sense to match that.

@wilzbach
Copy link
Contributor

To what? Would make sense to match that.

https://forum.dlang.org/post/drcekmxvfszpwifbukzk@forum.dlang.org (no clear consensus has been found though).

@joakim-noah
Copy link
Contributor

Time to push this through?

@kinke
Copy link
Member Author

kinke commented Jul 5, 2018

Well, people seem not to really care, at least that's the picture I get, so IMO we can wait and see if the DMD versioning discussion is going anywhere.

@joakim-noah
Copy link
Contributor

People don't really care about what? Certainly, no issues with matching dmd's versioning have materialized. As for the upstream versioning changing, that was proposed 8 months ago to be changed earlier this year, and nothing has happened, with no updates to that thread for a couple months. Unless one of you has inside info on an imminent dmd versioning change, I don't think they will change their versioning anytime soon.

If you mean users on the forum or github haven't called for this versioning change, that's the point: this kind of change is not made for them. It's made for the majority of users who will never read the forum or github in the first place, to make it easy for them to know what D version they're using when they use ldc.

@joakim-noah
Copy link
Contributor

Alright, can we finally get this in?

@joakim-noah
Copy link
Contributor

Ping, if you update this to latest, we can get this in before the next beta.

@joakim-noah
Copy link
Contributor

I still think we should do this, but if you're not going to, just close the pull.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants