reggae phobos build triggering autotester #4194
Conversation
atilaneves
force-pushed
the
reggae_autotester
branch
4 times, most recently
from
2b5fd17
to
5588936
Compare
atilaneves
changed the title
|
Ping @atilaneves - did you get it to work? |
|
No, I couldn't find a working dmd on the autotester machine to build and bootstrap reggae. |
Wouldn't ENV['DMD'] be it? |
atilaneves
force-pushed
the
reggae_autotester
branch
2 times, most recently
from
602c70a
to
3f07df0
Compare
|
|
|
|
atilaneves
force-pushed
the
reggae_autotester
branch
from
e21a858
to
acc8409
Compare
|
Yeah, from my last attempt, |
what about downloading the dmd compiler then? Linux/MacOS Window: |
atilaneves
force-pushed
the
reggae_autotester
branch
2 times, most recently
from
08067de
to
d8b2507
Compare
|
@atilaneves what were the result of your informal poll yesterday? |
|
@wilzbach The poll result was 20 in favour of getting rid of the Makefiles, 10 didn't care, 3 liked them as they are. |
|
@atilaneves wow 20 vs 3 is an 86% majority. Did you by chance also ask Walter or Andrei? |
|
@wilzbach Makefiles are a DSL and no one wants to maintain them (how many times have people forgot to update the win32 file). Replacing it with D is a good in and of itself. I don't think a poll need at this point. |
I fully agree, it just felt that we need to show @atilaneves that once reggae is ready, we are willing to replace the makefiles.
|
atilaneves
force-pushed
the
reggae_autotester
branch
from
ad0b456
to
0646400
Compare
|
... And the autotester is green. Now I can really start playing |
atilaneves
force-pushed
the
reggae_autotester
branch
from
9b958d3
to
142982b
Compare
|
Hmm unfortunately this seemed to have gone stalled, so I will close this with regrets :/ |
|
@wilzbach I don't like any of the Makefiles, but I don't know what else I can do at this point. |
|
Why isn't this "a clear net benefit?" As for what else can be done, how about taking out the reggae source and any changes to the makefiles and just submitting the reggaefile alone, for use by those who have reggae and would rather use it? It was not clear this pull was ever actually submitted, as you kept saying you were working on it. |
|
IMHO it is a clear net benefit. But not enough interest was generated, so... |
|
OK, because you said a couple comments up that you didn't think it was. @wilzbach and @CyberShadow, what do you think about just merging the reggaefile alone, as I suggested above, so those who want to use reggae can try building Phobos regularly with it? I'll note that I'm not sold that a build script written in D is the best syntax, but it's certainly better than Make for a D project. If it's in the repo, we can experiment with actually using and modifying it and decide whether to use it more based on some experience with how well it works for a large D project. |
First of all - I (and I think @CyberShadow as well) are deeply unhappy about the current Makefile situation. We invest a ton of time in maintaining them across all repos, especially dlang.org is a real PITA (-> just have a look at the history of dlang.org's -> I'm absolutely pro dropping the
Hmm, I think no one (except @atilaneves) is 100% convinced that reggae is the best replacement. @CyberShadow what's your take on this? |
There is some discussion in a thread from two weeks ago:
Well, I'm not really sure what the benefit of that would be. For one thing, without some sort of CI or such, the reggaefile is going to be maintained only by people who use reggae regularly to build Phobos, and that's probably not going to be many people. So, it will often be outdated. At this point, the question is why even keep it in this repository? A separate repository which contains the reggaefile to build Phobos seems like it would work better and be completely uncontroversial. The other thing is that I'm not really clear how useful this is when you still must use Make to build dmd and druntime. |
I've been over this in the forum thread you linked: to try it out.
I don't see why it would be "controversial" to add a build script for another build system, with a note that it's not the main one and to ping Atila or some other maintainer if it's not up to date. Either way, can you put this online elsewhere, Atila? For example, reggae is notoriously missing good examples of reggae files, especially a big one like this that doesn't do everything at compile-time. |
|
@joakim-noah I could put this online somewhere but it probably won't build anymore (who knows) so I don't know what the point would be. It's not like the diff isn't visible here, so... And as much as I wouldn't like this to be controversial, it apparently is. |
|
Nobody using reggae is ever going to find this diff. The point is that reggae has very little docs or worthwhile examples. I know because I actually spent hours trying to build my Android samples with reggae, and when I needed an example reggae file that used environment variables, you pointed me at this one, because nothing in your reggae repo does that. From your vote listed earlier, this pull doesn't sound very controversial. It sounds more like a very small minority is pulling rank because they're used to the outdated default build system and are blocking any potential change to minimize their own discomfort, at the cost of the project. It doesn't affect me much as I haven't built these dlang repos in a year or two: I only work with ldc and its CMake build system, which is marginally better. I'm only pushing this because I think it will help the project, it does nothing for me. |
|
@atilaneves Please don't give up on this. If the leadership fails to recognize how bad our build system is it doesn't mean the rest should suffer. |
What the hell. I sure hope that wasn't directed at me, because I'm one of the loudest to complain about the current state of the build system. But, listen: dropping an incomplete, under-documented, unmaintained "replacement" in our lap is not a solution. Dividing people into camps does not help anyone. We're all on the same side, OK? There are no ranks in @dlang other than BDFL and contributor. Any build system replacement is going to require work, lots of it. A proposal is not enough - commitment is mandatory. If someone runs into a problem with reggae, do you expect them to drop what they're doing and fix the reggae bug? If they're dealing with the build system, they're likely already two or three problems deep. Any highly-utilized piece of code in our repos needs an "owner" / maintainer who can address issues when they appear, and when it's something as important as the build system, you can bet there's going to be lots of them. Finally, almost nothing is going to get in simply by being proposed - someone needs to keep pushing it all the way until it's in. This isn't some "unfair harsh truth", but a simple consequence that almost everyone here is a volunteer, and just because you have a problem doesn't mean that anyone else has to care about it. In this particular case, I do care about this, but definitely not to the point where I am going to single-handedly undertake all the effort of maintaining reggae, all reggae files in all repos, all the support issues that are going to appear with regards to using reggae, pushing reggae to be accepted by BDFLs, and all the running around in between. I'm certain that the only way this has a chance of moving forward is as a collaboration between reggae's maintainer (@atilaneves or anyone willing to step up to the position) and people maintaining D's build infrastructure (essentially Martin, Sebastian, and myself). I've attempted to open a dialogue in this direction in the above-linked forum thread, and what's the result? Grumbling about rank elitism and status-quo entrenchment in GitHub comments. So, let's talk. And no pro/anti-makefile/reggae-ism please, but purely finding what's best for D. |
I know, I've read that thread a couple times, long before you linked me to it above. You seemed open to change, but I agree with Russel that saying we need a big bang change to all dlang repos to even consider a new build system is excessive.
What does it matter, since I've repeatedly said it doesn't have to be a "replacement" right away? In fact, that should not be the goal. The only way we're ever going to move off Make is by experimenting with alternatives and seeing what sticks. I suggested a whole scheme in that thread to Walter, where we could let in some experimental build scripts, with no guarantees of maintenance, and see what happens. It could be nothing we try will prove better than Make, but my suggestion was met with silence. Since then, others keep repeating the false premise that an experimental build script would need to be complete or maintained, when the whole point of experimenting is to see if it would be maintained by people who like it enough.
The vote for keeping Make was 3 out of 33, it is ridiculous that we're even talking about not experimenting with other build systems given that vote. The only reason I can think of is someone with rank obstructing this.
Experimental build scripts in the repo require none of this overhead. I'll point you at my suggestion to Walter in the the thread you linked, rather than repeating myself while you keep raising objections that I already addressed a couple weeks ago.
I'm not proposing switching from Make now, only experimenting by accepting some alternate, non-default build scripts in the official repo, so Phobos users can discover and play with them. Maybe we'll find that none are better than Make. That would require no effort from you or anybody other than the reggae/Meson proponents, yet it's met either with silence or with repeated protestations of too much work for the contributors, ignoring that these build scripts would be experimental. If their proponents don't maintain them much and they don't gain a champion who does, you simply remove them some day, as I said in the forum post.
I agree that you gave a nice overview of what would be necessary to switch over to a new build system someday. Yet, you and Walter have ignored all suggestions of experimentation. I find this weird, given the overwhelming majority who wants to ditch Make or doesn't care if we do. Maybe my concept of experimentally accepting some unmaintained build scripts hasn't been understood, or maybe someone is blocking the will of the vast majority of the D voters. I can't know which it is, but an objective outsider would find the latter plausible.
I'm not pro-anything, I noted that the reggae syntax is probably suboptimal. But I'll take suboptimal over horrible, which is what I think of Make. Either way, I just want to run an experiment, one that will have no effect on Phobos contributors who ignore it. I wonder who doesn't want to run that experiment and why. |
Where I agree with you: any potential replacement will need to go through a trial (experimental, if you will) stage and used along-side the current one. Where I disagree: anything that relies on "seeing what's going to happen" is doomed to fail. Unless there is commitment and critical mass behind a proposal, consensus from all sides that this is something that we may want to move forward with one day, or if it's not even clear that it satisfies the technical requirements, then the whole thing is pointless and just a waste of everyone's time.
The very notion of "people who like it enough" relies on the uncertain assumption that this group exists, will continue to exist, and is big enough to keep the project going. It's not enough.
What vote, was it an anonymous Internet poll? A vote could mean anything from "I deal with Make problems every day and I'm sick and tired of it" to "I never touched a D makefile in my life, but Make is old and stinky, meson/reggae/etc. is new and shiny, therefore it is better". They will not convince anyone. What we need is user stories of how Make is making their lives difficult and proposals of how potential replacements will deal with them.
We are talking about it right now. Here's what I don't understand, though: why do you think it's so important that it's in the official repository? What's stopping anyone from maintaining a fork/branch that includes the reggaefile (such as this pull request), or a separate repository with just the build scripts? And why are you so focused on its inclusion here when we haven't even begun to discuss the technical merits of reggae? It seems like a failure of priorities.
Experimental build scripts in the official repository are pointless unless they are backed by the ultimate goal of replacing Make, and that in turn requires commitment.
OK... do you honestly think that satisfying the whims of an Internet vote is more important than carefully discussing the problems at hand and finding the best solution? If so, I might as well give up any hope of reason.
If you don't want to discuss things in a constructive manner, then I see no point in wasting my time on this. Final attempt to salvage this discussion: Why do you find Make horrible, and how does reggae address it? |
No, you've got this all wrong. Experimental build scripts require no consensus, just a bare minimum of workability, which this reggae file passes. You put a bunch of experimental build scripts alongside Make, let contributors play with them and then see what consensus develops. The whole point of running an experiment is to form a consensus someday: asking for consensus before accepting anything even as an experiment temporarily is doomed to a veto by ignorance, from those who never used it.
The point of an experiment is to see if such a group exists. You don't know the answer and neither do I, that's why you run an experiment.
I don't know where the vote was taken, Atila will have to fill us in. As for user stories, we have two right here in this thread, from you and Seb.
Why are you so focused on experimental build scripts not being in Phobos? Obviously no Phobos contributor will ever find some random reggae fork somewhere, or care if the official repo is not experimenting with it. The idea is to put some Make replacements in front of them and see what they like the best. They may still like Make the best, I don't know. You keep raising the bar of "discussing the technical merits" of an experiment. We could discuss some of that beforehand, but you will never know as much as after trying and experimenting with a technology. There is no point in discussing much about an experiment beforehand, because even if reggae or Meson is missing some features, if we like it enough otherwise, we could fill those in.
Obviously the ultimate goal is to replace Make, but I disagree that commitment is required now. We are running the experiment to see if the commitment is there.
You keep coming back to discussing a solution, when the solution is almost always found by experimentation, not by design. The time to discuss is after the experiment, not before we even run it.
I don't know what you don't find "constructive" there: I honestly find it ridiculous that I'm having to justify an experiment with alternate build scripts so much, given the overwhelming desire for a change and basically no one in favor of the status quo.
Make's syntax issues are well known, and reggae being D obviously is somewhat better. But I'm not going to discuss this, because this is not the time to do it. You run an experiment with all comers, keep an open mind, and see what works out best. Your notion that we're going to pass each build system option through a heavy filter of technical debate before we even experiment with it is all wrong. |
Err. If you need to run an experiment to find these things out, I'm pretty sure I can tell you the answer right now.
Because a pointless addition is just junk. Why would I not be against adding junk to the official repos?
I think this whole argument stems from some confusion in how development happens here (and not just). There is this misconception that once something gets accepted, the maintainers will thereon take care of it and it will never bit-rot. There is also another misconception that just because something is written in D, it will get preferential treatment and everyone will want to improve it. There is a third misconception, that putting a project in the spotlight will automatically attract more contributors. We have plenty experience that show that all of the above are just not true. Dogfooding only works either if you're the software's author or the author is around and can fix issues promptly. If a tool is broken, reaching for another tool is almost always going to be simpler that trying to fix the first tool. So, to me it seems pretty obvious that without some commitment (and not just some hand-wavy "merge it and they will come"), any proposal is a non-starter. But if you still think that I'm wrong or my experience is irrelevant, feel free to get someone else to merge it - I won't stop you, but I also don't have to help you.
I don't need this merged before I - one of the users you yourself mentioned with use cases against Make - can ask questions about and criticize the current implementation. But if after all said you still prefer discussing politics and process over the technical issues which actually matter, then I see no point in wasting my time here. Good night. |
Yes, it is obvious you or someone else doesn't want to run the experiment because you know the answer is that Make will be ditched.
By that rationale,
The only blatant misconception here is your own. I'm saying let's run an experiment to see if they're liked and do not bitrot much. I have repeatedly said that the Phobos maintainers should just ignore the experiment if they want: there is no expectation of official maintenance for an experiment. Your repeated statement of this canard shows that you are not interested in an honest debate.
Again, the misconception is only your own: I have suggested accepting Meson, based on Python, and any other build system, as long as someone is willing to put in the time to submit a basic build script for Phobos, no low barrier.
Sadly, you're now three for three on misconceptions. I said, let us see if being in the spotlight will attract more contributors, a completely different proposition. I never said being in Phobos guarantees anything will replace Make, that is why I always call it an experiment.
So let's reach for one of these other build tools and ditch Make. :)
I think you're thinking about this all wrong. This is not a new department in a company, where we justify a new direction and then give it a budget, whether in committment or money. This is an OSS project, where we lightly experiment with various approaches and depending on what works best and has the most contributors behind it, we pick one. Any social or sufficiently complex technical domain is all about floating an idea (in this case, merging in experimental build scripts) and seeing what works better and who gets behind it. As Linus notes in the link I gave, you will almost always lose if you try to design a solution to death yourself beforehand.
I don't know what you're trying to say here: you want to "ask questions about and criticize the current implementation" of Make? If you mean you'd like to discuss reggae, I've repeatedly said now is not the time to discuss an experimental build script. Almost nobody in this project has much experience with reggae or Meson, that is why we experiment with them first.
There was almost no discussion of "politics," and "process over the technical issues" is the whole point of this discussion: your process of deciding whether to accept experimental build scripts is wrong. We should try things out experimentally before discussing the technical issues, not after. But you're right that this discussion between you and me is pointless, as I keep repeating the same things over and over and you keep having misconceptions about what I'm saying. |
|
FWIW, I'm actually in favor of ditching make, especially if the replacement is something based on D. (Dogfooding and all that.) However, I don't think this PR is the right approach. The current devs are not going to maintain an alternate build script, and it's just going to bitrot. Plus, it will require a stream of continuous PRs to keep the scripts updated alongside the makefiles, which only adds more load to the already clogged PR queue. In the end, it will become a reason not to ditch make. I don't want to see that. I think a better approach is to fork the relevant repos and maintain the reggae build scripts there, and regularly sync it with upstream. Git lets you do this relatively painlessly. More importantly, refine the scripts to do everything the makefiles do today, except better. Once we get to that point, we can make it do more than the makefiles do today. If interest in a make replacement is strong enough, we ought to get enough manpower to do this. Then we will have solid proof that the new build system is superior to make. Otherwise, if there is really only little interest, then it will languish and we just let it die. Arguing and getting angry over this PR is not helping the cause. |
Nobody has asked them to maintain it and you don't know if it will bitrot or not.
I count 43 non-merge commits to
I don't know in what world this could be true.
This is the usual forking approach, when the project maintainers are obstinate and won't listen to reason. I had hoped that wasn't the case, but you may be right it is the only option left. It is a much higher barrier, because someone will have to do all that work and then still might be rejected. Since a build system is not a core compoment of a project, nobody is likely to take that risk. A better approach is experimenting in the official repo like I suggested, but you're right that nuclear option is always there, just as ldc ditched Make and uses CMake. I will be using dub or some other D build tool, like reggae or button, for building cross-compiled Phobos and the sample Android apps in my repo, as I mentioned recently on the forum. Thankfully, I never have to use these Makefiles. I was trying to save others from that pain, but somebody seems to be against running an experiment to see if that's possible.
There is almost no discussion of this PR anymore, but of my idea of experimenting with other build systems. When people are against running an experiment that costs them nothing to run, it's usually because they know they will not like the results. |
|
Conspiracy theories are not helping the cause either. It's really very simple. Fork the repo, put in the alternative build scripts so that they replicate everything the makefiles do alongside, make the whole thing streamlined and buildable. In fact, let's do it the Right Way. Have a single build script handle building all of dmd, druntime, phobos, tools, and dlang.org, including generating all the docs, binaries, packages, website files, etc.. Of course, most of this will be optional, the default target is just a working dmd toolchain. But make the build script handle all the other stuff seamlessly: one push of the button, and it builds the whole damn thing, bells, whistles, and all. A single command to do it all, and do it right, as in, modify That should be the selling point of the alternative build system. I don't think the core devs will be able to resist that once it's actually all working. :-D |
|
(And if the alternative build system is unable to do all that, well, then perhaps the devs have a point that there isn't yet a strong reason to ditch make. I disagree, of course, but the point is that we want to eliminate all possible reasons not to switch.) |
|
As I noted earlier in this thread to Vlad, the alternative you two are suggesting is ludicrous, just as Russel said. It is unlikely anybody will put in all that work, only to have the project maintainers likely reject it with the same irrational objections they've given to just accepting some experimental build scripts that don't implement all the "bells, whistles" right away. If they won't even experiment, why would they ever accept a switch from Make? The whole idea of experimenting in this official repo is for those Phobos contributors who want to switch off Make to experiment with some new build systems, to acclimatize themselves slowly with some alternatives. This is the way real change works, almost never with big bang replacements like you're suggesting. It is why we have I can only repeat myself on this point so many times. |
This isn't a real PR, please disconsider. It only exists to trigger the autotester