Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

small brain teaser #207

Closed
MarcWeber opened this Issue · 8 comments

2 participants

@MarcWeber
rake -m  -j 4 -t abc

How long is this going to run?

multitask "a" do
  sh "echo a > a; sleep 2"
end

multitask "b" => "a" do
  sh "echo b > b; sleep 1"
end

multitask "c" => "b" do
  sh "echo c > c; sleep 1"
end

multitask "z" do
  sh 'sleep 1'
  rasie "bad"
end

multitask "abc" => ["c", "z"] do
end

Seriously: Why does'nt it run z and a in parallel, then stopping after 2 secs because "z" fails?

I mean devs want both: "--keep-going-in-case-of-failures" and "--stop-on-first-failure" like behaviour, fail early is more important. Am I missunderstanding anything this time?

@jimweirich
Owner

I haven't reviewed the code, but I suspect that the exception is only transmitted to the main task when it waits for the result of task :z. Since task :z is run in a separate thread, exceptions thrown in that thread are not seen in the main thread.

@MarcWeber

rake's documentation says "rake's version of makefiles" - as dev - I think its not enough. Worse: I somehow feel that it would have taken me less time to rewrite something "rake" like from scratch than trying to understand why -j 1 silently behaves like -j 2, why I should bother the user to lookup whether -m is ok or not etc. You know the rake community better than I do. Is it worth fixing? If so in which way? I'm not happy with drake either (without -j it behaves similar to rake, and with -j 4 it takes too long till it starts a build for whatever reason) Is there yet another alternative which is more dev and user friendly? Is there yet another library I may want to look at?

Thanks for your support. Its appreciated. I just don't think that the design of rake fits my needs very well.

@jimweirich
Owner

Thanks for the feedback, but I'm not sure what you are asking to be fixed. The -j1/-j2 issue? Removing -m? The fail early issue? User friendliness issues?

If you wish to make a concrete proposal, I would be happy to evaluate it.

FYI: the -j1/j2 issue is already fixed in HEAD.

@MarcWeber

There should be 2 ways to run rake:

  • fail as early as possible whenever a task failed (keep running tasks running)
  • fail as late as possible - try to run as many tasks as possible successfully (thus continue till no tasks are left which could be run because prerequisites failed)

Imagine task a and b taking 2min to run. But you have to rerun them (for whatever reason, eg because you use clean because something rake doesn't track has changed). Then why run b if z failed? If you're in a console the difference may not be that "big", because you can abort by ctrl-c. If you have to use editors loading the result into an error list aborting manually is not that nice. Fail early is what I want then.

So I want rake to abort. And I want a way to specify "-m" in the Rakefile or have a nice way to make a task a "multitask". The user should just run "rake target" without bothering thinking about -m or parallel jobs.

I'm aware than "-m" cannot (and should not) be removed for compatibility reasons.

@jimweirich
Owner

Go ahead an make a feature request for the fail early part (i.e. open a specific issue for just that feature).

Then review issue #205 and see if that meets your desire for an easy way of making any task or file a "multitask". Feel free to add comments there.

Feel free to close this discussion thread once you've done that.

@MarcWeber

Fixing "rake" and testing would have taken too much time.

Instead I've implemented a minimal alternative make like system for Ruby which does exactly what I need:
https://github.com/MarcWeber/rumake

There is yet another new feature: It builds the dependency chain which will take longest first.
Eg if there are 100 tasks not depending on anything and 20 task depending on each other
one thread will start with the 20 chain first.

Rake users can give it a try to find out how much value this provides to them for their projects.
Then it can be discussed whether those features are worth integrating into rake.

You may want to keep this issue open for a while or add this as "future ideas" to
your README to gather more feedback about whether users (beside me) want such.

@jimweirich
Owner

Thanks for the feedback. I'll keep this request in mind, but I'm going to go ahead a close it for now.

@jimweirich jimweirich closed this
@MarcWeber

small update for those who are following this thread: I implemented "eta" feature and redoing a task if the command line changes. eta and previous commands are stored in a cache file

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.