Makesure can not be used to execute two goals with effects that counter each other

[koljakube]

I sometimes need to implement update tasks.
A basic setup can look like this:

@goal 'x-created'
@reached_if test -f x
  touch x

@goal 'x-deleted'
@reached_if ! test -e x
  rm x

Now the update task would look like this:

@goal 'x-updated'
@depends_on 'x-deleted'
@depends_on 'x-created'

This does not work, since the reached-checks are checked during build graph construction, as far as I can tell.

Workarounds:

• Calling ./makesure in x-updated instead of dependencies.
• Simply executing ./makesure x-updated twice.
• Removing checks (not very useful).

I could think of ways for makesure to support this better:

• reached_if-checks could happen before goal execution (but I guess that would change too much of makesure's behavior).
• A variable could be set for goals that were invoked directly from the CLI, and unset for dependencies.
• More general, a variable that contains the "depenency depth level", à la $SHLVL.
• Some "always execute" pseudo-parameter that could be passed to every goal.

I'm writing this primarily to start discussion, since I use makesure quite heavily at the moment. What do you think about this?

[xonixx]

This is interesting question indeed.

For sure changing the operational semantics can hurt the intended declarative nature of the tool. For example, now any goal runs at most one time, even if depended on multiple times.

I think the root of the issue is that dependency on a goal is not equivalent to a goal invocation.

What you could also try is something like:

@goal 'x-created'
@reached_if test -f x
@depends_on 'create-x'

@goal 'x-deleted'
@reached_if ! test -e x
@depends_on 'delete-x'

@goal 'create-x'
  touch x

@goal 'delete-x'
  rm -f x

@goal 'x-updated'
@depends_on 'delete-x'
@depends_on 'create-x'

I see how it’s bulky and repetitive, though.

Also the example with a file, though serves a good role of showing the problem, looks a bit abstract. If possible, I would like to see a more realistic example (from real project). This could help finding more elegant/appropriate solution.

[koljakube]

A practical example would be the following, a chain of goals for task that is meant to be executed at regular intervals and that cleans up after itself:

• A goal to build a docker image (reached_if the image exists)
• A goal to save this image to a tar archive (reached_if the archive exists)
• A goal to upload this image to a remote server (always performed)
• A goal to remove the tar archive (reached_if the archive does not exist)
• A goal to remove the docker image (reached_if the image does not exist)
• A goal that depends_on all of the above, in the order listed

Given an empty directory and docker cache, the latter two goals would never be executed, leaving the files around.

In the end, I guess there is no right or wrong here, it's a matter of preference when to evaluate the reached_ifs. In the end, the same effect can be achieved by replacing @reached_if with shell if; then; fis. I just like the high visibility of the presence of a @reached_if.

[xonixx]

I'm considering the next idea. In your initial example you will have:

@goal 'x-updated'
@call 'x-deleted'
@call 'x-created'

So using @call (maybe I'll prefer @calls) instead of @depends_on.
The idea is the @call represents explicitly non-declarative semantics. So the above we be funtionally equivalent to

@goal 'x-updated'
  "$MAKESURE" x-deleted
  "$MAKESURE" x-created

Many things to consider with this approach, but may be good enough for the scenario you describe and the like.

• Do we allow both @call and @depends_on on the same level?
• Do we allow @call goal_name @args 'arg'
• Can we implement it more efficient than just running more makesure processes?
• How do we output the dependency tree ( -d,--resolved)
• the @define inheritance
• etc.