`dir` does not create a directory in some cases

[AlekSi]

• Task version: v3.20.0
• Operating System: macOS 13.2

Taskfile.yml:

  test-dir:
    dir: does-not-exist
    cmds:
      - echo {{.TEST}}
    vars:
      TEST:
        sh: cat ../exist.txt

dir is documented to create a directory if it does not exist, but that not happens there. If directory "does-not-exist" does not exist, that task fails with:

task: Command "cat ../exist.txt" failed: chdir /<...>/does-not-exist: no such file or directory

So it tries to change directory to "does-not-exist" without creating it.

[pd93]

Unfortunately, the fact that dir accepts variables means that the code that resolves variables (the compiler), runs before the code that creates the directory. If we were to change it so that the mkdir code runs first, your dynamic variable would work (as the directory would exist prior to you trying to compile it). However, it would break dir for anybody using it with a variable (This is probably quite common).

The only possible solution I can think of, would be to create the dir before compilation if there are no variables, and after compilation if there are variables. We would then have to create an error if the user tries to use dir with a dynamically defined variable.

The area of code around variables and how/when they are resolved is quite complex, so I wouldn't expect any significant changes on this for now.

[AlekSi]

I think resolving variables should not change the directory, right? So it does not matter when variables are resolved relative to directory creation. What matters is that mkdir should be called before chdir.

[richard-hajek]

I think resolving variables should not change the directory, right?

tasks:
   default:
       cmds:
             - pwd
       dir: {{.VAR}}
       vars:
           VAR:
                sh: cat ./target.txt

What matters is that mkdir should be called before chdir

Admittedly, just calling mkdir inbefore each chdir might make sense? idk

[AlekSi]

We use Taskfile to have some abstraction over Unix variants and Windows. The latter has mkdir command, but the flags are different. Given the Taskfile provides dir option almost makes it a cross-platform equivalent.

[trulede]

This problem is likely resolved with the latest versions of Task. Please try them, and if not better, reopen this issue.

[AlekSi]

❯ bin/task --version
3.45.5

❯ bin/task test-dir
task: Command "cat ../exist.txt" failed: chdir /<...>/does-not-exist: no such file or directory

It is not resolved. And only maintainers can reopen the issue:

[trulede]

This is the problem with fixing this:

version: '3'

tasks:
  default:
    # dir: does-not-exist
    dir: '{{.SOMEDIR}}'
    cmds:
      - pwd
      - echo {{.TEST}}
    vars:
      TEST:
        sh: cat ../exist.txt
      SOMEDIR:
        sh: echo "some/dir"

where do you think TEST will cat from? some/dir/../exist.txt?

$ echo "foo" > ../exist.txt
@blue:~/git/task/ISSUES/1001$ mkdir some
@blue:~/git/task/ISSUES/1001$ echo "bar" > some/exist.txt
@blue:~/git/task/ISSUES/1001$ task
task: [default] pwd
/home/blue/git/task/ISSUES/1001/some/dir
task: [default] echo "foo"
foo

Nope, not the one that is wanted. I don't really think it can be solved. Its kind of stupid, vars have to be evaluated before the task can run (and create its directory), so that templates can be evaluated, at which point the dir is known. But the vars attempt to run in whatever the task dir evaluates too - don't be tricked, its not so easy to know what that will be.

@AlekSi @andreynering it could be possible to mitigate the problem; if, when evaluating the vars the task dir does not exist, then fall back to the special variable TASKFILE_DIR. The dynamic vars would then run, but in some cases, not where you think.

[trulede]

task.dir cannot be reliably known until after a task is compiled.
Therefore, dynamic vars of a task will not work with relative paths if the task has a configured task.dir.

@AlekSi so the situation here is that task is working as designed. The problem could possibly be mitigated, but not solved.

So, your options at the moment are:

• design your tasks with that behaviour in mind. For example use absolute paths in these dynamic vars for such cases.
• use the latest version of task which has versions of linux commands that run on windows, so you can design cross-platform tasks.

The "obvious" solution is this:

1. At the beginning, before attempting to run a task, template the task.dir without evaluating SH commands. If it evaluates then create the task.dir.
2. Evaluate the vars and compile the task.
3. Run the task.

If you task uses a templated value for task.dir ... then the directory would not be created before point 2 ... and you will get confused.

Now, rather than have something that does not work in a clear way (see highlighted statement above), you get a more complicated mix of behaviour. The outcome of that is something to ruminate over.

The place for the "fix" would be here:

task/task.go

Line 76 in 19d8fae

The task.dir is created here, so something similar would be needed at the aforementioned location:

task/task.go

Line 263 in 19d8fae

if err := e.mkdir(t); err != nil {

I think its reasonable. Feel free to write a PR.