"Error determining list of magefiles" in a package that does not compile

[mirogta]

I'm trying to use Mage on a package which does not compile and I think it's a valid scenario where it would be great it Mage has worked somehow.

Scenario:
I have a package which uses protobuf and generates *.pb.go files from source *.proto files. When you check out the (private) repo, go build would fail as expected. What I want to do is to run mage protoc where the func Protoc() wraps the protoc-gen-go generator with all its complexity. Needless to say, this has worked quite smoothly in a Makefile.

It however does not work with Mage, because if the package does not compile, Mage does not work.

I.e. the error when running mage -l or any other Mage command is:

Error determining list of magefiles: failed to list non-mage gofiles: exit status 1: go: finding github.com/<user>/<package> latest
build github.com/<user>/<package>: cannot load github.com/<user>/<package>: no matching versions for query "latest"

Running go list gives the same error:

go: finding github.com/<user>/<package> latest
build github.com/<user>/<package>: cannot load github.com/<user>/<package>: no matching versions for query "latest"

As a workaround, I've moved the magefile.go to a build subdirectory and then I could run mage -d build. This does work, but... considering I have about 20 build targets and some of them depend on config files in the repo, I would also need to pass the -w flag so Mage doesn't try to find the files in the build directory. So it gets quite complex. I would really prefer a solution where I can have the magefile.go in the root of the repo even when go list fails, so that I can simply run mage. Can this be done?

[ntrrg]

You can create a .mage directory (or any other name), compile it in there and use the binary in your project root directory. It would be something like

$ cd .mage
$ go run mage.go -compile ../mage
$ cd ..
$ ./mage -l

See https://github.com/ntrrg/ntweb/tree/master/.mage

What I usually do is

$ .mage/compile.sh
$ ./mage -l

And I have this in my .gitignore

/mage

[mirogta]

@ntrrg thanks, that is also a good option and "your" .mage is even better name than "my" build directory.

I'd still prefer a simpler solution where I can develop it iteratively/quickly without any unnecessary steps and I don't need to have the magefile in a subdirectory, whether executed with -d or compiled with -compile.

[natefinch]

This is a drawback of having mage rely on the default build tools - if you have broken stuff in the same directory, the go tools barf.

I wonder if a solution could be something like having a .mage file in the current directory that mage reads to set the directory in which to build and run mage (i.e. sets -d and -w) ... then you could run mage in the current directory but it'll build from a subdirectory where things aren't broken.

[mirogta]

Brilliant idea, that would work really well!

Could it be .mage.yml, if it's not a too much of a problem to add a yaml config reading dependency to Mage?

[natefinch]

Yaml would require importing an external library which I would like to avoid. This can probably be a super simple file with just foo = bar style settings. Yaml would be overkill I think.

…

On Tue, Jan 21, 2020, 4:51 PM Miro ***@***.***> wrote: Brilliant idea, that would work really well! Could it be .mage.yml, if it's not a too much of a problem to add a yaml config reading dependency to Mage? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#283?email_source=notifications&email_token=AAYJZSBSCZK6FCNISONOZJ3Q65U4PA5CNFSM4KJR7EJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJRMR3I#issuecomment-576899309>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAYJZSB344NXIAYTGYPQNC3Q65U4PANCNFSM4KJR7EJA> .

[mirogta]

Yaml would require importing an external library which I would like to avoid. This can probably be a super simple file with just foo = bar style settings. Yaml would be overkill I think.
…
On Tue, Jan 21, 2020, 4:51 PM Miro @.***> wrote: Brilliant idea, that would work really well! Could it be .mage.yml, if it's not a too much of a problem to add a yaml config reading dependency to Mage?

Yeah. I was thinking just to make it nice and readable, but keep it simple by all means. .mageconfig perhaps than .mage? I've already renamed my directory to .mage following up on @ntrrg 's reply, and guess having a directory and file with the same name would technically work but could be confusing.

So something like

#.mageconfig
mageDir=.mage
workDir=.

rather than

#.mageconfig
-d=.mage
-w=.

?

[natefinch]

Yeah, the former is what I was thinking. You could almost just use something like direnv, but adding environment variables for magedir and working directory just doesn't make sense outside of this one case... so I think a custom file is probably the way to go.... and pretty easy, tbh.