a coherent set of config variables/flags

[oconnor663]

@olson-sean-k and I had a discussion in https://phabricator.buildinspace.com/D197, and I'm just copying the result of that here, partly in case we lose the Phabricator server :)

We want to make the following configs available. Each one determines all the ones that come after it, unless those are explicitly set too.

• $PERU_FILE --peru-file: The filepath to the YAML file to use, instead of searching for peru.yaml.
• $PERU_SYNC_DIR --sync-dir: The directory where the root of the imports tree goes.
• $PERU_STATE_DIR --state-dir: Where to put stuff that normally goes in the .peru dir.
• $PERU_CACHE_DIR --cache-dir: Where to put the cache, which normally goes in cache/ in the state dir.

Note that we are not including a config to change the peru file search name. (We'll be getting rid of the existing $PERU_FILE_NAME config.) When peru searches for the file on its own -- which works even if it's somewhere above the current directory -- it will always look for "peru.yaml". It's hard to imagine how changing that in a global config could be useful to anyone, and if you're going to set it locally for some projects, you might as well set the full filepath explicitly.

We're also probably going to require that, if either one of --peru-file or --sync-dir are set, then both must be set. @olson-sean-k pointed out some very confusing cases that could result from allowing either one to take a default value based on the other. I've forgotten what they were, but they were bad :)

[oconnor663]

Slowly remembering some of the stuff we talked about, for why --peru-file and --sync-dir must go together. Part of the issue is that, whichever one of --peru-file and --sync-dir comes later in the order of operations, you have to think about the situation where the other one is found by searching parent directories for peru.yaml. Could get very confusing.

[oconnor663]

I put a branch up for this: https://github.com/buildinspace/peru/tree/flags

We'll probably need to refactor the docopt parser before we finish this, because throwing all these flags in front of every subcommand doesn't sound like fun.

[oconnor663]

https://phabricator.buildinspace.com/D210

[ghost]

It is very sad to hear you get rid of PERU_FILE_NAME. It breaks all my previous projects because I set it to .peru.yaml which is hidden by default under Linux. So it makes my project directory looks very clean. Really hope you can bring it back.

[oconnor663]

@enzochiau please tell me a little more about how you were using that variable. Were you setting it globally, or calling peru from an alias with it, or what? We added the more explicit PERU_FILE option, which might also work for you.

[ghost]

I set it globally in my login shell as export PERU_FILE_NAME=.peru.yaml. Then I can go to any project and just run peru <sub-cmd> directly from any sub-directory in that project.

The PERU_FILE sounds like a local variable to single project. I am not sure how to use it.

[oconnor663]

One of the reasons we removed PERU_FILE_NAME was that, by encouraging users to set it globally, we were making it more likely you'd break other projects in a confusing way. For scripts and Makefiles, I think --peru-file and --sync-dir are better options. And for interactive use, you could define an alias that used those flags. What you'd miss out on is the ability to call peru from subdirectories of your project, and still have it find your YAML file. If that's something you were missing, I could see adding back in a --peru-search-name flag or something like that, again with the goal of letting you define an alias. But that global PERU_FILE_NAME setting scares me.

[oconnor663]

Ah ok, I just saw your last comment. Yes, that login shell setting is what I was worried about. The problem is, if you cloned one of my projects and tried to run peru, it would be broken, because we use different file names. Maybe that's not too bad if you're calling peru directly and you can see the error message, but if it's inside a Makefile or something I worry it would be very confusing.

[oconnor663]

What I had in mind for your login shell was something like

alias myperu="peru --sync-dir=. --peru-file=.peru.yaml"

Then you would just call myperu in your projects. I think you could even name that alias peru if you want, and it would still be safer than the environment variable, because it wouldn't affect other scripts you call. If you miss the any-subdir behavior, we could define a new --peru-search-name flag and let you use that in the alias. Thoughts?

[oconnor663]

This raises the question: maybe we shouldn't even have created the PERU_FILE and SYNC_DIR variables. My thinking was that, because it's so unlikely anyone would set them globally, maybe it was safe. The one that really can make sense to set globally is PERU_CACHE_DIR.

[ghost]

How about the fallback mechanism? For example: Use --peru-file if it is given. otherwise if PERU_FILE_NAME is defined, use it. if not, use default name Peru.yaml.

[oconnor663]

I think even if we do that, we still have most of the downside. Most projects will just use the defaults, so they'll still get broken by the global setting.

[oconnor663]

Just to be clear, I'm not worried about you specifically breaking other projects :) What I'm worried about is someone new who's just starting to learn peru (maybe even just starting to learn bash), and the instructions for their project say

# Put the following line in ~/.bashrc
PERU_FILE_NAME=our-special-name

The new developer does what they're told and then a week later, when they try to build someone else's random vim plugin, everything is broken, and they have no idea why.

That wouldn't really totally be our fault, but it's kind of our fault, because we encouraged projects to use that global setting. So if we can accomplish the same goals without the risk of confusing people, that would be great.

This is kind of like how Make has the --makefile flag, but no environment variable to set it globally. Because it would break so much stuff.

[oconnor663]

And now that I think about it more, I'm pretty sure we should scrap PERU_FILE, PERU_SYNC_DIR, and PERU_STATE_DIR, and leave those just as flags. PERU_CACHE_DIR is the only one I like anymore :) @olson-sean-k what do you think?

[oconnor663]

@enzochiau do you think the alias approach will work for you?

[ghost]

@oconnor663 I think the alias will work for me if I do not run it in subdirectory. I think i can accept that.

I agree the global variable is not good.

Allow user to change the config may not be a good idea either. It may be very hard to guess what is the special name used in the sub modules When you want to check their configuration.

Is it possible to fix the config name as .peru. yaml which hidden by default just like other tools, such as pylint, flake8 etc.

[oconnor663]

It sounds like we should give you a dedicated flag for this use case, which preserves the ability to sync from subdirectories. I just threw up an example diff: https://phabricator.buildinspace.com/D213

@enzochiau and @olson-sean-k, please take a look at that and let me know what you think.

I don't think I want the peru file hidden by default. To me it feels more like make or setup.py, where looking at it is an important part of figuring out how a project works, especially if it's going to fetch files into the project root. (If not, it's always possible to move the peru.yaml file down into a subdirectory.) Can you give me more of your thoughts?

[ghost]

The code should work for me.

[olson-sean-k]

I think projects should generally avoid custom names for peru.yaml. The issues we've seen so far involving custom names have been about cosmetic rather than functional problems, which aren't worth the potential complications in my opinion. I also agree that peru.yaml is analogous to Makefile or setup.py: visibility is a good thing.

That all being said, I think it's very useful to be able to configure peru to sync into an arbitrary directory using an arbitrary peru.yaml file. It makes peru a more general tool.

See the discussion in D197. Ultimately, I think the mutually exclusive arguments discussed in that diff are a reasonable choice:

peru sync [--peru-file-name=<basename> | (--peru-file=<path> --sync-dir=<path>)]

Note that we could support only the --peru-file and --sync-dir arguments, but that doesn't prevent hacky aliases like peru --peru-file=find.sh --peru-file --sync-dir=find.sh --sync-dir``, which are arguably worse than just supporting a custom name via a mutually exclusive --peru-file-name argument. We could also completely prevent custom names using different arguments, but that obviates some legitimate use cases, in particular those that don't involve the typical project layout.

[olson-sean-k]

As an aside, if we support custom names, we may want to call out some of the problems that users may encounter in the docs. For example:

1. More difficult to integrate with other tools. Tools can't easily find peru.yaml for a given project.
2. More difficult or confusing to work with projects using custom names.
3. Less apparent usage of peru. A project using custom names or hidden files obscures the fact that it uses peru. I don't think this is ever a good thing.