sync without revisions?

[questor]

sorry for writing tickets and not fixing them myself, but I'm unsure if it's a desired behavior or not.

I had: config-file with 3 git repos WITHOUT revisions written in them. Yesterday I did a fresh "peru sync" and got the repos at master (but I didn't do a reup!). Today there were changes in the git-repos and I thought a "peru sync" would be enough, but it didn't update to the lastest revisions on the repositories, instead a "peru reup" and afterwards a "peru sync" got me the latest changes... shouldn't peru sync to head every time if there's no revision in the yaml-file? (btw, in the hg-plugin the value for not specified revisions is "default", shouldn't that be "tip"?)

[oconnor663]

Not a problem at all -- we don't get enough user feedback right now, so we love tickets. (Seriously.)

What you're seeing is by design, but it could be a bad design, so I want to get more of your thoughts. What peru does when it fetches a module is hash all the fields into a cache key, and put the fetched tree in cache under that key. Then the next time you sync, as long as you haven't changed any module fields, peru pulls that tree out of cache instead of doing a fetch. Peru actually doesn't know anything about url or rev; those are defined by the plugins, and peru treats them as opaque strings.

There are a few upsides to this default: Plugins get caching for free. The curl plugin doesn't download a big file every time you sync just to see whether it's changed. (curl could have implemented that for itself, to be fair, but we wouldn't want different plugins to be inconsistent about how caching works.) It also means that most syncs can be very fast, because peru doesn't have to run plugins at all, much less hit the network. Finally, there's some benefit to stability, even if you haven't specified a rev: if peru sync is part of your build, you probably don't want new and potentially broken dependency code every time you build, and you want your builds to keep working if your network goes down.

The downside is that people who expect the behavior you mentioned aren't getting what they want. Right now, the way to get that is to wipe out the cache directory (.peru/cache) before you sync, which isn't convenient at all. At some point we should definitely provide a flag like --skip-cache or --always-fetch or something, which pretends that all cache checks are misses. We could also define a module field like do not cache me: true that works for all types, which would have the same effect in a persistent way. (Though maybe it would be annoying to type that on all your modules? If you forgot one, it would be hard to spot. Currently there's no notion of global configs that apply to all modules, but we could invent something...) I think there was another issue filed for this, but I can't find it. Anyway, do any of those ideas sound like they would work well for you?

For the hg plugin, I'm totally inexperienced with Mercurial in general, so I might've done the wrong thing. But I think tip means "the most recent commit in the repo, on any branch." If that's true, that's probably not what we want, because then you won't know what branch you're fetching from. I think default is the Mercurial equivalent of master in git, is that right?

[questor]

okay, currently the workflow is like this:

• "peru sync" does a sync on master if it's running the first time, later without revisions in the config file it will check if cache and destination is different or do nothing. with a revision it will check if it's up-to-date and if not will bring in the new rev.
• "peru reup" will check ALL repos in the revision file for updates (does it also sync to the latest implicit?)

from my workflow I would like to have:

• a possibility to check for updates, but don't update the revisions in the yaml-file (sometimes if you have repositories depending on each other you don't want to update all repositories).
• I want to be able to check if the cache and the destination are the same or not (to find out if I have changed something in a synced repo; sync does complain if there were changes so functionality is already available)
• I want to be able to specify a module for reup to be able to select only some modules for updates instead of all

but this all doesn't help with the sync-stuff :) currently I'm thinking about the following workflow:

• "peru sync" always syncs to the revision from the yaml-config, if there is no revision it will not sync anything and complain!
• "peru init" as new command to bring in repositories without a revision in the yaml-config, but init will modify the yaml-file with the downloaded revision. if a repo already has a revision "init" will not touch it.

[questor]

the "specify module for reup" is related to this: #77

[oconnor663]

(Haven't forgotten about this issue, but I probably won't have time to think about it properly until next week.)

[oconnor663]

peru reup can currently take module names as arguments, for when you only want to reup a specific set of modules.

We should definitely add a --skip-cache flag or something like that, to force modules to sync the latest stuff.

[oconnor663]

I'm not sure I understood the last two workflow ideas you mentioned. If you want it to be an error when you try to sync unversioned modules, why not just run peru reup and give everything a version?

[questor]

of course it's hard to describe workflows without knowing if everything is handled or not....

so I try to write down my structures and what I want to do, maybe it's clearer that way:
first about the possibility to find out if there are updates or not:

• I have a peru.yaml with repo_util (taken from github) at master and on single file taken from repo_buildfiles also at master
• now some time is passing and repo_util on github got an update
• now I'd like to find out which repo's from my list got updates (repo_util in my case), but don't want to update because first I want to check what changes are done to the repo_util and if it breaks my code or not
• at the same time I want to check if I have done some changes to my local version of repo_util (and needs to be copied and checked in). I understood that peru doesn't can handle the copy or back-checkin and that's fine, but I want to be able to check if I have modified something or not.

regarding the last two points: my intension was that peru-files always should contain commit-ids to sync to and in the normal workflow peru will complain about missing ones (so that the user can check which commit-id he wants and can do it with a command not usually used during the normal workflow of the tool). If I run simply "reup" I will get "any" version that is just now available and I have to be careful that I specify which module I want to get the version from (if in a different module a new revision is available and I forgot to specify the module for reup I will get ALL changes). by separating "init" and "reup" you could try to separate the intention a user wanted to do.

does this sound sensible to you? :)

[oconnor663]

I'm not sure I'm following 100%, but it sounds like you have a couple different problems you want to solve, that you could probably solve in a different way now:

• How to force modules to include a commit id. The easiest way to do this is to make sure that whenever you add module foo to your peru.yaml file, you immediately run peru reup foo. That will give the new module a rev field, and then you can check it in. (We don't want to make it an error to leave out the rev, because some people prefer to always pull from master by default.)
• How to see which modules can be updated, without actually updating them. We should add a flag for this. Maybe peru reup --dry-run. But until then, the easiest thing to do is to just make sure your peru.yaml file is under version control. Then you can do a regular peru reup and look at all the changes, and if there are more than you want you just git checkout -- peru.yaml to undo the changes. You can then reup modules individually if that's needed.

[oconnor663]

Created #109 for the --dry-run feature. For other feature requests, create more smaller issues. I'm going to close this one for now.