Update from template?

[Nekroze]

It would be really neat if we could have a way to update the generated project with a newer version of the template.

By having separate templates maintained by various people, myself included, there will obviously be problems both introduced and solved over time. Because of this many or, if there was a serious problem, all of the projects generated from that template may need to be "regenerated". Support for this would definitely be an enhancement I believe.

However the idea of persevering the users current setup at the same time could be a challenge. For the basic values that cookiecutter needs to generate its templates, the json could just be written into the generated project as a dotfile or something to allow simple regeneration. However if some one has edited a generated file, not to replace the templates structure, but to add say, information about the project to the readme, it should be kept somehow.

I have not dug into the implementation details of cookiecutter and its code but if no one else wants to I may make a fork and attempt this in the future and submit a pull request if i can pull it off. However I would like to hear any thoughts on fixing the already edited file issue first.

As a possible extension to this update idea, maybe it could be possible to use the updating system to change a project from one base template to another... just an interesting thought, although admittedly of likely limited use..

Thanks

[Nekroze]

To get the ball rolling:

Probably an overly complex solution to the already edited problem. Perhaps if a project is generated using a git repo as the template, we can store the current commit hash and then just use the diff's between that commit hash and the head when updating. This would solve everything above i believe.

However it would add a fair bit more complexity to cookiecutter itself i think. Probably need to use some git/github api to retrieve the right diffs and apply them etc.

[audreyfeldroy]

As an end user and project template maintainer, I badly want this and have been wishing for it before you mentioned it :)

As Cookiecutter maintainer, I'm definitely nervous about the added complexity, but I encourage you to experiment.

I am open to adding dependencies. Also, I'm going to see if I can use https://github.com/codeinn/vcs or similar, to replace git/hg clone with something more generic. It might have some useful functionality for you. If you have a preferred package in mind, I am flexible about this.

I'm not sure if I'm re-describing your solution above or offering a different idea, but here's what comes to mind when I think about how the implementation might work:

• Upon first-time project generation, do an initial commit in the post-generate hook (dependent on Add pre_generate and post_generate hooks #29 & Hooks #48). Save not just the generated project, but the input context variables.
• To update from the project template:
  1. Diff the input context vars to see what has changed. (Deal with the changes somehow.)
  2. Re-generate the project with the original context vars.
  3. Diff the re-generated project and the first-time generated project.

[Nekroze]

Let me see if i understand your idea right. Also, I don't see why the original context variables would be changed and if they are I am guessing it wouldn't be that common so i will skip that thought for now as it isn't the main concern.

Generate the project from the template and then store a copy of it along with the context variables. Generate and store a new version of the project template generated from the stored context variables. Then just diff the new stored version against the older stored project. Then apply the diffs to the real in use project and delete the older stored project.

So we would be generating a skeleton of the project from the template and storing that skeleton for use in the update process alongside the project itself along with the context variables? And the project files the end user edits are just a copy of the generated skeleton. This way patches are just generated between the two skeletons and applied to the real project? Because if we didn't store the unedited skeletons and diffed a newly generated project from the original then everything the user edited would be diffed out. By diffing between two skeletons and then applying it to the full in use project then we can just apply the changes between the two skeleton generated templates. Rather then applying changes between a generated skeleton and the in use probably edited files which would update it but revert it back to being an empty skeleton.

This could work but i doubt many people would like the idea of having another, possible large, directory in their repo just to manage this update process.

What if we store the context variables and the commit id the project was generated from and then just iterate over each of the diffs from one commit to the next down the chain till the most recent. Rendering the diffs as a template before we apply each one so they are still project specific? This way the only thing we would need to store is a single file that has the context variables and the commit id/hash along side them. That is kind of what i was talking about.

It may be a bit harder to accomplish but it removes the need for a repository to track an entire skeleton directory. It would just need the json or whatever context variables of which only the commit id/hash would change ever. I can't see any logic problems with that, it should work fine like that right?

[audreyfeldroy]

I'd say try it your way (or whatever feels simplest as you're implementing), and see how it goes. You're probably right about the other directory getting in the way.

[Nekroze]

I might not have the time to start it this week as I have never done anything like that and it would take some time to learn these systems and how cookiecutter works before even trying an implementation. Add in that I have never really contributed like this to another repo... So no promises on a timescale at this point sorry. Just want to get it right.

[audreyfeldroy]

No pressure and don't worry - if you get around to it, great, otherwise I understand, that's how open source just is :)

That actually gives me some time to do cleanup this week and finish working on other pull requests.

[michaeljoseph]

@Nekroze please could have a look at #188 and see if it meets your needs?
Closing this discussion issue.