Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Simplify migrations #326

Open
yajo opened this issue Jan 17, 2021 · 6 comments · May be fixed by #1510
Open

Simplify migrations #326

yajo opened this issue Jan 17, 2021 · 6 comments · May be fixed by #1510
Milestone
Later

Comments

@yajo
Copy link
Member

yajo commented Jan 17, 2021

The current migrations workflow is quite difficult to maintain.

The goal was to give full flexibility, but after some usage, it seems to make things more difficult.

Instead of having a migrations definition such as this:

_migrations:
  - version: v1.0.0
    before:
      - &mig
        - "[[ _copier_conf.src_path / 'migrations.py' ]]"
        - "[[ _copier_conf|tojson ]]"
    after:
      - *mig
  - version: v2
    before: [*mig]
    after:
      - *mig
      - "rm delete-in-migration-$VERSION_CURRENT.txt"

We should be having something like this:

_migrations:
  working_directory: "[[ _copier_conf.src_path ]]" # default value
  command: [poetry, run, migrations.py]

Then, all data needed for migrations will be passed as env variables, which that script can load and use to handle all of them comfortably.

At the end of the day, that code is easier to maintain in a migrations script, and this way we avoid the need to repeat the same definition once and again for each version (as seen in the 1st example using *mig).
@yajo yajo added this to the v6.0.0 milestone Jan 17, 2021
@yajo yajo linked a pull request Jan 17, 2021 that will close this issue
Refactor #314
Merged
@yajo yajo removed a link to a pull request Jan 17, 2021
Refactor #314
Merged
@pawamoy
Copy link
Contributor

pawamoy commented Jan 17, 2021

Just curious: what kind of commands do you need to run for every new version, and also before and after each update? Or is you script dealing with all these states, and most of the time not executing anything?

@yajo
Copy link
Member Author

yajo commented Jan 17, 2021

Or is you script dealing with all these states, and most of the time not executing anything?

Yes, that's what will happen most of the time

@pawamoy
Copy link
Contributor

pawamoy commented Jan 17, 2021

OK thanks. Indeed it might be easier to maintain a separate shell/Python script than cluttering copier.yml over time and new versions (even though technically one can get rid of old migrations in copier.yml at each new release, since the old migrations are still contained in the old versions themselves, thanks to git).

@yajo
Copy link
Member Author

yajo commented Jan 17, 2021

even though technically one can get rid of old migrations in copier.yml at each new release, since the old migrations are still contained in the old versions themselves, thanks to git

Beware with that... Migrations are loaded on the latest tag always, and all of them that apply are run sequentially, but Copier won't checkout each of them individually. Otherwise, you could never fix a bug in your migration script.

@yajo yajo mentioned this issue Jan 17, 2021
Open
@pawamoy
Copy link
Contributor

pawamoy commented Jan 17, 2021

Oh yes you're right, my bad. Then I'm definitely in favor of what you propose here.

@yajo yajo mentioned this issue Feb 10, 2021
Closed
4 tasks
@yajo yajo modified the milestones: v6.0.0, v6.1.0 Nov 20, 2021
@AdrianFreundQC
Copy link
Contributor

I'll take a look at implementing this.

@AdrianFreundQC AdrianFreundQC linked a pull request Feb 8, 2024 that will close this issue
Open
@sisp sisp mentioned this issue Feb 9, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
Later
Development

Successfully merging a pull request may close this issue.

Simplify migrations AdrianFreundQC/copier
3 participants
@yajo @pawamoy @AdrianFreundQC