New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'-' chars in revision name do not allow upgrading #441

Closed
sqlalchemy-bot opened this Issue Aug 4, 2017 · 15 comments

Comments

Projects
None yet
2 participants
@sqlalchemy-bot

sqlalchemy-bot commented Aug 4, 2017

Migrated issue, originally created by Orestes Sanchez

I am afraid that using some chars as '-' and '@' on the names of a revision produces weird results.

I am using flask-migrate, but I think the problem is only alembic related.

I have created a bunch of migrations called "models-XXX_*", where XXX is a number. Here it is a listing of the migration files:

migrations/versions/models-001_backoffice.py
migrations/versions/models-002_device_not_unique.py
migrations/versions/models-003_add_tm_registration.py
migrations/versions/models-004_remove_is_active.py
migrations/versions/models-005_resize_fields.py

Say I am on version models-004_remove_is_active.py and I would like to migrate to models-005_resize_fields.py.

When performing an upgrade I get the following error:

orestesmac:snip-backoffice orestes$ 
FLASK_APP=$PWD/app.py flask db upgrade models-005_resize_fields
/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask_admin/model/base.py:1368: UserWarning: Fields missing from ruleset: current_login_at,last_login_ip,confirmed_at,current_login_ip,last_login_at,login_count
  warnings.warn(text)
Traceback (most recent call last):
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 348, in _revision_for_ident
    revision = self._revision_map[resolved_id]
KeyError: 'models'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 143, in _catch_revision_errors
    yield
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 333, in _upgrade_revs
    destination, current_rev, implicit_base=True)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 545, in iterate_revisions
    inclusive, assert_relative_length
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 490, in _relative_iterate
    inclusive=inclusive, implicit_base=implicit_base))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 652, in _iterate_revisions
    uppers = util.dedupe_tuple(self.get_revisions(upper))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 300, in get_revisions
    resolved_id, branch_label = self._resolve_revision_number(id_)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 441, in _resolve_revision_number
    current_head = self.get_current_head(branch_label)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 266, in get_current_head
    current_heads = self.filter_for_lineage(current_heads, branch_label)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 399, in filter_for_lineage
    tg for tg in targets
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 401, in <listcomp>
    tg, shares, include_dependencies=include_dependencies)]
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 415, in _shares_lineage
    in util.to_tuple(test_against_revs, default=())
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 414, in <listcomp>
    for test_against_rev
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 365, in _revision_for_ident
    ), resolved_id)
alembic.script.revision.ResolutionError: Multiple revisions start with 'models': 'models-002', 'models-005', 'models-004'...

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/orestes/Downloads/venv/py-backoffice/bin/flask", line 11, in <module>
    sys.exit(main())
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask/cli.py", line 507, in main
    cli.main(args=args, prog_name=name)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask/cli.py", line 374, in main
    return AppGroup.main(self, *args, **kwargs)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask/cli.py", line 251, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask_migrate/cli.py", line 134, in upgrade
    _upgrade(directory, revision, sql, tag, x_arg)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask_migrate/__init__.py", line 247, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/command.py", line 254, in upgrade
    script.run_env()
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 421, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/util/compat.py", line 64, in load_module_py
    module_id, path).load_module(module_id)
  File "<frozen importlib._bootstrap_external>", line 388, in _check_name_wrapper
  File "<frozen importlib._bootstrap_external>", line 809, in load_module
  File "<frozen importlib._bootstrap_external>", line 668, in load_module
  File "<frozen importlib._bootstrap>", line 268, in _load_module_shim
  File "<frozen importlib._bootstrap>", line 693, in _load
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 665, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "migrations/env.py", line 88, in <module>
    run_migrations_online()
  File "migrations/env.py", line 81, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/runtime/environment.py", line 817, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/runtime/migration.py", line 320, in run_migrations
    for step in self._migrations_fn(heads, self):
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/command.py", line 243, in upgrade
    return script._upgrade_revs(revision, rev)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 338, in _upgrade_revs
    for script in reversed(list(revs))
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 174, in _catch_revision_errors
    compat.raise_from_cause(util.CommandError(resolution))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/util/compat.py", line 117, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=exc_value)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/util/compat.py", line 110, in reraise
    raise value.with_traceback(tb)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 143, in _catch_revision_errors
    yield
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 333, in _upgrade_revs
    destination, current_rev, implicit_base=True)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 545, in iterate_revisions
    inclusive, assert_relative_length
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 490, in _relative_iterate
    inclusive=inclusive, implicit_base=implicit_base))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 652, in _iterate_revisions
    uppers = util.dedupe_tuple(self.get_revisions(upper))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 300, in get_revisions
    resolved_id, branch_label = self._resolve_revision_number(id_)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 441, in _resolve_revision_number
    current_head = self.get_current_head(branch_label)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 266, in get_current_head
    current_heads = self.filter_for_lineage(current_heads, branch_label)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 399, in filter_for_lineage
    tg for tg in targets
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 401, in <listcomp>
    tg, shares, include_dependencies=include_dependencies)]
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 415, in _shares_lineage
    in util.to_tuple(test_against_revs, default=())
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 414, in <listcomp>
    for test_against_rev
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 365, in _revision_for_ident
    ), resolved_id)
alembic.util.exc.CommandError: Can't locate revision identified by 'models'
(py-backoffice) orestesmac:snip-backoffice orestes$ 

It looks like that it breaks the name of a revision into pieces using the following RE:

_relative_destination = re.compile(r'(?:(.+?)@)?(\w+)?((?:\+|-)\d+)')

I tried to find the location in the documentation that describes any constraint on the name of revisions, but I didn't find it. I suggest updating the doc to describe this constraint.

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Aug 4, 2017

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Aug 4, 2017

Michael Bayer (@zzzeek) wrote:

Raise if manual revision id contains dashes or at signs

A :class:.CommandError is raised if the "--rev-id" passed to the
:func:.revision command contains dashes or at-signs, as this interferes
with the command notation used to locate revisions.

Change-Id: I60a794a5c80bf47b149998b8c5cb04ecbfd05bfa
Fixes: #441

2645faf

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Aug 4, 2017

Changes by Michael Bayer (@zzzeek):

  • changed status to closed
@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Aug 5, 2017

Orestes Sanchez wrote:

Thank you!!! So fast!

I think it is not only "@" and "-", but also "+". In fact, all the special chars that are used as delimiters in the regular expression. What do you think?

_relative_destination = re.compile(r'(?:(.+?)@)?(\w+)?((?:\+|-)\d+)')
@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Aug 9, 2017

Michael Bayer (@zzzeek) wrote:

yeah. arg. i released and forgot to get to that.

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Aug 9, 2017

Michael Bayer (@zzzeek) wrote:

add @ to the list and try again

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Aug 9, 2017

Changes by Michael Bayer (@zzzeek):

  • changed status to reopened
@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Aug 9, 2017

Michael Bayer (@zzzeek) wrote:

der add + that is. When i looked at the reg i didn't see the "" there.

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Aug 10, 2017

Idris Yusupov wrote:

I have the following format rX_Task-Y_descr.py.
Adding - restriction in migrations name broke my migrations in 0.9.5.

So I have to downgrade to 0.9.4 😧

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Aug 10, 2017

Michael Bayer (@zzzeek) wrote:

why cant you change the name of your migration? the filename can stay as is, you just need to change the name inside the file itself

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Aug 10, 2017

Michael Bayer (@zzzeek) wrote:

also...how did that name even work when I can reproduce this bug with any name that has a dash in it

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Aug 10, 2017

Orestes Sanchez wrote:

I also had migration names with a dash before realizing about this problem, but I didn't use the name to upgrade or downgrade, I run a full

alembic upgrade

So if you run an upgrade without any name, then alembic does not complain, but when I tried to use one of those dashed revision names, it broke.

alembic upgrade models-005

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 16, 2017

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Nov 27, 2017

Michael Bayer (@zzzeek) wrote:

this was merged in b6a0ba0, the commit message refers to the wrong issue number.

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Nov 27, 2017

Changes by Michael Bayer (@zzzeek):

  • changed status to closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment