'-' chars in revision name do not allow upgrading #441

sqlalchemy-bot · 2017-08-04T16:19:58Z

Migrated issue, originally created by Orestes Sanchez

I am afraid that using some chars as '-' and '@' on the names of a revision produces weird results.

I am using flask-migrate, but I think the problem is only alembic related.

I have created a bunch of migrations called "models-XXX_*", where XXX is a number. Here it is a listing of the migration files:

migrations/versions/models-001_backoffice.py
migrations/versions/models-002_device_not_unique.py
migrations/versions/models-003_add_tm_registration.py
migrations/versions/models-004_remove_is_active.py
migrations/versions/models-005_resize_fields.py

Say I am on version models-004_remove_is_active.py and I would like to migrate to models-005_resize_fields.py.

When performing an upgrade I get the following error:

orestesmac:snip-backoffice orestes$ 
FLASK_APP=$PWD/app.py flask db upgrade models-005_resize_fields
/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask_admin/model/base.py:1368: UserWarning: Fields missing from ruleset: current_login_at,last_login_ip,confirmed_at,current_login_ip,last_login_at,login_count
  warnings.warn(text)
Traceback (most recent call last):
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 348, in _revision_for_ident
    revision = self._revision_map[resolved_id]
KeyError: 'models'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 143, in _catch_revision_errors
    yield
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 333, in _upgrade_revs
    destination, current_rev, implicit_base=True)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 545, in iterate_revisions
    inclusive, assert_relative_length
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 490, in _relative_iterate
    inclusive=inclusive, implicit_base=implicit_base))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 652, in _iterate_revisions
    uppers = util.dedupe_tuple(self.get_revisions(upper))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 300, in get_revisions
    resolved_id, branch_label = self._resolve_revision_number(id_)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 441, in _resolve_revision_number
    current_head = self.get_current_head(branch_label)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 266, in get_current_head
    current_heads = self.filter_for_lineage(current_heads, branch_label)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 399, in filter_for_lineage
    tg for tg in targets
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 401, in <listcomp>
    tg, shares, include_dependencies=include_dependencies)]
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 415, in _shares_lineage
    in util.to_tuple(test_against_revs, default=())
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 414, in <listcomp>
    for test_against_rev
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 365, in _revision_for_ident
    ), resolved_id)
alembic.script.revision.ResolutionError: Multiple revisions start with 'models': 'models-002', 'models-005', 'models-004'...

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/orestes/Downloads/venv/py-backoffice/bin/flask", line 11, in <module>
    sys.exit(main())
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask/cli.py", line 507, in main
    cli.main(args=args, prog_name=name)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask/cli.py", line 374, in main
    return AppGroup.main(self, *args, **kwargs)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask/cli.py", line 251, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask_migrate/cli.py", line 134, in upgrade
    _upgrade(directory, revision, sql, tag, x_arg)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask_migrate/__init__.py", line 247, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/command.py", line 254, in upgrade
    script.run_env()
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 421, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/util/compat.py", line 64, in load_module_py
    module_id, path).load_module(module_id)
  File "<frozen importlib._bootstrap_external>", line 388, in _check_name_wrapper
  File "<frozen importlib._bootstrap_external>", line 809, in load_module
  File "<frozen importlib._bootstrap_external>", line 668, in load_module
  File "<frozen importlib._bootstrap>", line 268, in _load_module_shim
  File "<frozen importlib._bootstrap>", line 693, in _load
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 665, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "migrations/env.py", line 88, in <module>
    run_migrations_online()
  File "migrations/env.py", line 81, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/runtime/environment.py", line 817, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/runtime/migration.py", line 320, in run_migrations
    for step in self._migrations_fn(heads, self):
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/command.py", line 243, in upgrade
    return script._upgrade_revs(revision, rev)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 338, in _upgrade_revs
    for script in reversed(list(revs))
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 174, in _catch_revision_errors
    compat.raise_from_cause(util.CommandError(resolution))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/util/compat.py", line 117, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=exc_value)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/util/compat.py", line 110, in reraise
    raise value.with_traceback(tb)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 143, in _catch_revision_errors
    yield
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 333, in _upgrade_revs
    destination, current_rev, implicit_base=True)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 545, in iterate_revisions
    inclusive, assert_relative_length
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 490, in _relative_iterate
    inclusive=inclusive, implicit_base=implicit_base))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 652, in _iterate_revisions
    uppers = util.dedupe_tuple(self.get_revisions(upper))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 300, in get_revisions
    resolved_id, branch_label = self._resolve_revision_number(id_)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 441, in _resolve_revision_number
    current_head = self.get_current_head(branch_label)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 266, in get_current_head
    current_heads = self.filter_for_lineage(current_heads, branch_label)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 399, in filter_for_lineage
    tg for tg in targets
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 401, in <listcomp>
    tg, shares, include_dependencies=include_dependencies)]
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 415, in _shares_lineage
    in util.to_tuple(test_against_revs, default=())
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 414, in <listcomp>
    for test_against_rev
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 365, in _revision_for_ident
    ), resolved_id)
alembic.util.exc.CommandError: Can't locate revision identified by 'models'
(py-backoffice) orestesmac:snip-backoffice orestes$

It looks like that it breaks the name of a revision into pieces using the following RE:

_relative_destination = re.compile(r'(?:(.+?)@)?(\w+)?((?:\+|-)\d+)')

I tried to find the location in the documentation that describes any constraint on the name of revisions, but I didn't find it. I suggest updating the doc to describe this constraint.

The text was updated successfully, but these errors were encountered:

sqlalchemy-bot · 2017-08-04T17:35:28Z

Michael Bayer (@zzzeek) wrote:

https://gerrit.sqlalchemy.org/466

sqlalchemy-bot · 2017-08-04T18:33:51Z

Michael Bayer (@zzzeek) wrote:

Raise if manual revision id contains dashes or at signs

A :class:.CommandError is raised if the "--rev-id" passed to the
:func:.revision command contains dashes or at-signs, as this interferes
with the command notation used to locate revisions.

Change-Id: I60a794a5c80bf47b149998b8c5cb04ecbfd05bfa
Fixes: #441

→ 2645faf

sqlalchemy-bot · 2017-08-04T18:33:51Z

Changes by Michael Bayer (@zzzeek):

changed status to closed

sqlalchemy-bot · 2017-08-05T09:26:21Z

Orestes Sanchez wrote:

Thank you!!! So fast!

I think it is not only "@" and "-", but also "+". In fact, all the special chars that are used as delimiters in the regular expression. What do you think?

_relative_destination = re.compile(r'(?:(.+?)@)?(\w+)?((?:\+|-)\d+)')

sqlalchemy-bot · 2017-08-09T18:39:05Z

Michael Bayer (@zzzeek) wrote:

yeah. arg. i released and forgot to get to that.

sqlalchemy-bot · 2017-08-09T18:39:21Z

Michael Bayer (@zzzeek) wrote:

add @ to the list and try again

sqlalchemy-bot · 2017-08-09T18:39:21Z

Changes by Michael Bayer (@zzzeek):

changed status to reopened

sqlalchemy-bot · 2017-08-09T18:39:58Z

Michael Bayer (@zzzeek) wrote:

der add + that is. When i looked at the reg i didn't see the "" there.

sqlalchemy-bot · 2017-08-10T12:13:59Z

Idris Yusupov wrote:

I have the following format rX_Task-Y_descr.py.
Adding - restriction in migrations name broke my migrations in 0.9.5.

So I have to downgrade to 0.9.4 😧

sqlalchemy-bot · 2017-08-10T14:37:58Z

Michael Bayer (@zzzeek) wrote:

why cant you change the name of your migration? the filename can stay as is, you just need to change the name inside the file itself

sqlalchemy-bot · 2017-08-10T14:39:32Z

Michael Bayer (@zzzeek) wrote:

also...how did that name even work when I can reproduce this bug with any name that has a dash in it

sqlalchemy-bot · 2017-08-10T19:38:00Z

Orestes Sanchez wrote:

I also had migration names with a dash before realizing about this problem, but I didn't use the name to upgrade or downgrade, I run a full

alembic upgrade

So if you run an upgrade without any name, then alembic does not complain, but when I tried to use one of those dashed revision names, it broke.

alembic upgrade models-005

sqlalchemy-bot · 2017-09-16T00:26:56Z

Michael Bayer (@zzzeek) wrote:

https://gerrit.sqlalchemy.org/520

sqlalchemy-bot · 2017-11-27T22:40:16Z

Michael Bayer (@zzzeek) wrote:

this was merged in b6a0ba0, the commit message refers to the wrong issue number.

sqlalchemy-bot · 2017-11-27T22:40:26Z

Changes by Michael Bayer (@zzzeek):

changed status to closed

sqlalchemy-bot closed this as completed Nov 27, 2017

sqlalchemy-bot added documentation bug Something isn't working labels Nov 27, 2018

zzzeek removed low priority labels Nov 30, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

'-' chars in revision name do not allow upgrading #441

'-' chars in revision name do not allow upgrading #441

sqlalchemy-bot commented Aug 4, 2017

sqlalchemy-bot commented Aug 4, 2017

sqlalchemy-bot commented Aug 4, 2017

sqlalchemy-bot commented Aug 4, 2017

sqlalchemy-bot commented Aug 5, 2017

sqlalchemy-bot commented Aug 9, 2017

sqlalchemy-bot commented Aug 9, 2017

sqlalchemy-bot commented Aug 9, 2017

sqlalchemy-bot commented Aug 9, 2017

sqlalchemy-bot commented Aug 10, 2017

sqlalchemy-bot commented Aug 10, 2017

sqlalchemy-bot commented Aug 10, 2017

sqlalchemy-bot commented Aug 10, 2017

sqlalchemy-bot commented Sep 16, 2017

sqlalchemy-bot commented Nov 27, 2017

sqlalchemy-bot commented Nov 27, 2017

'-' chars in revision name do not allow upgrading #441

'-' chars in revision name do not allow upgrading #441

Comments

sqlalchemy-bot commented Aug 4, 2017

sqlalchemy-bot commented Aug 4, 2017

sqlalchemy-bot commented Aug 4, 2017

sqlalchemy-bot commented Aug 4, 2017

sqlalchemy-bot commented Aug 5, 2017

sqlalchemy-bot commented Aug 9, 2017

sqlalchemy-bot commented Aug 9, 2017

sqlalchemy-bot commented Aug 9, 2017

sqlalchemy-bot commented Aug 9, 2017

sqlalchemy-bot commented Aug 10, 2017

sqlalchemy-bot commented Aug 10, 2017

sqlalchemy-bot commented Aug 10, 2017

sqlalchemy-bot commented Aug 10, 2017

sqlalchemy-bot commented Sep 16, 2017

sqlalchemy-bot commented Nov 27, 2017

sqlalchemy-bot commented Nov 27, 2017