'-' chars in revision name do not allow upgrading

[sqlalchemy-bot]

Migrated issue, originally created by Orestes Sanchez

I am afraid that using some chars as '-' and '@' on the names of a revision produces weird results.

I am using flask-migrate, but I think the problem is only alembic related.

I have created a bunch of migrations called "models-XXX_*", where XXX is a number. Here it is a listing of the migration files:

migrations/versions/models-001_backoffice.py
migrations/versions/models-002_device_not_unique.py
migrations/versions/models-003_add_tm_registration.py
migrations/versions/models-004_remove_is_active.py
migrations/versions/models-005_resize_fields.py

Say I am on version models-004_remove_is_active.py and I would like to migrate to models-005_resize_fields.py.

When performing an upgrade I get the following error:

orestesmac:snip-backoffice orestes$ 
FLASK_APP=$PWD/app.py flask db upgrade models-005_resize_fields
/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask_admin/model/base.py:1368: UserWarning: Fields missing from ruleset: current_login_at,last_login_ip,confirmed_at,current_login_ip,last_login_at,login_count
  warnings.warn(text)
Traceback (most recent call last):
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 348, in _revision_for_ident
    revision = self._revision_map[resolved_id]
KeyError: 'models'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 143, in _catch_revision_errors
    yield
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 333, in _upgrade_revs
    destination, current_rev, implicit_base=True)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 545, in iterate_revisions
    inclusive, assert_relative_length
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 490, in _relative_iterate
    inclusive=inclusive, implicit_base=implicit_base))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 652, in _iterate_revisions
    uppers = util.dedupe_tuple(self.get_revisions(upper))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 300, in get_revisions
    resolved_id, branch_label = self._resolve_revision_number(id_)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 441, in _resolve_revision_number
    current_head = self.get_current_head(branch_label)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 266, in get_current_head
    current_heads = self.filter_for_lineage(current_heads, branch_label)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 399, in filter_for_lineage
    tg for tg in targets
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 401, in <listcomp>
    tg, shares, include_dependencies=include_dependencies)]
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 415, in _shares_lineage
    in util.to_tuple(test_against_revs, default=())
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 414, in <listcomp>
    for test_against_rev
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 365, in _revision_for_ident
    ), resolved_id)
alembic.script.revision.ResolutionError: Multiple revisions start with 'models': 'models-002', 'models-005', 'models-004'...

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/orestes/Downloads/venv/py-backoffice/bin/flask", line 11, in <module>
    sys.exit(main())
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask/cli.py", line 507, in main
    cli.main(args=args, prog_name=name)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask/cli.py", line 374, in main
    return AppGroup.main(self, *args, **kwargs)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask/cli.py", line 251, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask_migrate/cli.py", line 134, in upgrade
    _upgrade(directory, revision, sql, tag, x_arg)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/flask_migrate/__init__.py", line 247, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/command.py", line 254, in upgrade
    script.run_env()
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 421, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/util/compat.py", line 64, in load_module_py
    module_id, path).load_module(module_id)
  File "<frozen importlib._bootstrap_external>", line 388, in _check_name_wrapper
  File "<frozen importlib._bootstrap_external>", line 809, in load_module
  File "<frozen importlib._bootstrap_external>", line 668, in load_module
  File "<frozen importlib._bootstrap>", line 268, in _load_module_shim
  File "<frozen importlib._bootstrap>", line 693, in _load
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 665, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "migrations/env.py", line 88, in <module>
    run_migrations_online()
  File "migrations/env.py", line 81, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/runtime/environment.py", line 817, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/runtime/migration.py", line 320, in run_migrations
    for step in self._migrations_fn(heads, self):
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/command.py", line 243, in upgrade
    return script._upgrade_revs(revision, rev)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 338, in _upgrade_revs
    for script in reversed(list(revs))
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 174, in _catch_revision_errors
    compat.raise_from_cause(util.CommandError(resolution))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/util/compat.py", line 117, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=exc_value)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/util/compat.py", line 110, in reraise
    raise value.with_traceback(tb)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 143, in _catch_revision_errors
    yield
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/base.py", line 333, in _upgrade_revs
    destination, current_rev, implicit_base=True)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 545, in iterate_revisions
    inclusive, assert_relative_length
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 490, in _relative_iterate
    inclusive=inclusive, implicit_base=implicit_base))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 652, in _iterate_revisions
    uppers = util.dedupe_tuple(self.get_revisions(upper))
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 300, in get_revisions
    resolved_id, branch_label = self._resolve_revision_number(id_)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 441, in _resolve_revision_number
    current_head = self.get_current_head(branch_label)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 266, in get_current_head
    current_heads = self.filter_for_lineage(current_heads, branch_label)
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 399, in filter_for_lineage
    tg for tg in targets
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 401, in <listcomp>
    tg, shares, include_dependencies=include_dependencies)]
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 415, in _shares_lineage
    in util.to_tuple(test_against_revs, default=())
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 414, in <listcomp>
    for test_against_rev
  File "/Users/orestes/Downloads/venv/py-backoffice/lib/python3.5/site-packages/alembic/script/revision.py", line 365, in _revision_for_ident
    ), resolved_id)
alembic.util.exc.CommandError: Can't locate revision identified by 'models'
(py-backoffice) orestesmac:snip-backoffice orestes$ 

It looks like that it breaks the name of a revision into pieces using the following RE:

_relative_destination = re.compile(r'(?:(.+?)@)?(\w+)?((?:\+|-)\d+)')

I tried to find the location in the documentation that describes any constraint on the name of revisions, but I didn't find it. I suggest updating the doc to describe this constraint.

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

https://gerrit.sqlalchemy.org/466

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

Raise if manual revision id contains dashes or at signs

A :class:.CommandError is raised if the "--rev-id" passed to the
:func:.revision command contains dashes or at-signs, as this interferes
with the command notation used to locate revisions.

Change-Id: I60a794a5c80bf47b149998b8c5cb04ecbfd05bfa
Fixes: #441

→ 2645faf

[sqlalchemy-bot]

Changes by Michael Bayer (@zzzeek):

• changed status to closed

[sqlalchemy-bot]

Orestes Sanchez wrote:

Thank you!!! So fast!

I think it is not only "@" and "-", but also "+". In fact, all the special chars that are used as delimiters in the regular expression. What do you think?

_relative_destination = re.compile(r'(?:(.+?)@)?(\w+)?((?:\+|-)\d+)')

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

yeah. arg. i released and forgot to get to that.

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

add @ to the list and try again

[sqlalchemy-bot]

Changes by Michael Bayer (@zzzeek):

• changed status to reopened

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

der add + that is. When i looked at the reg i didn't see the "" there.

[sqlalchemy-bot]

Idris Yusupov wrote:

I have the following format rX_Task-Y_descr.py.
Adding - restriction in migrations name broke my migrations in 0.9.5.

So I have to downgrade to 0.9.4 😧

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

why cant you change the name of your migration? the filename can stay as is, you just need to change the name inside the file itself

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

also...how did that name even work when I can reproduce this bug with any name that has a dash in it

[sqlalchemy-bot]

Orestes Sanchez wrote:

I also had migration names with a dash before realizing about this problem, but I didn't use the name to upgrade or downgrade, I run a full

alembic upgrade

So if you run an upgrade without any name, then alembic does not complain, but when I tried to use one of those dashed revision names, it broke.

alembic upgrade models-005

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

https://gerrit.sqlalchemy.org/520

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

this was merged in b6a0ba0, the commit message refers to the wrong issue number.

[sqlalchemy-bot]

Changes by Michael Bayer (@zzzeek):

• changed status to closed