Errors while doing upgrade

[sfc-gh-pkommini]

I get the following error message when I try to setup tables using ECS. The easiest way I found was to do the dispatch database upgrade using a separate container and have the web and scheduler containers run only after that migrate container has finished execution.

Towards doing the database upgrade, I get the following errors:

Traceback (most recent call last):
--
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2336, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 364, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 778, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 495, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 140, in _do_get
self._dec_overflow()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 137, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 309, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 440, in __init__
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 661, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 656, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 493, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: FATAL:  database "dispatch_user" does not exist
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/dispatch", line 8, in <module>
sys.exit(entrypoint())
File "/usr/local/lib/python3.8/site-packages/dispatch/cli.py", line 569, in entrypoint
dispatch_cli()
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/dispatch/cli.py", line 302, in upgrade_database
if not database_exists(str(config.SQLALCHEMY_DATABASE_URI)):
File "/usr/local/lib/python3.8/site-packages/sqlalchemy_utils/functions/database.py", line 466, in database_exists
return bool(get_scalar_result(engine, text))
File "/usr/local/lib/python3.8/site-packages/sqlalchemy_utils/functions/database.py", line 445, in get_scalar_result
result_proxy = engine.execute(sql)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2234, in execute
connection = self._contextual_connect(close_with_result=True)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2302, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2339, in _wrap_pool_connect
Connection._handle_dbapi_exception_noconnection(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1583, in _handle_dbapi_exception_noconnection
util.raise_(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2336, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 364, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 778, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 495, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 140, in _do_get
self._dec_overflow()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 137, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 309, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 440, in __init__
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 661, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 656, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 493, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL:  database "dispatch_user" does not exist
(Background on this error at: http://sqlalche.me/e/13/e3q8)

Do you think this is a connection issue or is it a table setup issue? Are you familiar with this error?

[sfc-gh-pkommini]

Issue persists. Not sure why the database username that is passed is being used for the database name?

[kevgliss]

Is dispatch_user defined in your .env? I don't see it defined anywhere else so I'm not sure where that value is coming from.

[sfc-gh-pkommini]

dispatch_user is passed is via POSTGRES_USER and also via DATABASE_CREDENTIALS. But for some reason it says database "dispatch_user" doesn't exist

[sfc-gh-pkommini]

Seems like a case of this. Let me try this and report if the fix there works.

[sfc-gh-pkommini]

@kevgliss I believe this is an issue due to this. This issue might have to be moved to the dispatch repository due to being related to a recent version of python dependency SQLAlchemy-Utils==0.36.8.

As the sqlalchemy-utils 0.36.8 library onwards seems to have this error. There is a pull to fix it here. But until then the fix is to pin the library version SQLAlchemy-Utils==0.36.7 in requirements-base.txt.

Towards that, I was wondering is SQLAlchemy-Utils pulled in transitively? I don't see it in requirements-dev.txt file.

Hope this helps someone else facing the same error. Two days I'll never get back! 😅

[sfc-gh-pkommini]

I see it's pulled in transitively due in this library. I'll pin it while building our container for now and that should tape the issue for now.

[sfc-gh-pkommini]

@kevgliss @mvilanova

So I was able to run an ECS container with the fix above just to run the dispatch database upgrade command and I got the following logs:

The upgrade itself was successful, but it had a bunch of endpoint related errors that I wanted to understand. Would you know how to resolve them?

DEBUG:Configuring extensions...:/usr/local/lib/python3.8/site-packages/dispatch/extensions.py:configure_extensions:25
--
INFO:Attempting to load plugin: dispatch_basic_auth:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin dispatch_basic_auth:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: dispatch_contact:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin dispatch_contact:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: dispatch_document_resolver:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin dispatch_document_resolver:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: dispatch_participant_resolver:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin dispatch_participant_resolver:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: dispatch_pkce_auth:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin dispatch_pkce_auth:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: dispatch_ticket:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin dispatch_ticket:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: google_calendar_conference:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin google_calendar_conference:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: google_docs_document:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin google_docs_document:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: google_drive_storage:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin google_drive_storage:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: google_drive_task:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin google_drive_task:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: google_gmail_email:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin google_gmail_email:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: google_groups_participants:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin google_groups_participants:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: jira_ticket:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin jira_ticket:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: opsgenie_oncall:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin opsgenie_oncall:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: pagerduty_oncall:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin pagerduty_oncall:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: slack_contact:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin slack_contact:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: slack_conversation:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin slack_conversation:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
INFO:Attempting to load plugin: zoom_conference:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:27
ERROR:Failed to load plugin zoom_conference:Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py", line 29, in install_plugins
plugin = ep.load()
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load
self.require(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})
:/usr/local/lib/python3.8/site-packages/dispatch/common/utils/cli.py:install_plugins:39
DEBUG:Available Endpoints
Path                                         Authenticated    Methods
-------------------------------------------  ---------------  ---------
/auth/login                                  False            POST
/auth/register                               False            POST
/docs/openapi.json                           False            GET
/docs/                                       False            GET
/healthcheck                                 False            GET
/user/                                       True             GET
/user/{user_id}                              True             GET
/user/{user_id}                              True             PUT
/user/me                                     True             GET
/documents/                                  True             GET
/documents/{document_id}                     True             GET
/documents/                                  True             POST
/documents/{document_id}                     True             PUT
/documents/{document_id}                     True             DELETE
/tags/                                       True             GET
/tags/{tag_id}                               True             GET
/tags/                                       True             POST
/tags/{tag_id}                               True             PUT
/tags/{tag_id}                               True             DELETE
/tag_types/                                  True             GET
/tag_types/{tag_type_id}                     True             GET
/tag_types/                                  True             POST
/tag_types/{tag_type_id}                     True             PUT
/tag_types/{tag_type_id}                     True             DELETE
/services/                                   True             GET
/services/                                   True             POST
/services/{service_id}                       True             PUT
/services/{service_id}                       True             GET
/services/{service_id}                       True             DELETE
/teams/                                      True             GET
/teams/                                      True             POST
/teams/{team_id}                             True             GET
/teams/{team_contact_id}                     True             PUT
/teams/{team_contact_id}                     True             DELETE
/individuals/                                True             GET
/individuals/                                True             POST
/individuals/{individual_contact_id}         True             GET
/individuals/{individual_contact_id}         True             PUT
/individuals/{individual_contact_id}         True             DELETE
/policies/                                   True             GET
/policies/                                   True             POST
/policies/{policy_id}                        True             PUT
/policies/{policy_id}                        True             DELETE
/definitions/                                True             GET
/definitions/{definition_id}                 True             GET
/definitions/                                True             POST
/definitions/{definition_id}                 True             PUT
/definitions/{definition_id}                 True             DELETE
/terms/                                      True             GET
/terms/                                      True             POST
/terms/{term_id}                             True             GET
/terms/{term_id}                             True             PUT
/terms/{term_id}                             True             DELETE
/tasks/                                      True             GET
/tasks/                                      True             POST
/tasks/{task_id}                             True             PUT
/tasks/{task_id}                             True             DELETE
/search/                                     True             GET
/incidents/                                  True             GET
/incidents/{incident_id}                     True             GET
/incidents/                                  True             POST
/incidents/{incident_id}                     True             PUT
/incidents/{incident_id}/join                True             POST
/incidents/{incident_id}                     True             DELETE
/incidents/metric/forecast/{incident_type}   True             GET
/incident_types/                             True             GET
/incident_types/                             True             POST
/incident_types/{incident_type_id}           True             PUT
/incident_types/{incident_type_id}           True             GET
/incident_priorities/                        True             GET
/incident_priorities/                        True             POST
/incident_priorities/{incident_priority_id}  True             PUT
/incident_priorities/{incident_priority_id}  True             GET
/workflows/                                  True             GET
/workflows/{workflow_id}                     True             GET
/workflows/                                  True             POST
/workflows/{workflow_id}                     True             PUT
/workflows/{workflow_id}                     True             DELETE
/plugins/                                    True             GET
/plugins/{plugin_type}                       True             GET
/plugins/{plugin_id}                         True             GET
/plugins/{plugin_id}                         True             PUT
/feedback/                                   True             GET
/feedback/{feedback_id}                      True             GET
/feedback/                                   True             POST
/feedback/{feedback_id}                      True             PUT
/feedback/{feedback_id}                      True             DELETE:/usr/local/lib/python3.8/site-packages/dispatch/main.py:<module>:141
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running stamp_revision  -> 1c3f808aa00c
Success.

[kevgliss]

That issue with sqlalchemy-utils is a good catch (we've always had the database and the user match so we've never encountered it).

The second error regarding ContextualVersionConflict is typically due to some dependency deciding to pin their version and another library requiring a different incompatible version. In this case it looks like aiohttp is the culprit: aio-libs/aiohttp#5334

In the near term we can pin aiohttp, or attempt to figure out which library is requiring <chardet4.0 and see if they have an upgrade path.

[sfc-gh-pkommini]

Oh yeah, also wanted to mention, RDS uses dispatch as a keyword and hence can't have a database with that name. ☹️

So regarding the ContextualVersionConflict, Is it normal for dispatch.common.utils.cli's install_plugins to run during database upgrade?

[kevgliss]

Running install_plugins is expected to run during any CLI function. This really only registers the plugins in an internal repository so they can be used. Are you facing issues with it?

[sfc-gh-pkommini]

@kevgliss @mvilanova Yes. I was able to pin the version and then dispatch database upgrade worked like a charm, however I'm unable to register a new user. This was the sequence of events.

1. I downloaded the sample data dump and add the files to the database
2. I then did the equivalent of docker-compose up -d on ECS which was to spin up the ECS service and that is setup to run the dispatch database upgrade, upto this point everything worked without any errors.
3. Now I visited the http://url-endpoint/register and I get the following error:
   

Somehow it seems like the /register endpoint is not setup.

I'm using the latest code on master to run this and also data dump from the link in install.sh from this repo.
Not sure what I'm doing wrong. :(

[kevgliss]

Ah, we recently moved the register url to /auth/register I've updated the install.sh here: #96

Sorry about the confusion!

[sfc-gh-pkommini]

@kevgliss Ah. That makes sense. It is auth related. I'm happy to report that I was able to get as deep as setting up the data dump and got the endpoint up and running using ECS and RDS and was going to test integrations next.

I did get a few errors while doing the data dump, I'm not sure how critical they are, but I thought I'd let you know that I'm looking into them and any insights from you will be super useful.

psql:./dispatch-sample-data.dump:548: ERROR:  unrecognized configuration parameter "default_table_access_method"

psql:./dispatch-sample-data.dump:2457: ERROR:  invalid input syntax for integer: "Post Incident Review Template"
CONTEXT:  COPY document, line 4, column id: "Post Incident Review Template"

psql:./dispatch-sample-data.dump:4229: ERROR:  insert or update on table "incident_type" violates foreign key constraint "incident_type_template_document_id_fkey"
DETAIL:  Key (template_document_id)=(1) is not present in table "document".

[sfc-gh-pkommini]

@kevgliss The first one seems to be based on the postgres version so is an easy fix. But the 2nd and 3rd errors seem like they could break something.

[sfc-gh-pkommini]

@kevgliss Like I suspected. The 1st one doesn't occur on local (macOS) run of the install.sh. But the 2nd and 3rd error happen even on localhost.

[kevgliss]

I haven't been able to reproduce these errors on the latest example data dump (released today). Closing for now, please create a new issue if you encounter them again.