Stuck at "AWX is Upgrading" page from hours

[jains99]

Everything installed smoothly but I am stuck at "AWX is Upgrading" page from last 2 hours. I have tried refreshing multiple times. How to check what is wrong?

Page Message -

AWX is Upgrading
AWX is currently upgrading or installing, this page will refresh when done.

[sasskinn12]

Same issue here...

[pmozdzynski]

I had same issue ,tried to remove the link and open in another browser or private browsing.
Success!!

[geerlingguy]

@jains99 - What OS are you using? It could be that the shared volume is causing a slowdown, or something else on the system. If you run docker-compose logs -f does it give any output during that time?

[jains99]

@geerlingguy I was on centOS 7.3. I didn't check the docker logs at that time. I will try this again in few days and update.

[devashishmeher2205]

Hi

Stuck with the same issue from last 1.5 hours , any leads on this issue as in how was it solved ?

Regards

[v1k0d3n]

was getting the same thing. @geerlingguy did you ever figure out what's causing this?

[stephrobert]

postgres_1 | ERROR: relation "conf_setting" does not exist at character 158

[moltar]

Same issue as @stephrobert here...

[ksator]

same issue here ...

[marcosgeorgopoulos]

I have the same issue

[chazzly]

Same issue here - "awx_postgres | ERROR: relation "conf_setting" does not exist at character 158". Did anyone ever find a solution?

Running on CentOS Linux release 7.3.1611

This is on a fresh install of a new VM, so I was not attempting to "Upgrade".

I actually ran this playbook successfully a few months ago on a clone from the same template (VMWare) and did not run into this problem. Haven't found what's different, yet.

[chazzly]

ran docker-compose logs -f and found the following:

awx_postgres | ERROR: relation "conf_setting" does not exist at character 158
awx_postgres | STATEMENT: SELECT "conf_setting"."id", "conf_setting"."created", "conf_setting"."modified", "conf_setting"."key", "conf_setting"."value", "conf_setting"."user_id" FROM "conf_setting" WHERE ("conf_setting"."key" IN ('ACTIVI
TY_STREAM_ENABLED', 'ACTIVITY_STREAM_ENABLED_FOR_INVENTORY_SYNC', 'ORG_ADMINS_CAN_SEE_ALL_USERS', 'MANAGE_ORGANIZATION_AUTH', 'TOWER_ADMIN_ALERTS', 'TOWER_URL_BASE', 'REMOTE_HOST_HEADERS', 'PROXY_IP_WHITELIST', 'LICENSE', 'INSTALL_UUID',
'CUSTOM_VENV_PATHS', 'AD_HOC_COMMANDS', 'ALLOW_JINJA_IN_EXTRA_VARS', 'AWX_PROOT_BASE_PATH', 'AWX_PROOT_HIDE_PATHS', 'AWX_PROOT_SHOW_PATHS', 'AWX_ISOLATED_VERBOSITY', 'AWX_ISOLATED_CHECK_INTERVAL', 'AWX_ISOLATED_LAUNCH_TIMEOUT', 'AWX_ISOL$
TED_CONNECTION_TIMEOUT', 'AWX_ISOLATED_KEY_GENERATION', 'AWX_ISOLATED_PRIVATE_KEY', 'AWX_ISOLATED_PUBLIC_KEY', 'AWX_TASK_ENV', 'INSIGHTS_TRACKING_STATE', 'AWX_ROLES_ENABLED', 'STDOUT_MAX_BYTES_DISPLAY', 'EVENT_STDOUT_MAX_BYTES_DISPLAY', $
SCHEDULE_MAX_JOBS', 'AWX_ANSIBLE_CALLBACK_PLUGINS', 'DEFAULT_JOB_TIMEOUT', 'DEFAULT_INVENTORY_UPDATE_TIMEOUT', 'DEFAULT_PROJECT_UPDATE_TIMEOUT', 'ANSIBLE_FACT_CACHE_TIMEOUT', 'LOG_AGGREGATOR_HOST', 'LOG_AGGREGATOR_PORT', 'LOG_AGGREGATOR_$
YPE', 'LOG_AGGREGATOR_USERNAME', 'LOG_AGGREGATOR_PASSWORD', 'LOG_AGGREGATOR_LOGGERS', 'LOG_AGGREGATOR_INDIVIDUAL_FACTS', 'LOG_AGGREGATOR_ENABLED', 'LOG_AGGREGATOR_TOWER_UUID', 'LOG_AGGREGATOR_PROTOCOL', 'LOG_AGGREGATOR_TCP_TIMEOUT', 'LOG$
AGGREGATOR_VERIFY_CERT', 'LOG_AGGREGATOR_LEVEL', 'BROKER_DURABILITY', 'SESSION_COOKIE_AGE', 'SESSIONS_PER_USER', 'AUTH_BASIC_ENABLED', 'OAUTH2_PROVIDER', 'ALLOW_OAUTH2_FOR_EXTERNAL_USERS', 'PENDO_TRACKING_STATE', 'CUSTOM_LOGIN_INFO', 'CU$
TOM_LOGO', 'MAX_UI_JOB_EVENTS', 'UI_LIVE_UPDATES_ENABLED', 'SOCIAL_AUTH_ORGANIZATION_MAP', 'SOCIAL_AUTH_TEAM_MAP', 'SOCIAL_AUTH_USER_FIELDS', 'AUTH_LDAP_SERVER_URI', 'AUTH_LDAP_BIND_DN', 'AUTH_LDAP_BIND_PASSWORD', 'AUTH_LDAP_START_TLS', $
AUTH_LDAP_CONNECTION_OPTIONS', 'AUTH_LDAP_USER_SEARCH', 'AUTH_LDAP_USER_DN_TEMPLATE', 'AUTH_LDAP_USER_ATTR_MAP', 'AUTH_LDAP_GROUP_SEARCH', 'AUTH_LDAP_GROUP_TYPE', 'AUTH_LDAP_GROUP_TYPE_PARAMS', 'AUTH_LDAP_REQUIRE_GROUP', 'AUTH_LDAP_DENY_$
ROUP', 'AUTH_LDAP_USER_FLAGS_BY_GROUP', 'AUTH_LDAP_ORGANIZATION_MAP', 'AUTH_LDAP_TEAM_MAP', 'AUTH_LDAP_1_SERVER_URI', 'AUTH_LDAP_1_BIND_DN', 'AUTH_LDAP_1_BIND_PASSWORD', 'AUTH_LDAP_1_START_TLS', 'AUTH_LDAP_1_CONNECTION_OPTIONS', 'AUTH_LD$
P_1_USER_SEARCH', 'AUTH_LDAP_1_USER_DN_TEMPLATE', 'AUTH_LDAP_1_USER_ATTR_MAP', 'AUTH_LDAP_1_GROUP_SEARCH', 'AUTH_LDAP_1_GROUP_TYPE', 'AUTH_LDAP_1_GROUP_TYPE_PARAMS', 'AUTH_LDAP_1_REQUIRE_GROUP', 'AUTH_LDAP_1_DENY_GROUP', 'AUTH_LDAP_1_USE$
_FLAGS_BY_GROUP', 'AUTH_LDAP_1_ORGANIZATION_MAP', 'AUTH_LDAP_1_TEAM_MAP', 'AUTH_LDAP_2_SERVER_URI', 'AUTH_LDAP_2_BIND_DN', 'AUTH_LDAP_2_BIND_PASSWORD', 'AUTH_LDAP_2_START_TLS', 'AUTH_LDAP_2_CONNECTION_OPTIONS', 'AUTH_LDAP_2_USER_SEARCH',
'AUTH_LDAP_2_USER_DN_TEMPLATE', 'AUTH_LDAP_2_USER_ATTR_MAP', 'AUTH_LDAP_2_GROUP_SEARCH', 'AUTH_LDAP_2_GROUP_TYPE', 'AUTH_LDAP_2_GROUP_TYPE_PARAMS', 'AUTH_LDAP_2_REQUIRE_GROUP', 'AUTH_LDAP_2_DENY_GROUP', 'AUTH_LDAP_2_USER_FLAGS_BY_GROUP',
'AUTH_LDAP_2_ORGANIZATION_MAP', 'AUTH_LDAP_2_TEAM_MAP', 'AUTH_LDAP_3_SERVER_URI', 'AUTH_LDAP_3_BIND_DN', 'AUTH_LDAP_3_BIND_PASSWORD', 'AUTH_LDAP_3_START_TLS', 'AUTH_LDAP_3_CONNECTION_OPTIONS', 'AUTH_LDAP_3_USER_SEARCH', 'AUTH_LDAP_3_USER$
DN_TEMPLATE', 'AUTH_LDAP_3_USER_ATTR_MAP', 'AUTH_LDAP_3_GROUP_SEARCH', 'AUTH_LDAP_3_GROUP_TYPE', 'AUTH_LDAP_3_GROUP_TYPE_PARAMS', 'AUTH_LDAP_3_REQUIRE_GROUP', 'AUTH_LDAP_3_DENY_GROUP', 'AUTH_LDAP_3_USER_FLAGS_BY_GROUP', 'AUTH_LDAP_3_ORGA$
IZATION_MAP', 'AUTH_LDAP_3_TEAM_MAP', 'AUTH_LDAP_4_SERVER_URI', 'AUTH_LDAP_4_BIND_DN', 'AUTH_LDAP_4_BIND_PASSWORD', 'AUTH_LDAP_4_START_TLS', 'AUTH_LDAP_4_CONNECTION_OPTIONS', 'AUTH_LDAP_4_USER_SEARCH', 'AUTH_LDAP_4_USER_DN_TEMPLATE', 'AU$
H_LDAP_4_USER_ATTR_MAP', 'AUTH_LDAP_4_GROUP_SEARCH', 'AUTH_LDAP_4_GROUP_TYPE', 'AUTH_LDAP_4_GROUP_TYPE_PARAMS', 'AUTH_LDAP_4_REQUIRE_GROUP', 'AUTH_LDAP_4_DENY_GROUP', 'AUTH_LDAP_4_USER_FLAGS_BY_GROUP', 'AUTH_LDAP_4_ORGANIZATION_MAP', 'AU$
H_LDAP_4_TEAM_MAP', 'AUTH_LDAP_5_SERVER_URI', 'AUTH_LDAP_5_BIND_DN', 'AUTH_LDAP_5_BIND_PASSWORD', 'AUTH_LDAP_5_START_TLS', 'AUTH_LDAP_5_CONNECTION_OPTIONS', 'AUTH_LDAP_5_USER_SEARCH', 'AUTH_LDAP_5_USER_DN_TEMPLATE', 'AUTH_LDAP_5_USER_ATT$
MAP', 'AUTH_LDAP_5_GROUP_SEARCH', 'AUTH_LDAP_5_GROUP_TYPE', 'AUTH_LDAP_5_GROUP_TYPE_PARAMS', 'AUTH_LDAP_5_REQUIRE_GROUP', 'AUTH_LDAP_5_DENY_GROUP', 'AUTH_LDAP_5_USER_FLAGS_BY_GROUP', 'AUTH_LDAP_5_ORGANIZATION_MAP', 'AUTH_LDAP_5_TEAM_MAP$
, 'RADIUS_SERVER', 'RADIUS_PORT', 'RADIUS_SECRET', 'TACACSPLUS_HOST', 'TACACSPLUS_PORT', 'TACACSPLUS_SECRET', 'TACACSPLUS_SESSION_TIMEOUT', 'TACACSPLUS_AUTH_PROTOCOL', 'SOCIAL_AUTH_GOOGLE_OAUTH2_KEY', 'SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET', $
SOCIAL_AUTH_GOOGLE_OAUTH2_WHITELISTED_DOMAINS', 'SOCIAL_AUTH_GOOGLE_OAUTH2_AUTH_EXTRA_ARGUMENTS', 'SOCIAL_AUTH_GOOGLE_OAUTH2_ORGANIZATION_MAP', 'SOCIAL_AUTH_GOOGLE_OAUTH2_TEAM_MAP', 'SOCIAL_AUTH_GITHUB_KEY', 'SOCIAL_AUTH_GITHUB_SECRET', $
SOCIAL_AUTH_GITHUB_ORGANIZATION_MAP', 'SOCIAL_AUTH_GITHUB_TEAM_MAP', 'SOCIAL_AUTH_GITHUB_ORG_KEY', 'SOCIAL_AUTH_GITHUB_ORG_SECRET', 'SOCIAL_AUTH_GITHUB_ORG_NAME', 'SOCIAL_AUTH_GITHUB_ORG_ORGANIZATION_MAP', 'SOCIAL_AUTH_GITHUB_ORG_TEAM_MA$
', 'SOCIAL_AUTH_GITHUB_TEAM_KEY', 'SOCIAL_AUTH_GITHUB_TEAM_SECRET', 'SOCIAL_AUTH_GITHUB_TEAM_ID', 'SOCIAL_AUTH_GITHUB_TEAM_ORGANIZATION_MAP', 'SOCIAL_AUTH_GITHUB_TEAM_TEAM_MAP', 'SOCIAL_AUTH_AZUREAD_OAUTH2_KEY', 'SOCIAL_AUTH_AZUREAD_OAUT$
2_SECRET', 'SOCIAL_AUTH_AZUREAD_OAUTH2_ORGANIZATION_MAP', 'SOCIAL_AUTH_AZUREAD_OAUTH2_TEAM_MAP', 'SOCIAL_AUTH_SAML_SP_ENTITY_ID', 'SOCIAL_AUTH_SAML_SP_PUBLIC_CERT', 'SOCIAL_AUTH_SAML_SP_PRIVATE_KEY', 'SOCIAL_AUTH_SAML_ORG_INFO', 'SOCIAL$
UTH_SAML_TECHNICAL_CONTACT', 'SOCIAL_AUTH_SAML_SUPPORT_CONTACT', 'SOCIAL_AUTH_SAML_ENABLED_IDPS', 'SOCIAL_AUTH_SAML_SECURITY_CONFIG', 'SOCIAL_AUTH_SAML_SP_EXTRA', 'SOCIAL_AUTH_SAML_EXTRA_DATA', 'SOCIAL_AUTH_SAML_ORGANIZATION_MAP', 'SOCIA$
_AUTH_SAML_TEAM_MAP', 'SOCIAL_AUTH_SAML_ORGANIZATION_ATTR', 'SOCIAL_AUTH_SAML_TEAM_ATTR') AND "conf_setting"."user_id" IS NULL) ORDER BY "conf_setting"."id" ASC
awx_postgres | ERROR: relation "main_instance" does not exist at character 24
awx_postgres | STATEMENT: SELECT (1) AS "a" FROM "main_instance" WHERE "main_instance"."hostname" = 'awx' LIMIT 1
awx_task | Traceback (most recent call last):
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
awx_task | return self.cursor.execute(sql, params)
awx_task | psycopg2.ProgrammingError: relation "main_instance" does not exist
awx_task | LINE 1: SELECT (1) AS "a" FROM "main_instance" WHERE "main_instance"...
awx_task | ^
awx_task | The above exception was the direct cause of the following exception:
awx_task |
awx_task | Traceback (most recent call last):
awx_task | File "/usr/bin/awx-manage", line 11, in
awx_task | load_entry_point('awx==6.1.0.0', 'console_scripts', 'awx-manage')()
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/init.py", line 142, in manage
awx_task | execute_from_command_line(sys.argv)
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/init.py", line 381, in execute_from_command_line
awx_task | utility.execute()
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/init.py", line 375, in execute
awx_task | self.fetch_command(subcommand).run_from_argv(self.argv)
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/base.py", line 323, in run_from_argv
awx_task | self.execute(*args, **cmd_options)
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/base.py", line 364, in execute
awx_task | output = self.handle(*args, **options)
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/run_dispatcher.py", line 123, in handle
awx_task | reaper.reap()
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/dispatch/reaper.py", line 36, in reap
awx_task | me = instance or Instance.objects.me()
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/managers.py", line 114, in me
awx_task | if node.exists():
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/models/query.py", line 766, in exists
awx_task | return self.query.has_results(using=self.db)
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/models/sql/query.py", line 522, in has_results
awx_task | return compiler.has_results()
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1070, in has_results
awx_task | return bool(self.execute_sql(SINGLE))
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1100, in execute_sql
awx_task | cursor.execute(sql, params)
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
awx_task | return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
awx_task | return executor(sql, params, many, context)
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
awx_task | return self.cursor.execute(sql, params)
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/utils.py", line 89, in exit
awx_task | raise dj_exc_value.with_traceback(traceback) from exc_value
awx_task | File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
awx_task | return self.cursor.execute(sql, params)
awx_task | django.db.utils.ProgrammingError: relation "main_instance" does not exist
awx_task | LINE 1: SELECT (1) AS "a" FROM "main_instance" WHERE "main_instance"...
awx_task | ^
awx_task |
awx_task | 2019-08-02 15:03:21,021 INFO exited: dispatcher (exit status 1; not expected)

[chazzly]

not sure why this worked, but here's what I did:

After reading - http://elatov.github.io/2018/12/setting-up-and-using-awx-with-docker-compose/

cd /tmp/awxcompose/
docker-compose stop
cd ~/awx/installer
ansible-playbook -i inventory install.yml

This time, it gave me the upgrade screen for a couple minutes, then came up.

[stale]

This issue has been marked 'stale' due to lack of recent activity. If there is no further activity, the issue will be closed in another 30 days. Thank you for your contribution!

Please read this blog post to see the reasons why I mark issues as stale.

[stale]

This issue has been closed due to inactivity. If you feel this is in error, please reopen the issue or file a new issue with the relevant details.

[danielMandelblat]

not sure why this worked, but here's what I did:

After reading - http://elatov.github.io/2018/12/setting-up-and-using-awx-with-docker-compose/

cd /tmp/awxcompose/
docker-compose stop
cd ~/awx/installer
ansible-playbook -i inventory install.yml

This time, it gave me the upgrade screen for a couple minutes, then came up.

It's worked for me on Ubuntu 20.4
Thanks a lot.

[AnandTigadikar]

Thanks @danielMandelblat , it helped me to resolve my issue too :)

[ksarsri]

Thanks @danielMandelblat, it helped me too resolve the problem.

[daved0107]

I Solved problem, I deploy awx in k8s. The solution is:
enter to deploy awx-controller > run script /usr/bin/launch_awx_task.sh
./usr/bin/launch_awx_task.sh

Just done.