Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

During AWS_Task Setup Django failing #116

Closed
knechtionscoding opened this issue Sep 11, 2017 · 15 comments
Closed

During AWS_Task Setup Django failing #116

knechtionscoding opened this issue Sep 11, 2017 · 15 comments

Comments

@knechtionscoding
Copy link

ISSUE TYPE
  • Bug Report
COMPONENT
  • Installer
SUMMARY

After successful completion of the local docker installation on Centos 7.3.1611 docker properly has 5 containers running. When looking at the logs for the AWS Task container however the celery process keeps dying due to anissue in the Django python code.

ENVIRONMENT
STEPS TO REPRODUCE

Run installation on Centos 7.3.1611

EXPECTED RESULTS

Correct configuration of AWX post installation

ACTUAL RESULTS
2017-09-11 16:11:31,969 INFO exited: celery (exit status 1; not expected)
2017-09-11 16:11:32,972 INFO spawned: 'celery' with pid 669
2017-09-11 16:11:33,974 INFO success: celery entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-09-11 16:11:37,017 WARNING  py.warnings /var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/base.py:260: RemovedInDjango19Warning: "requires_model_validation" is deprecated in favor of "requires_system_checks".
  RemovedInDjango19Warning)

2017-09-11 16:11:37,087 INFO     awx.main.tasks Syncing Schedules
2017-09-11 16:11:37,087 INFO     awx.main.tasks Syncing Schedules
Traceback (most recent call last):
  File "/usr/bin/awx-manage", line 9, in <module>
    load_entry_point('awx==1.0.0.327', 'console_scripts', 'awx-manage')()
  File "/usr/lib/python2.7/site-packages/awx/__init__.py", line 107, in manage
    execute_from_command_line(sys.argv)
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
    utility.execute()
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/djcelery/management/commands/celery.py", line 23, in run_from_argv
    ['{0[0]} {0[1]}'.format(argv)] + argv[2:],
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/celery/bin/celery.py", line 769, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/celery/bin/base.py", line 307, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/celery/bin/celery.py", line 761, in handle_argv
    return self.execute(command, argv)
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/celery/bin/celery.py", line 693, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/celery/bin/worker.py", line 179, in run_from_argv
    return self(*args, **options)
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/celery/bin/base.py", line 270, in __call__
    ret = self.run(*args, **kwargs)
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/celery/bin/worker.py", line 212, in run
    state_db=self.node_format(state_db, hostname), **kwargs
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/celery/worker/__init__.py", line 96, in __init__
    self.on_before_init(**kwargs)
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/celery/apps/worker.py", line 120, in on_before_init
    conf=self.app.conf, options=kwargs,
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/celery/utils/dispatch/signal.py", line 166, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/usr/lib/python2.7/site-packages/awx/main/tasks.py", line 102, in celery_startup
    for sch in Schedule.objects.all():
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/query.py", line 162, in __iter__
    self._fetch_all()
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/query.py", line 965, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/query.py", line 238, in iterator
    results = compiler.execute_sql()
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 840, in execute_sql
    cursor.execute(sql, params)
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/utils.py", line 98, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "main_schedule" does not exist
LINE 1: ...le"."next_run", "main_schedule"."extra_data" FROM "main_sche...
                                                             ^

2017-09-11 16:11:37,225 INFO exited: celery (exit status 1; not expected)
2017-09-11 16:11:38,228 INFO spawned: 'celery' with pid 680

This continues repeatedly. I've let it run for 15 minutes with no change in the attempts.

ADDITIONAL INFORMATION

This is unrelated to issue #115. I managed to get the installer working on another instance.

@matburt
Copy link
Member

matburt commented Sep 11, 2017

This is pretty similar to this issue: #23 for which I have a PR available.

Basically, when migrations are running (and they can take a while) the interface will just return errors.

However, the task container is the one that should be running the migrations. The log seems clipped though... did it generate an error when attempting to run the migrations?

@knechtionscoding
Copy link
Author

@matburt I'm getting the logs from the "docker logs -f awx_task" command.

During the install.yml run it returned no errors, as far as I could see. The output is gone because I've been trying to watch the docker logs to see if there was any change in them.

@matburt
Copy link
Member

matburt commented Sep 11, 2017

One of the first things the task container will do is attempt to run migrations so if you can grab the output of that when it experiences the problem that'd be super helpful.

@knechtionscoding
Copy link
Author

@matburt Is there a recommended way for me to grab those?

@matburt
Copy link
Member

matburt commented Sep 11, 2017

just get to the log output early I'd reckon?

  1. Stop all containers
  2. remove pg data dir
  3. start postgres and let it refresh that dir
    3.5) wait a brief time for pg to start up
  4. start awx_ containers and immediately start pulling the logs for awx_task

@phandolin
Copy link

I can second having this issue, the error messages have been repeating for about 10 minutes.

The other thing I'm noticing which may be related is when I navigate to http://localhost a pop up box says "ERROR Failed to read/api. GET status: 500 A server error has occured". I can close the box, and when I attempt to login, it just hangs with the 'Working' gear spinning in the bottom corner.

@knechtionscoding
Copy link
Author

@matburt Following Steps taken:

  1. Stop each container in descending order:
  • awx_task
  • awx_web
  • memcached
  • rabbitmq
  • postgres
  1. Remove /tmp/pgdocker/

  2. Re-run install.yml with no other changes.

migrations finished and everything seemed to be happy.

Curious if there is an issue on the wait time, or something similar?

Here are the migration logs from the second run.

Using /etc/ansible/ansible.cfg as config file
127.0.0.1 | SUCCESS => {
    "changed": true, 
    "db": "awx"
}
Operations to perform:
  Synchronize unmigrated apps: solo, api, staticfiles, messages, channels, django_extensions, ui, rest_framework, polymorphic
  Apply all migrations: sso, taggit, sessions, djcelery, sites, kombu_transport_django, social_auth, contenttypes, auth, conf, main
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying contenttypes.0001_initial... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0001_initial... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying taggit.0001_initial... OK
  Applying taggit.0002_auto_20150616_2121... OK
  Applying main.0001_initial... OK
  Applying main.0002_squashed_v300_release... OK
  Applying main.0003_squashed_v300_v303_updates... OK
  Applying main.0004_squashed_v310_release... OK
  Applying conf.0001_initial... OK
  Applying conf.0002_v310_copy_tower_settings... OK
  Applying conf.0003_v310_JSONField_changes... OK
  Applying conf.0004_v320_reencrypt... OK
  Applying djcelery.0001_initial... OK
  Applying kombu_transport_django.0001_initial... OK
  Applying main.0005_squashed_v310_v313_updates... OK
  Applying main.0006_v320_release... OK
2017-09-11 18:56:32,944 DEBUG    awx.main.migrations Removing all Rackspace InventorySource from database.
2017-09-11 18:56:32,986 DEBUG    awx.main.migrations Removing all InventorySource that have no link to an Inventory from database.
  Applying main.0007_v320_data_migrations... OK
  Applying main.0008_v320_drop_v1_credential_fields... OK
  Applying sessions.0001_initial... OK
  Applying sites.0001_initial... OK
  Applying social_auth.0001_initial... OK
  Applying social_auth.0002_add_related_name... OK
  Applying social_auth.0003_alter_email_max_length... OK
  Applying social_auth.0004_auto_20160423_0400... OK
  Applying social_auth.0005_auto_20160727_2333... OK
  Applying sso.0001_initial... OK
  Applying sso.0002_expand_provider_options... OK
Python 2.7.5 (default, Nov  6 2016, 00:28:07) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)

>>> <User: admin>
>>> Default organization added.
Demo Credential, Inventory, and Job Template added.
Successfully registered instance awx
(changed: True)
Creating instance group tower
Added instance awx to tower
(changed: True)
2017-09-11 18:57:36,840 CRIT Supervisor running as root (no user in config file)
2017-09-11 18:57:36,840 WARN For [program:awx-celeryd-beat], redirect_stderr=true but stderr_logfile has also been set to a filename, the filename has been ignored
Unlinking stale socket /tmp/supervisor.sock
2017-09-11 18:57:37,159 INFO RPC interface 'supervisor' initialized
2017-09-11 18:57:37,159 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2017-09-11 18:57:37,160 INFO supervisord started with pid 90
2017-09-11 18:57:38,163 INFO spawned: 'celery' with pid 93
2017-09-11 18:57:38,168 INFO spawned: 'channels-worker' with pid 94
2017-09-11 18:57:38,174 INFO spawned: 'callback-receiver' with pid 95
2017-09-11 18:57:38,176 INFO spawned: 'awx-celeryd-beat' with pid 96
2017-09-11 18:57:39,180 INFO success: celery entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-09-11 18:57:39,180 INFO success: channels-worker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-09-11 18:57:39,180 INFO success: callback-receiver entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-09-11 18:57:39,181 INFO success: awx-celeryd-beat entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-09-11 18:57:45,018 WARNING  py.warnings /var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/base.py:260: RemovedInDjango19Warning: "requires_model_validation" is deprecated in favor of "requires_system_checks".
  RemovedInDjango19Warning)

2017-09-11 18:57:45,243 INFO     awx.main.tasks Syncing Schedules
2017-09-11 18:57:45,243 INFO     awx.main.tasks Syncing Schedules
2017-09-11 18:57:45,491 WARNING  py.warnings /var/lib/awx/venv/awx/lib/python2.7/site-packages/celery/platforms.py:766: RuntimeWarning: You are running the worker with superuser privileges, which is
absolutely not recommended!

Please specify a different user using the -u option.

User information: uid=0 euid=0 gid=0 egid=0

  uid=uid, euid=euid, gid=gid, egid=egid,

[2017-09-11 18:57:45,532: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2017-09-11 18:57:45,538: DEBUG/MainProcess] | Worker: Building graph...
[2017-09-11 18:57:45,538: DEBUG/MainProcess] | Worker: New boot order: {StateDB, Beat, Timer, Hub, Queues (intra), Pool, Autoscaler, Autoreloader, Consumer}
[2017-09-11 18:57:45,556: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2017-09-11 18:57:45,556: DEBUG/MainProcess] | Consumer: Building graph...
[2017-09-11 18:57:45,579: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Events, Heart, Mingle, Tasks, Control, Agent, Gossip, event loop}
[2017-09-11 18:57:45,591: DEBUG/MainProcess] | Worker: Starting Hub
[2017-09-11 18:57:45,591: DEBUG/MainProcess] ^-- substep ok
[2017-09-11 18:57:45,591: DEBUG/MainProcess] | Worker: Starting Pool
[2017-09-11 18:57:45,592: DEBUG/MainProcess] ^-- substep ok
[2017-09-11 18:57:45,592: DEBUG/MainProcess] | Worker: Starting Autoscaler
[2017-09-11 18:57:45,592: DEBUG/MainProcess] ^-- substep ok
[2017-09-11 18:57:45,592: DEBUG/MainProcess] | Worker: Starting Consumer
[2017-09-11 18:57:45,593: DEBUG/MainProcess] | Consumer: Starting Connection
[2017-09-11 18:57:45,630: DEBUG/MainProcess] Start from server, version: 0.9, properties: {u'information': u'Licensed under the MPL.  See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2017 Pivotal Software, Inc.', u'capabilities': {u'exchange_exchange_bindings': True, u'connection.blocked': True, u'authentication_failure_close': True, u'direct_reply_to': True, u'basic.nack': True, u'per_consumer_qos': True, u'consumer_priorities': True, u'consumer_cancel_notify': True, u'publisher_confirms': True}, u'cluster_name': u'rabbit@164ba75c698a', u'platform': u'Erlang/OTP 19.2.1', u'version': u'3.6.11'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US']
[2017-09-11 18:57:45,634: DEBUG/MainProcess] Open OK!
[2017-09-11 18:57:45,634: INFO/MainProcess] Connected to amqp://guest:**@rabbitmq:5672/awx
[2017-09-11 18:57:45,635: DEBUG/MainProcess] ^-- substep ok
[2017-09-11 18:57:45,635: DEBUG/MainProcess] | Consumer: Starting Events
[2017-09-11 18:57:45,677: DEBUG/MainProcess] Start from server, version: 0.9, properties: {u'information': u'Licensed under the MPL.  See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2017 Pivotal Software, Inc.', u'capabilities': {u'exchange_exchange_bindings': True, u'connection.blocked': True, u'authentication_failure_close': True, u'direct_reply_to': True, u'basic.nack': True, u'per_consumer_qos': True, u'consumer_priorities': True, u'consumer_cancel_notify': True, u'publisher_confirms': True}, u'cluster_name': u'rabbit@164ba75c698a', u'platform': u'Erlang/OTP 19.2.1', u'version': u'3.6.11'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US']
[2017-09-11 18:57:45,682: DEBUG/MainProcess] Open OK!
[2017-09-11 18:57:45,683: DEBUG/MainProcess] using channel_id: 1
[2017-09-11 18:57:45,686: DEBUG/MainProcess] Channel open
[2017-09-11 18:57:45,697: DEBUG/MainProcess] ^-- substep ok
[2017-09-11 18:57:45,697: DEBUG/MainProcess] | Consumer: Starting Heart
[2017-09-11 18:57:45,701: DEBUG/MainProcess] ^-- substep ok
[2017-09-11 18:57:45,701: DEBUG/MainProcess] | Consumer: Starting Mingle
[2017-09-11 18:57:45,701: INFO/MainProcess] mingle: searching for neighbors
[2017-09-11 18:57:45,702: DEBUG/MainProcess] using channel_id: 1
[2017-09-11 18:57:45,705: DEBUG/MainProcess] Channel open
2017-09-11 18:57:46,118 - INFO - runworker - Using single-threaded worker.
2017-09-11 18:57:46,118 - INFO - runworker - Running worker against channel layer default (asgi_amqp.core.AMQPChannelLayer)
2017-09-11 18:57:46,118 - INFO - worker - Listening on channels websocket.connect, websocket.disconnect, websocket.receive
2017-09-11 18:57:46,221 WARNING  py.warnings /var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/base.py:260: RemovedInDjango19Warning: "requires_model_validation" is deprecated in favor of "requires_system_checks".
  RemovedInDjango19Warning)

[2017-09-11 18:57:46,283: DEBUG/MainProcess] Setting default socket timeout to 30
[2017-09-11 18:57:46,283: INFO/MainProcess] beat: Starting...
[2017-09-11 18:57:46,302: DEBUG/MainProcess] Current schedule:
<Entry: purge_stdout_files awx.main.tasks.purge_old_stdout_files() <freq: 7.00 days>
<Entry: cluster_heartbeat awx.main.tasks.cluster_node_heartbeat() <freq: 1.00 minute>
<Entry: authtoken_cleanup awx.main.tasks.cleanup_authtokens() <freq: 30.00 days>
<Entry: celery.backend_cleanup celery.backend_cleanup() <crontab: 0 4 * * * (m/h/d/dM/MY)>
<Entry: task_manager awx.main.scheduler.tasks.run_task_manager() <freq: 20.00 seconds>
<Entry: tower_scheduler awx.main.tasks.awx_periodic_scheduler() <freq: 30.00 seconds>
<Entry: admin_checks awx.main.tasks.run_administrative_checks() <freq: 30.00 days>
[2017-09-11 18:57:46,302: DEBUG/MainProcess] beat: Ticking with max interval->1.00 minute
[2017-09-11 18:57:46,325: DEBUG/MainProcess] Start from server, version: 0.9, properties: {u'information': u'Licensed under the MPL.  See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2017 Pivotal Software, Inc.', u'capabilities': {u'exchange_exchange_bindings': True, u'connection.blocked': True, u'authentication_failure_close': True, u'direct_reply_to': True, u'basic.nack': True, u'per_consumer_qos': True, u'consumer_priorities': True, u'consumer_cancel_notify': True, u'publisher_confirms': True}, u'cluster_name': u'rabbit@164ba75c698a', u'platform': u'Erlang/OTP 19.2.1', u'version': u'3.6.11'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US']
[2017-09-11 18:57:46,327: DEBUG/MainProcess] Open OK!
[2017-09-11 18:57:46,331: INFO/MainProcess] Scheduler: Sending due task tower_scheduler (awx.main.tasks.awx_periodic_scheduler)
[2017-09-11 18:57:46,337: DEBUG/MainProcess] using channel_id: 1
[2017-09-11 18:57:46,339: DEBUG/MainProcess] Channel open
[2017-09-11 18:57:46,342: DEBUG/MainProcess] beat: Synchronizing schedule...
[2017-09-11 18:57:46,344: DEBUG/MainProcess] awx.main.tasks.awx_periodic_scheduler sent. id->c8a96c7f-bd56-4238-bcc0-3cbe985c4c72
[2017-09-11 18:57:46,348: INFO/MainProcess] Scheduler: Sending due task task_manager (awx.main.scheduler.tasks.run_task_manager)
[2017-09-11 18:57:46,352: DEBUG/MainProcess] awx.main.scheduler.tasks.run_task_manager sent. id->d071b1f5-cdf0-4a8b-ab8f-cedbed9cea4f
[2017-09-11 18:57:46,353: INFO/MainProcess] Scheduler: Sending due task cluster_heartbeat (awx.main.tasks.cluster_node_heartbeat)
[2017-09-11 18:57:46,358: DEBUG/MainProcess] awx.main.tasks.cluster_node_heartbeat sent. id->dfe26436-7f3b-4e72-9b64-ce7dc070c9d3
[2017-09-11 18:57:46,359: DEBUG/MainProcess] beat: Waking up in 19.99 seconds.
[2017-09-11 18:57:46,733: INFO/MainProcess] mingle: all alone
[2017-09-11 18:57:46,733: DEBUG/MainProcess] ^-- substep ok
[2017-09-11 18:57:46,734: DEBUG/MainProcess] | Consumer: Starting Tasks
[2017-09-11 18:57:46,757: DEBUG/MainProcess] ^-- substep ok
[2017-09-11 18:57:46,757: DEBUG/MainProcess] | Consumer: Starting Control
[2017-09-11 18:57:46,757: DEBUG/MainProcess] using channel_id: 2
[2017-09-11 18:57:46,758: DEBUG/MainProcess] Channel open
[2017-09-11 18:57:46,763: DEBUG/MainProcess] ^-- substep ok
[2017-09-11 18:57:46,763: DEBUG/MainProcess] | Consumer: Starting Gossip
[2017-09-11 18:57:46,763: DEBUG/MainProcess] using channel_id: 3
[2017-09-11 18:57:46,764: DEBUG/MainProcess] Channel open
[2017-09-11 18:57:46,770: DEBUG/MainProcess] ^-- substep ok
[2017-09-11 18:57:46,770: DEBUG/MainProcess] | Consumer: Starting event loop
[2017-09-11 18:57:46,773: WARNING/MainProcess] celery@localhost ready.
[2017-09-11 18:57:46,773: DEBUG/MainProcess] | Worker: Hub.register Autoscaler...
[2017-09-11 18:57:46,774: DEBUG/MainProcess] | Worker: Hub.register Pool...
[2017-09-11 18:57:46,774: DEBUG/MainProcess] basic.qos: prefetch_count->4
[2017-09-11 18:57:47,706: INFO/MainProcess] Discarding revoked task: awx.main.scheduler.tasks.run_task_manager[1dd61323-5f3a-4862-8127-e063de61cf3e]
[2017-09-11 18:57:47,714: INFO/MainProcess] Discarding revoked task: awx.main.tasks.awx_periodic_scheduler[c75397ac-f367-4c2f-ab98-aeffa31e428a]
[2017-09-11 18:57:47,721: INFO/MainProcess] Discarding revoked task: awx.main.scheduler.tasks.run_task_manager[abd084f7-a2fd-4aa9-84e0-4b7e41135d1b]
[2017-09-11 18:57:47,727: INFO/MainProcess] Discarding revoked task: awx.main.scheduler.tasks.run_task_manager[c40e5b5b-79b0-4b04-9b60-c0b2feae378d]
[2017-09-11 18:57:47,733: INFO/MainProcess] Discarding revoked task: awx.main.tasks.cluster_node_heartbeat[34265ba0-8df3-417c-8949-73ff71212fa4]
[2017-09-11 18:57:47,746: INFO/MainProcess] Discarding revoked task: awx.main.tasks.awx_periodic_scheduler[a30d8fc8-eba2-4ac9-ba69-c6823811050d]
[2017-09-11 18:57:47,760: INFO/MainProcess] Discarding revoked task: awx.main.scheduler.tasks.run_task_manager[20541bac-a730-47be-9e8f-35c3b2e1a412]

@phandolin
Copy link

@knechtionscoding @matburt I followed those steps and it's also working here now. Thanks!

@knechtionscoding
Copy link
Author

@phandolin Good to hear.

@matburt
Copy link
Member

matburt commented Sep 11, 2017

blah, part of me hoped you would run into it again so I could see what the error was :/ I'll close this and if you have any other information feel free to open it with more logs.

@krispayne
Copy link

Before I attach the ad nauseum logs from awx_task repeating the celery failure every 10 seconds, is there some sort of persistent data directory for the database that should be removed/moved during a re-install even if you're not using the postgres docker container? I rebuilt my setup because the postgres container would constantly restart and setup a separate host for postgres and when I install into the new database, I'm having the same issues listed here. I'm on the most up to date git pull.

@matburt
Copy link
Member

matburt commented Mar 13, 2018

If you are managing the database externally then you'll need to drop and re-create the database and let AWX run the migrations... otherwise you're just using the same data.

@krispayne
Copy link

Yes I do that in place of removing the postgres container. Each time I run the installer.yml I am starting with a brand new database. I remove the docker images with docker rmi $(docker kill $(docker ps -aq))

I've just downloaded a fresh copy of 1.0.4 (instead of cloning the repo). We'll see how it goes!

@krispayne
Copy link

krispayne commented Mar 13, 2018

  • I did nothing different, except instead of cloning the repo (per install.md) I downloaded the 1.0.4 tarball and untarred it. I am able to reach the login screen now, instead of migrating. -

Actually, I failed to list my external hostname, so it created a postgres docker :-/.

When I'm setting up the external one (with another freshly dropped/created database) I get the celery errors repeating again and the web interface is the migrations_notran page.

@krispayne
Copy link

The 1.0.4 tar from https://github.com/ansible/awx/releases

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants