AwxTaskError.TaskError(instance, rc) Exception: project_update xxx (failed) encountered an error #2083

itcrowdsource · 2018-07-21T09:42:31Z

ISSUE TYPE

Bug Report

COMPONENT NAME

API
UI

SUMMARY

While launching a job an activity tries to retrieve the latest updates from the GIT source repository. This activity fails.

ENVIRONMENT

AWX version: 1.0.6.35
AWX install method: docker on linux
Ansible version: 2.6.1
Operating System: Ubuntu Xenial
Web Browser: Chrome

STEPS TO REPRODUCE

Create a job template
Create a GIT source project (SCM update options: Clean, Update Revision On Launch)
Configure the job template to use a GIT source
Run the job template

EXPECTED RESULTS

Job runs succesfully

ACTUAL RESULTS

Job fails:
Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/awx/main/tasks.py", line 881, in run self.pre_run_hook(instance, **kwargs) File "/usr/lib/python2.7/site-packages/awx/main/tasks.py", line 2123, in pre_run_hook task_instance.run(local_project_sync.id) File "/usr/lib/python2.7/site-packages/awx/main/tasks.py", line 574, in _wrapped return f(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/awx/main/tasks.py", line 1027, in run raise AwxTaskError.TaskError(instance, rc) Exception: project_update 567 (failed) encountered an error (rc=2), please see task stdout for details.

ADDITIONAL INFORMATION

The text was updated successfully, but these errors were encountered:

matburt · 2018-07-24T15:56:05Z

Sooo... what was the error in the project update detail view?

Post the reason for the failure and we can reopen as needed.

itcrowdsource · 2018-07-25T14:38:25Z

@matburt what information is further needed? I've already provided the details about the error message. I'm currently unable to sync a source. In my case a GIT source. If I go into inventory -> Sources ->Click the sync button I'm getting the following error message in the task details view:

Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/awx/main/tasks.py", line 881, in run self.pre_run_hook(instance, **kwargs) File "/usr/lib/python2.7/site-packages/awx/main/tasks.py", line 2123, in pre_run_hook task_instance.run(local_project_sync.id) File "/usr/lib/python2.7/site-packages/awx/main/tasks.py", line 574, in _wrapped return f(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/awx/main/tasks.py", line 1027, in run raise AwxTaskError.TaskError(instance, rc) Exception: project_update 619 (failed) encountered an error (rc=2), please see task stdout for details. SHOW LESS

matburt · 2018-07-25T14:46:23Z

Look at the error in the job stdout view for the update itself.

itcrowdsource · 2018-07-25T14:51:24Z

The job's stdout view is empty

matburt · 2018-07-25T14:59:34Z

can you grab the api output for this job?

itcrowdsource · 2018-07-26T09:18:46Z

API doesn't seem to contain data for this job...

matburt · 2018-07-27T16:25:46Z

It's not a job... those are playbook runs. It'll be a project update under /api/v2/project_updates/n/

itcrowdsource · 2018-07-27T17:59:19Z

{
    "id": 623,
    "type": "project_update",
    "url": "/api/v2/project_updates/623/",
    "related": {
        "credential": "/api/v2/credentials/2/",
        "unified_job_template": "/api/v2/projects/6/",
        "stdout": "/api/v2/project_updates/623/stdout/",
        "project": "/api/v2/projects/6/",
        "cancel": "/api/v2/project_updates/623/cancel/",
        "notifications": "/api/v2/project_updates/623/notifications/",
        "scm_inventory_updates": "/api/v2/project_updates/623/scm_inventory_updates/",
        "events": "/api/v2/project_updates/623/events/"
    },
    "summary_fields": {
        "instance_group": {
            "id": 1,
            "name": "tower"
        },
        "credential": {
            "id": 2,
            "name": "Bitbucket",
            "description": "Bitbucket",
            "kind": "scm",
            "cloud": false,
            "credential_type_id": 2
        },
        "unified_job_template": {
            "id": 6,
            "name": "GIT source",
            "description": "GIT source",
            "unified_job_type": "project_update"
        },
        "project": {
            "id": 6,
            "name": "GIT source",
            "description": "GIT source",
            "status": "successful",
            "scm_type": "git"
        },
        "user_capabilities": {
            "start": true,
            "delete": true
        }
    },
    "created": "2018-07-25T14:49:56.103826Z",
    "modified": "2018-07-25T14:49:56.119828Z",
    "name": "GIT source",
    "description": "GIT source",
    "local_path": "_6__git_source",
    "scm_type": "git",
    "scm_url": "https://bitbucket.org/xxxxxxxxxxxxxxxxxx/xxxxxxxxxxxxxx.git",
    "scm_branch": "",
    "scm_clean": true,
    "scm_delete_on_update": false,
    "credential": 2,
    "timeout": 0,
    "unified_job_template": 6,
    "launch_type": "sync",
    "status": "failed",
    "failed": true,
    "started": "2018-07-25T14:49:56.102266Z",
    "finished": "2018-07-25T14:50:02.519523Z",
    "elapsed": 6.417,
    "job_args": "[\"ssh-agent\", \"-a\", \"/tmp/awx_623_8awFkx/ssh_auth.sock\", \"sh\", \"-c\", \"ssh-add /tmp/awx_623_8awFkx/credential_2 && rm -f /tmp/awx_623_8awFkx/credential_2 && ansible-playbook -i localhost, -v -e @/tmp/awx_623_8awFkx/tmpq3eTju project_update.yml\"]",
    "job_cwd": "/usr/lib/python2.7/site-packages/awx/playbooks",
    "job_env": {
        "MEMCACHED_PORT_11211_TCP": "tcp://172.17.0.3:11211",
        "ANSIBLE_RETRY_FILES_ENABLED": "False",
        "AWXWEB_ENV_AWX_ADMIN_USER": "admin",
        "http_proxy": "",
        "POSTGRES_ENV_POSTGRES_PASSWORD": "**********",
        "RABBITMQ_PORT_4369_TCP_ADDR": "172.17.0.4",
        "AWXWEB_ENV_DATABASE_USER": "awx",
        "AWXWEB_PORT_8052_TCP_PROTO": "tcp",
        "AWXWEB_PORT_8052_TCP_ADDR": "172.17.0.5",
        "RABBITMQ_ENV_RABBITMQ_LOGS": "-",
        "VIRTUAL_ENV": "/var/lib/awx/venv/ansible",
        "RABBITMQ_PORT_15671_TCP_PORT": "15671",
        "MEMCACHED_ENV_MEMCACHED_VERSION": "1.5.9",
        "SUPERVISOR_ENABLED": "1",
        "RABBITMQ_PORT_15672_TCP_PORT": "15672",
        "RABBITMQ_PORT_5672_TCP_PROTO": "tcp",
        "DJANGO_PROJECT_DIR": "/usr/lib/python2.7/site-packages",
        "RABBITMQ_PORT": "5672",
        "RABBITMQ_HOST": "rabbitmq",
        "PYTHONPATH": "/var/lib/awx/venv/ansible/lib/python2.7/site-packages:/usr/lib/python2.7/site-packages/awx/lib:",
        "POSTGRES_ENV_POSTGRES_USER": "awx",
        "POSTGRES_ENV_LANG": "en_US.utf8",
        "LC_CTYPE": "en_US.UTF-8",
        "MEMCACHED_PORT_11211_TCP_PROTO": "tcp",
        "_MP_FORK_LOGFILE_": "",
        "RABBITMQ_PORT_4369_TCP": "tcp://172.17.0.4:4369",
        "MEMCACHED_PORT_11211_TCP_PORT": "11211",
        "HOSTNAME": "awx",
        "AWXWEB_ENV_DATABASE_NAME": "awx",
        "ANSIBLE_VENV_PATH": "/var/lib/awx/venv/ansible",
        "MEMCACHED_HOST": "memcached",
        "AWXWEB_ENV_RABBITMQ_HOST": "rabbitmq",
        "PROJECT_UPDATE_ID": "623",
        "RABBITMQ_PORT_25672_TCP_PROTO": "tcp",
        "MEMCACHED_NAME": "/awx_task/memcached",
        "CELERY_LOG_REDIRECT_LEVEL": "WARNING",
        "SUPERVISOR_GROUP_NAME": "tower-processes",
        "SECRET_KEY": "**********",
        "POSTGRES_ENV_PGDATA": "/var/lib/postgresql/data/pgdata",
        "RABBITMQ_ENV_RABBITMQ_ERLANG_COOKIE": "cookiemonster",
        "RABBITMQ_PORT_5672_TCP_ADDR": "172.17.0.4",
        "SUPERVISOR_SERVER_URL": "unix:///tmp/supervisor.sock",
        "LANGUAGE": "en_US.UTF-8",
        "SHLVL": "2",
        "AWX_ADMIN_PASSWORD": "**********",
        "CELERY_LOG_FILE": "",
        "RABBITMQ_PORT_5672_TCP_PORT": "5672",
        "ANSIBLE_HOST_KEY_CHECKING": "False",
        "POSTGRES_PORT_5432_TCP": "tcp://172.17.0.2:5432",
        "LANG": "en_US.UTF-8",
        "ANSIBLE_STDOUT_CALLBACK": "awx_display",
        "AWXWEB_ENV_MEMCACHED_PORT": "11211",
        "SUPERVISOR_PROCESS_NAME": "celery",
        "RABBITMQ_PORT_5671_TCP_ADDR": "172.17.0.4",
        "RABBITMQ_PORT_4369_TCP_PORT": "4369",
        "POSTGRES_ENV_POSTGRES_DB": "awx",
        "POSTGRES_PORT_5432_TCP_PORT": "5432",
        "_": "/usr/bin/supervisord",
        "RABBITMQ_VHOST": "awx",
        "AWXWEB_ENV_RABBITMQ_PASSWORD": "**********",
        "DATABASE_NAME": "awx",
        "RABBITMQ_ENV_RABBITMQ_VERSION": "3.7.4",
        "POSTGRES_PORT_5432_TCP_ADDR": "172.17.0.2",
        "RABBITMQ_ENV_RABBITMQ_GITHUB_TAG": "v3.7.4",
        "DJANGO_SETTINGS_MODULE": "awx.settings.production",
        "AWXWEB_ENV_no_proxy": "",
        "TMP": "/tmp",
        "RABBITMQ_PORT_15671_TCP_PROTO": "tcp",
        "ANSIBLE_BECOME_ASK_PASS": "False",
        "ANSIBLE_PARAMIKO_RECORD_HOST_KEYS": "False",
        "AWX_PRIVATE_DATA_DIR": "/tmp/awx_623_8awFkx",
        "RABBITMQ_PORT_5671_TCP_PORT": "5671",
        "AWXWEB_ENV_MEMCACHED_HOST": "memcached",
        "LC_ALL": "en_US.UTF-8",
        "RABBITMQ_PORT_5671_TCP": "tcp://172.17.0.4:5671",
        "HOME": "/var/lib/awx",
        "DISPLAY": "",
        "RABBITMQ_PORT_15672_TCP_PROTO": "tcp",
        "POSTGRES_ENV_GOSU_VERSION": "1.10",
        "_MP_FORK_LOGFORMAT_": "[%(asctime)s: %(levelname)s/%(processName)s] %(message)s",
        "RABBITMQ_PORT_25672_TCP_PORT": "25672",
        "RABBITMQ_PORT_15672_TCP_ADDR": "172.17.0.4",
        "https_proxy": "",
        "RABBITMQ_PORT_4369_TCP_PROTO": "tcp",
        "RABBITMQ_ENV_RABBITMQ_HOME": "/opt/rabbitmq",
        "CACHE": "memcached:11211",
        "AWXWEB_ENV_DATABASE_PORT": "5432",
        "RABBITMQ_ENV_RABBITMQ_GPG_KEY": "**********",
        "AWXWEB_PORT_8052_TCP": "tcp://172.17.0.5:8052",
        "_MP_FORK_LOGLEVEL_": "10",
        "AWXWEB_ENV_RABBITMQ_USER": "guest",
        "AWXWEB_ENV_RABBITMQ_PORT": "5672",
        "CELERY_LOG_LEVEL": "10",
        "DATABASE_HOST": "postgres",
        "AWXWEB_PORT": "tcp://172.17.0.5:8052",
        "POSTGRES_NAME": "/awx_task/postgres",
        "POSTGRES_ENV_PG_VERSION": "9.6.9-2.pgdg90+1",
        "AWXWEB_ENV_RABBITMQ_VHOST": "awx",
        "POSTGRES_ENV_PG_MAJOR": "9.6",
        "RABBITMQ_PORT_15671_TCP": "tcp://172.17.0.4:15671",
        "RABBITMQ_ENV_RABBITMQ_DEFAULT_VHOST": "awx",
        "DJANGO_LIVE_TEST_SERVER_ADDRESS": "localhost:9013-9199",
        "ANSIBLE_ASK_PASS": "False",
        "CELERY_LOG_REDIRECT": "1",
        "AWXWEB_ENV_AWX_ADMIN_PASSWORD": "**********",
        "RABBITMQ_PORT_25672_TCP_ADDR": "172.17.0.4",
        "AWXWEB_ENV_DATABASE_PASSWORD": "**********",
        "PATH": "/var/lib/awx/venv/ansible/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
        "RABBITMQ_PORT_5672_TCP": "tcp://172.17.0.4:5672",
        "DATABASE_USER": "awx",
        "ANSIBLE_CALLBACK_PLUGINS": "/usr/lib/python2.7/site-packages/awx/plugins/callback",
        "RABBITMQ_PORT_15671_TCP_ADDR": "172.17.0.4",
        "TZ": "UTC",
        "POSTGRES_PORT_5432_TCP_PROTO": "tcp",
        "no_proxy": "",
        "RABBITMQ_ENV_RABBITMQ_SASL_LOGS": "-",
        "AWXWEB_PORT_8052_TCP_PORT": "8052",
        "AWXWEB_NAME": "/awx_task/awxweb",
        "AWXWEB_ENV_SECRET_KEY": "**********",
        "AWXWEB_ENV_DATABASE_HOST": "postgres",
        "MEMCACHED_PORT": "11211",
        "MEMCACHED_ENV_MEMCACHED_SHA1": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
        "RABBITMQ_NAME": "/awx_task/rabbitmq",
        "DATABASE_PORT": "5432",
        "CELERY_LOADER": "djcelery.loaders.DjangoLoader",
        "RABBITMQ_PORT_5671_TCP_PROTO": "tcp",
        "AWX_ADMIN_USER": "admin",
        "MEMCACHED_PORT_11211_TCP_ADDR": "172.17.0.3",
        "ANSIBLE_FORCE_COLOR": "True",
        "RABBITMQ_USER": "guest",
        "POSTGRES_PORT": "tcp://172.17.0.2:5432",
        "AWXWEB_ENV_http_proxy": "",
        "PWD": "/var/lib/awx",
        "AWXWEB_ENV_https_proxy": "",
        "RABBITMQ_PORT_15672_TCP": "tcp://172.17.0.4:15672",
        "DATABASE_PASSWORD": "**********",
        "RABBITMQ_PASSWORD": "**********",
        "RABBITMQ_PORT_25672_TCP": "tcp://172.17.0.4:25672"
    },
    "job_explanation": "",
    "execution_node": "awx",
    "result_traceback": "",
    "event_processing_finished": true,
    "project": 6,
    "job_type": "run"
}

itcrowdsource · 2018-08-09T12:15:16Z

@matburt I checked the API stdout of the project update, there was a problem with the requirements.yml file ... some roles are removed from ansible-galaxy .. the update process couldn't retrieve these roles which caused the failure. Would be nice to see the stdout of project updates also in the job output view in the UI. Maybe a proposal for an enhancement?

Sets default zoom to 100% on network UI

darthwalsh · 2020-11-10T19:57:34Z

If anybody else gets a blank stdout on inventory update job, the TL;DR is:

if your AWX GUI URL looks like i.e. /#/jobs/inventory/9135?
go to /api/v2/inventory_updates/9135/
you'll see a source_project_update field; follow that link
then follow resource stdout/ to see the stdout text.

I'm on 9.2.0 but maybe in a future version AWX will show the stdout directly.

awxbot added state:needs_triage type:bug component:api component:ui labels Jul 23, 2018

matburt closed this as completed Jul 24, 2018

ryanpetrello pushed a commit to ryanpetrello/awx that referenced this issue Aug 13, 2018

Merge pull request ansible#2083 from jaredevantabor/awx-net-zoom-default

86f840f

Sets default zoom to 100% on network UI

kakawait mentioned this issue Dec 13, 2019

Jobs Error After Upgrade to AWX 9.0.1 #5355

Closed

debianchim27 mentioned this issue Nov 18, 2021

Job run fails with traceback if project update fails #6199

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AwxTaskError.TaskError(instance, rc) Exception: project_update xxx (failed) encountered an error #2083

AwxTaskError.TaskError(instance, rc) Exception: project_update xxx (failed) encountered an error #2083

itcrowdsource commented Jul 21, 2018 •

edited by matburt

Loading

matburt commented Jul 24, 2018

itcrowdsource commented Jul 25, 2018 •

edited

Loading

matburt commented Jul 25, 2018

itcrowdsource commented Jul 25, 2018 •

edited

Loading

matburt commented Jul 25, 2018

itcrowdsource commented Jul 26, 2018

matburt commented Jul 27, 2018

itcrowdsource commented Jul 27, 2018

itcrowdsource commented Aug 9, 2018

darthwalsh commented Nov 10, 2020

AwxTaskError.TaskError(instance, rc) Exception: project_update xxx (failed) encountered an error #2083

AwxTaskError.TaskError(instance, rc) Exception: project_update xxx (failed) encountered an error #2083

Comments

itcrowdsource commented Jul 21, 2018 • edited by matburt Loading

ISSUE TYPE

COMPONENT NAME

SUMMARY

ENVIRONMENT

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

ADDITIONAL INFORMATION

matburt commented Jul 24, 2018

itcrowdsource commented Jul 25, 2018 • edited Loading

matburt commented Jul 25, 2018

itcrowdsource commented Jul 25, 2018 • edited Loading

matburt commented Jul 25, 2018

itcrowdsource commented Jul 26, 2018

matburt commented Jul 27, 2018

itcrowdsource commented Jul 27, 2018

itcrowdsource commented Aug 9, 2018

darthwalsh commented Nov 10, 2020

itcrowdsource commented Jul 21, 2018 •

edited by matburt

Loading

itcrowdsource commented Jul 25, 2018 •

edited

Loading

itcrowdsource commented Jul 25, 2018 •

edited

Loading