Instance Group or Job fails with AttributeError: 'NoneType' object has no attribute 'image'

[balonik]

ISSUE TYPE

• Bug Report

SUMMARY

In UI Instance Groups ends up with Error 500
It is not possible to run ANY Job.

api/v2/instance_groups/1/ returns

{
    "id": 1,
    "type": "instance_group",
    "url": "/api/v2/instance_groups/1/",
    "related": {
        "named_url": "/api/v2/instance_groups/tower/",
        "jobs": "/api/v2/instance_groups/1/jobs/",
        "instances": "/api/v2/instance_groups/1/instances/"
    },
    "name": "tower",
    "created": "2020-12-03T09:59:18.467551Z",
    "modified": "2021-04-08T08:56:34.198408Z",
    "capacity": 0,
    "committed_capacity": 0,
    "consumed_capacity": 0,
    "percent_capacity_remaining": 0.0,
    "jobs_running": 0,
    "jobs_total": 446,
    "instances": 0,
    "controller": null,
    "is_controller": false,
    "is_isolated": false,
    "is_container_group": true,
    "credential": null,
    "policy_instance_percentage": 100,
    "policy_instance_minimum": 0,
    "policy_instance_list": [],
    "pod_spec_override": "apiVersion: v1\nkind: Pod\nmetadata:\n  namespace: awx\nspec:\n  containers:\n    - image: registry.local/awx-ee:latest\n      name: worker\n      args:\n        - ansible-runner\n        - worker",
    "summary_fields": {
        "user_capabilities": {
            "edit": true,
            "delete": true
        }
    }
}

AWX spec snippet

apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
  namespace: awx
spec:
  tower_ee_images:
    - name: my default
      image: registry.local/awx-ee:latest
...

ENVIRONMENT

• AWX version: 19.0.0
• AWX install method: awx-operator 0.8.0 on RKE
• Ansible version: X.Y.Z
• Operating System:
• Web Browser:

STEPS TO REPRODUCE

Deploy AWX 19.0.0 using awx-operator 0.8.0 with custom tower_ee_images defined in YAML file

EXPECTED RESULTS

Working

ACTUAL RESULTS

HTTP 500 or Job Error

ADDITIONAL INFORMATION

UI error logs from awx-web container

2021-04-08 08:48:58,582 ERROR    [3ffbb21adba449eba374ddd15505ecd3] django.request Internal Server Error: /api/v2/instance_groups/
Traceback (most recent call last):
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib64/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/django/views/generic/base.py", line 71, in view
    return self.dispatch(request, *args, **kwargs)
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/api/generics.py", line 324, in dispatch
    return super(APIView, self).dispatch(request, *args, **kwargs)
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/rest_framework/views.py", line 520, in options
    data = self.metadata_class().determine_metadata(request, self)
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/api/metadata.py", line 245, in determine_metadata
    metadata = super(Metadata, self).determine_metadata(request, view)
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/rest_framework/metadata.py", line 68, in determine_metadata
    actions = self.determine_actions(request, view)
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/api/metadata.py", line 209, in determine_actions
    meta['default'] = AWXReceptorJob().pod_definition
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/main/tasks.py", line 3081, in pod_definition
    default_pod_spec = get_default_pod_spec()
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/main/utils/execution_environments.py", line 21, in get_default_pod_spec
    "image": get_default_execution_environment().image,
AttributeError: 'NoneType' object has no attribute 'image'

Jobs error from Job output

Traceback (most recent call last):
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/main/tasks.py", line 1397, in run
    res = receptor_job.run()
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/main/tasks.py", line 2957, in run
    return self._run_internal(receptor_ctl)
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/main/tasks.py", line 2973, in _run_internal
    result = receptor_ctl.submit_work(worktype=self.work_type, payload=sockout.makefile('rb'), params=self.receptor_params)
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/main/tasks.py", line 3037, in receptor_params
    spec_yaml = yaml.dump(self.pod_definition, explicit_start=True)
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/main/tasks.py", line 3081, in pod_definition
    default_pod_spec = get_default_pod_spec()
  File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/main/utils/execution_environments.py", line 21, in get_default_pod_spec
    "image": get_default_execution_environment().image,
AttributeError: 'NoneType' object has no attribute 'image'

[shanemcd]

The default EE is supposed to be created when the operator runs awx-manage create_preload_data here.

Can you please provide:

• Your version of awx-operator
• The output of kubectl get awx awx -o yaml

[balonik]

Hi @shanemcd
version of awx-operator (already written in the issue description) is 0.8.0

which part or properties of the awx spec are you interested? I don't believe you need the whole and I would need to sanitize it anyway,
I do have the following for ee_images and disabled the preload as I don't want it.

spec:
  tower_create_preload_data: false
  tower_ee_images:
  - image: registry.local/awx-ee:latest
    name: EGIT default

[shanemcd]

tower_create_preload_data: false <- this is your problem.

[balonik]

tower_create_preload_data: false <- this is your problem.

I had this feeling when you mentioned it is create in create_preload_data.
create_preload_data used to only create data for demonstration purposes. Example Inventory, Project, .... I don't want any of that. In 17.1.0 and older disabling it didn't mean AWX will be unusable completely.
Did it change? Why is default EE connected to some sample data generation?

Unless there is some reason why is a main part of the application connected to sample data I would consider this a bug.

[rebeccahhh]

removing myself because I believe #10321 will address this

[AlanCoding]

Is there anything else that needs to be done here? Moving to needs_test.

[Zokormazo]

Tested with an operator including ansible/awx-operator#356 , the command is working properly on awx side