Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jobs in error stays in Pending state #733

Closed
u1735067 opened this issue Jul 23, 2021 · 1 comment · Fixed by #930
Closed

Jobs in error stays in Pending state #733

u1735067 opened this issue Jul 23, 2021 · 1 comment · Fixed by #930
Labels
type: bug Something isn't working as expected

Comments

@u1735067
Copy link
Contributor

Environment

  • Python version: 3.9
  • Nautobot version: 1.1.0

Steps to Reproduce

  1. Start an early failing job as in Jobs launched with API error out (and stays in Pending state) #732

Expected Behavior

Error is reported in the UI, job state's changes to Errored

Observed Behavior

Job state's stays in Pending

Additional informations

Log example of errored job still displayed as pending:

celery_worker_1  | [2021-07-23 18:54:10,983: INFO/MainProcess] Task nautobot.extras.jobs.run_job[e43f026b-1361-4879-9906-19593fcc10f6] received
celery_worker_1  | 18:54:11.002 INFO    nautobot.jobs :
celery_worker_1  |   Repository successfully refreshed
celery_worker_1  | [2021-07-23 18:54:11,002: INFO/ForkPoolWorker-3] Repository successfully refreshed
celery_worker_1  | [2021-07-23 18:54:11,016: ERROR/ForkPoolWorker-3] Task nautobot.extras.jobs.run_job[e43f026b-1361-4879-9906-19593fcc10f6] raised unexpected: AttributeError("'str' object has no attribute 'items'")
celery_worker_1  | Traceback (most recent call last):
celery_worker_1  |   File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 450, in trace_task
celery_worker_1  |     R = retval = fun(*args, **kwargs)
celery_worker_1  |   File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 731, in __protected_call__
celery_worker_1  |     return self.run(*args, **kwargs)
celery_worker_1  |   File "/usr/local/lib/python3.9/site-packages/nautobot/extras/jobs.py", line 961, in run_job
celery_worker_1  |     data = job_class.deserialize_data(data)
celery_worker_1  |   File "/usr/local/lib/python3.9/site-packages/nautobot/extras/jobs.py", line 345, in deserialize_data
celery_worker_1  |     for field_name, value in data.items():
celery_worker_1  | AttributeError: 'str' object has no attribute 'items'
celery_worker_1  | [2021-07-23 15:31:08,187: ERROR/ForkPoolWorker-2] Task nautobot.extras.jobs.run_job[c34588d4-3957-49db-b019-759a4adf72d4] raised unexpected: TypeError('the JSON object must be str, bytes or bytearray, not int')
celery_worker_1  | Traceback (most recent call last):
celery_worker_1  |   File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 450, in trace_task
celery_worker_1  |     R = retval = fun(*args, **kwargs)
celery_worker_1  |   File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 731, in __protected_call__
celery_worker_1  |     return self.run(*args, **kwargs)
celery_worker_1  |   File "/usr/local/lib/python3.9/site-packages/nautobot/extras/jobs.py", line 934, in run_job
celery_worker_1  |     job_result = JobResult.objects.get(pk=job_result_pk)
celery_worker_1  |   File "/usr/local/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
celery_worker_1  |     return getattr(self.get_queryset(), name)(*args, **kwargs)
celery_worker_1  |   File "/usr/local/lib/python3.9/site-packages/cacheops/query.py", line 353, in get
celery_worker_1  |     return qs._no_monkey.get(qs, *args, **kwargs)
celery_worker_1  |   File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 425, in get
celery_worker_1  |     num = len(clone)
celery_worker_1  |   File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 269, in __len__
celery_worker_1  |     self._fetch_all()
celery_worker_1  |   File "/usr/local/lib/python3.9/site-packages/cacheops/query.py", line 283, in _fetch_all
celery_worker_1  |     self._result_cache = list(self._iterable_class(self))
celery_worker_1  |   File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 70, in __iter__
celery_worker_1  |     for row in compiler.results_iter(results):
celery_worker_1  |   File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1100, in apply_converters
celery_worker_1  |     value = converter(value, expression, connection)
celery_worker_1  |   File "/usr/local/lib/python3.9/site-packages/django/db/models/fields/json.py", line 83, in from_db_value
celery_worker_1  |     return json.loads(value, cls=self.decoder)
celery_worker_1  |   File "/usr/local/lib/python3.9/json/__init__.py", line 339, in loads
celery_worker_1  |     raise TypeError(f'the JSON object must be str, bytes or bytearray, '
celery_worker_1  | TypeError: the JSON object must be str, bytes or bytearray, not int

Please note that I don't know how to reproduce the latter, but would you like that I fill a but with this log?

@jathanism
Copy link
Contributor

Thanks for another detailed report @u1735067! This is a pretty straightforward fix. :)

@jathanism jathanism added type: bug Something isn't working as expected status: current labels Jul 27, 2021
glennmatthews pushed a commit that referenced this issue Oct 8, 2021
#930)

* Added exceptions to catch data validation errors.

* Fixed lintting error.

* Extend try and except

* Added test for when data is a string.

* Raise changed to TypeError. Improved test log grabbing. Try/except changed back to Exception catching.

* Linting fix

* Possible error in catch if job is not defined, fixed.

* Adding error catching for job object initialisation

* Fix lintting error

* Removed unused import

* Improved error messagign back to Nautobot.

* Fixed log messages, removed duplicate code.

* Removed unused variable

* Revert duplicate changes.

* Removed duplicate code

* Empty
glennmatthews added a commit that referenced this issue Oct 8, 2021
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jan 7, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
type: bug Something isn't working as expected
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants