Task status not working when using DynamoDB as backend

[ravigadila]

celery==4.4.2
kombu==4.6.8

I have 2 different python projects orders_proj and api_project.
Tasks are available in orders_proj. I'm sending tasks from api_proj to orders_proj.
When we use Redis as backend task status showing correct, but when moved to DynamoDB as backend task status always showing pending.

Sending Task(api_proj):

(venv) ➜  v2_api git:(celery_result) ✗ ipython
Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from tasks.celery_conf import make_ao_celery 
   ...: from api.app import get_app 
   ...: app = get_app() 
   ...: celery_app = make_ao_celery(app)

# send task to other python project
In [3]: celery_task = celery_app.send_task('tasks.order_ta
   ...: sk.start_order_with_worker', kwargs={"order_details": d
   ...: ata})                                                  

# check status after finishing task from orders_proj and after result saved in DynamoDB
In [4]: celery_task.status                                     
2020-05-12 20:17:45,067 - botocore.credentials - INFO - Found credentials in environment variables.
[2020-05-12 20:17:45] | INFO | credentials.py | load:1059 | Found credentials in environment variables.

Out[4]: 'PENDING' # this is wrong status. Actual status should be FAILURE

In [5]: celery_task.backend.as_uri()                           
Out[5]: 'dynamodb://us-west-2/ao_celery_result'

In [6]: celery_app.broker_connection()                         
Out[6]: <Connection: amqp://guest:**@localhost:5672/autods at 0x7f9b9f212080>

Task result saved correctly in DynamoDB.

above Task Output

(aenv) ➜  orders git:(celery_result) ✗ celery -A tasks.order_task.app worker
 
 -------------- celery@ravi v4.4.2 (cliffs)
--- ***** ----- 
-- ******* ---- Linux-5.3.0-51-generic-x86_64-with-Ubuntu-18.04-bionic 2020-05-12 20:17:03
- *** --- * --- 
- ** ---------- [config]
- ** ---------- .> app:         tasks:0x7f7295d0f190
- ** ---------- .> transport:   amqp://guest:**@localhost:5672/autods
- ** ---------- .> results:     dynamodb://us-west-2/ao_celery_result
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** ----- 
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery
                
................
................
    raise Error("wrong")
Error:  wrong

But when checking status from orders project where actual task exicuted showing correct status.

(aenv) ➜  orders git:(celery_result) ✗ ipython
Python 2.7.17 (default, Apr 15 2020, 17:20:14) 
Type "copyright", "credits" or "license" for more information.

In [1]: from tasks.order_task import app
In [2]: from celery.result import AsyncResult
In [3]: celery_task = AsyncResult('8209379c-6a6b-4670-93d3-cb2e
   ...: 65204b4e', app=app)
In [4]: celery_task.status
Out[4]: 'FAILURE'
In [5]: celery_task.backend.as_uri()
Out[5]: u'dynamodb://us-west-2/ao_celery_result'
In [6]: app.broker_connection()
Out[6]: <Connection: amqp://guest:**@localhost:5672/autods at 0x7f7cb5121510>

When tried to change backend to redis everything works fine.
Only phasing issue when moved to DynamoDB.

[auvipy]

did you dig deeper into it?

[ravigadila]

@auvipy I tried all possible ways to fetch results for DynamoDB it's not working.
DynamoDB works with a single project where tasks and calling tasks in same project.
phasing issue when checking task status from a different project.

[miha93]

having this exact issue with DynomoDB too. Is there an update on this ticket?

[ravigadila]

any update on above issue?