You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
celery failed to run the command in subprocess. when restarting the celery worker the command is running in the background.
the subprocess is working without calling as celery task
Checklist
I have verified that the issue exists against the master branch of Celery.
train_obj = subprocess.Popen(
cmd,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
shell=True
)
for line in iter(train_obj.stdout.read(1), ''): # replace '' with b'' for Python 3
sys.stdout.write(line)
Expected Behavior
subprocess need to be start and stdout can able to see in the logs
Actual Behavior
stopped before the line of subprocess
restarting the worker. the command is running in the background. not able to see in the logs
celery failed to run the command in subprocess. when restarting the celery worker the command is running in the background.
the subprocess is working without calling as celery task
Checklist
master
branch of Celery.contribution guide
on reporting bugs.
for similar or identical bug reports.
for existing proposed fixes.
to find out if the bug was already fixed in the master branch.
in this issue (If there are none, check this box anyway).
Mandatory Debugging Information
celery -A proj report
in the issue.(if you are not able to do this, then at least specify the Celery
version affected).
master
branch of Celery.pip freeze
in the issue.to reproduce this bug.
Optional Debugging Information
and/or implementation.
result backend.
broker and/or result backend.
ETA/Countdown & rate limits disabled.
and/or upgrading Celery and its dependencies.
Related Issues and Possible Duplicates
Related Issues
Possible Duplicates
Environment & Settings
Celery version:
celery report
Output:software -> celery:4.4.0 (cliffs) kombu:4.6.11 py:3.6.11
billiard:3.6.3.0 py-amqp:2.6.1
platform -> system:Linux arch:64bit, ELF
kernel version:5.3.0-51-generic imp:CPython
loader -> celery.loaders.app.AppLoader
settings -> transport:pyamqp results:rpc:///
broker_url: 'amqp://guest:********@192.168.43.150:5672//'
result_backend: 'rpc:///'
task_queue: <unbound Queue test_subprocess -> <unbound Exchange test_subprocess(direct)> -> model_builder.#>
task_routes: {
'*': {'queue': 'test_subprocess'}}
Python Packages
pip freeze
Output:absl-py==0.9.0 affine==2.3.0 amqp==2.6.1 astor==0.8.1 attrs==20.1.0 Babel==2.8.0 billiard==3.6.3.0 celery==4.4.0 certifi==2020.6.20 chardet==3.0.4 click==7.1.2 click-plugins==1.1.1 cligj==0.5.0 cycler==0.10.0 Cython==3.0a6 Flask==1.1.1 Flask-Cors==3.0.8 flower==0.9.3 gast==0.3.3 gevent==1.4.0 google-pasta==0.2.0 greenlet==0.4.16 grpcio==1.25.0 grpcio-tools==1.25.0 h5py==2.10.0 idna==2.10 importlib-metadata==1.7.0 itsdangerous==1.1.0 Jinja2==2.11.2 Keras-Applications==1.0.8 Keras-Preprocessing==1.1.2 kiwisolver==1.2.0 kombu==4.6.11 Markdown==3.2.2 MarkupSafe==1.1.1 matplotlib==3.3.1 minio==5.0.6 numpy==1.19.1 object-detection==0.1 opencv-python==4.1.2.30 pandas==1.1.0 pika==1.1.0 Pillow==7.0.0 protobuf==3.12.2 pyparsing==2.4.7 python-dateutil==2.8.1 python-dotenv==0.11.0 pytz==2020.1 rasterio==1.1.3 redis==3.3.11 requests==2.23.0 scipy==1.5.2 simplification==0.4.4 six==1.15.0 snuggs==1.4.7 tensorboard==1.14.0 tensorflow==1.14.0 tensorflow-estimator==1.14.0 termcolor==1.1.0 tf-slim==1.1.0 tornado==5.1.1 urllib3==1.25.10 vine==1.3.0 Werkzeug==1.0.1 wrapt==1.12.1 zipp==3.1.0
reference subprocess code
Expected Behavior
subprocess need to be start and stdout can able to see in the logs
Actual Behavior
stopped before the line of subprocess
restarting the worker. the command is running in the background. not able to see in the logs
similar question in stack overflow
https://stackoverflow.com/questions/36995868/how-to-execute-a-long-running-subprocess-inside-celery-task
The text was updated successfully, but these errors were encountered: