Skip to content

Сelery task from another project run from a Fastapi project cannot delete a file using os.remove() #1971

@FyZzyss

Description

@FyZzyss

When I try to delete a temporary file using the broker I gave the FastAPI task to, all the code is executed except os.remove(), subprocess.Popen(['rm', source_path]) and so on. If I run a task from a project where the task is written, everything works well.

My endpoint code:

@router.post(
    path="/po/{id_po}/upload",
    response_model=schema.ContentUploadedResponse,
)
async def upload_file(
        id_po: int,
        # background_tasks: BackgroundTasks,
        uploaded_file: UploadFile = File(...)):
    """pass"""
    with open(f"{uploaded_file.filename}", "wb") as fp:
        fp.write(await uploaded_file.read())
    await samizdat_file_upload(uploaded_file=os.path.abspath(fp.name), id_po=id_po)     
   #background_tasks.add_task(samizdat_file_upload, uploaded_file=os.path.abspath(fp.name), id_po=id_po)

    return schema.ContentUploadedResponse()

My celery initialization code:

APP = Celery('async_download_file')
APP.config_from_object(CONFIG['celery'])


async def samizdat_file_upload(uploaded_file: str, id_po: int):
    """Background task that upload file from samizdat"""
    APP.send_task(name='async_download_file.download_file',
                  args=([uploaded_file], id_po),
                  queue=CONFIG['celery']['task_routes']['samizdat_celery_task']['queue'])

P.S If I call a function that bores directly, and not through the endpoint, then everything works... This means that the endpoint is somehow distorting the task.

from lr.config import CONFIG
from celery import Celery

APP = Celery('async_download_file')
APP.config_from_object(CONFIG['celery'])


def adapter(uploaded_file: str, id_po: int):
    samizdat_file_upload(uploaded_file, id_po)


def samizdat_file_upload(uploaded_file: str, id_po: int):
    """Background task that upload file from samizdat"""
    APP.send_task(name='async_download_file.download_file',
                  args=([uploaded_file], id_po),
                  queue=CONFIG['celery']['task_routes']['samizdat_file_upload']['queue'])


if __name__ == '__main__':
    adapter('/home/fyzzy/some_picture.jpg', 12)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions