Calling DRF API internally within same server with urllib3 not working

[mr-engin3er]

What happened?

I have one project where I created two API and I am calling one API internally when someone is calling other API.

What should've happened instead?

Without Docker (cookie cutter) it is working fine, but not working in cookie-cutter project.

Additional details

Trying on local machine
Used Docker with cookie cutter

Django function views:-

def parent(request):
    return JsonResponse({"some": "response"})

@csrf_exempt
def child(request):
     url = request.POST.get("url")
     http = urllib3.PoolManager(maxsize=10, block=False)
     response = http.request("GET", url, timeout=10).data
     return JsonResponse(response)

I added two view so I am calling parent view endpoint from child view.
My Django server hangs after below log for infinite time if I didn't add timeout
url--------------------> http://172.22.0.5:8000/project/parent DEBUG 2022-09-27 15:09:22,223 connectionpool 80 139840035317504 Starting new HTTP connection (1): 172.22.0.5:8000

after adding timeout I am getting
raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='172.22.0.5', port=8000): Max retries exceeded with url: /project/parent (Caused by ReadTimeoutError("HTTPConnectionPool(host='172.22.0.5', port=8000): Read timed out. (read timeout=10)")) INFO: 192.168.1.7:34684 - "POST /project/child/ HTTP/1.1" 500 Internal Server Error

I tried with different hosts but got the same response.
tried hosts are:- localhost, my host system IP, docker container IP

And I can access both endpoints from Postmen without any issue with above hosts

I am able to get response with the same code on local python repl as well as from shell_plus inside different Django container and even the same container python repl.

• Host system configuration:

  • Version of cookiecutter CLI (get it with cookiecutter --version):

  • OS name and version: Ubuntu 20.04
    On Linux, run

    lsb_release -a 2> /dev/null || cat /etc/redhat-release 2> /dev/null || cat /etc/*-release 2> /dev/null || cat /etc/issue 2> /dev/null

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.5 LTS"
NAME="Ubuntu"
VERSION="20.04.5 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.5 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal     

* Python version, run `python3 -V`: 3.9-slim-bullseye (from dockerfile)
* Django version: django==3.2.12
* Docker version (if using Docker), run `docker --version`:  Docker version 20.10.18, build b40c2f6
* docker-compose version (if using Docker), run `docker-compose --version`:  docker-compose version 1.29.1, build c34c88b2
* ...

• Options selected and/or replay file:
  On Linux and MacOS: cat ${HOME}/.cookiecutter_replay/cookiecutter-django.json
  (Please, take care to remove sensitive information)

  {
  "cookiecutter": {
  "project_name": "name_replaced",
  "project_slug": "name_replaced",
  "description": "dummy.",
  "author_name": "mr-engin3er",
  "domain_name": "domain.com",
  "email": "admin@domain.com",
  "version": "0.1.0",
  "open_source_license": "Not open source",
  "timezone": "Asia/Kolkata",
  "windows": "n",
  "use_pycharm": "n",
  "use_docker": "y",
  "postgresql_version": "14.1",
  "js_task_runner": "None",
  "cloud_provider": "GCP",
  "mail_service": "Mailjet",
  "use_async": "y",
  "use_drf": "y",
  "custom_bootstrap_compilation": "n",
  "use_compressor": "n",
  "use_celery": "n",
  "use_mailhog": "y",
  "use_sentry": "n",
  "use_whitenoise": "n",
  "use_heroku": "n",
  "ci_tool": "Github",
  "keep_local_envs_in_vcs": "y",
  "debug": "y",
  "_template": "https://github.com/cookiecutter/cookiecutter-django"
  }

}
```

Logs:

$ cookiecutter https://github.com/cookiecutter/cookiecutter-django
project_name [Project Name]: ...

[browniebroke]

Interesting... It works fine if I starting the server using runserver_plus in PyCharm, but it doesn't work when running docker-compose -f local.yml up django. I see you've used use_async=y, which means we're starting the local server with uvicorn, instead of runserver_plus:

cookiecutter-django/{{cookiecutter.project_slug}}/compose/local/django/start

Lines 9 to 13 in 68f21b6

	{%- if cookiecutter.use_async == 'y' %}
	uvicorn config.asgi:application --host 0.0.0.0 --reload --reload-include '*.html'
	{%- else %}
	python manage.py runserver_plus 0.0.0.0:8000
	{%- endif %}

I'm not sure why it doesn't work in this case. Maybe a protection to avoid circular requests in uvicorn?

Taking a step back, would be interesting to see why you need to do internal requests like this in the first place, that seems like a rather uncommon design. Can you find another way around it?

If you really have no way around it, I would suggest a few things:

• Try with another ASGI server (e.g. Daphne or Hypercorn)
• Try to isolate the problem with a smaller Uvicorn app and bring it up with the uvicorn project, if the other servers don't behave the same way

Not sure it's an issue with cookiecutter django at this point, unless there is a config change that fixes it.

[mr-engin3er]

Hey, @browniebroke Thanks for your response. Due to requirements, I have one API which has a huge nested payload and I don't want to use form data on the client side. So I decided to use JSON payload due to nested objects. That's why I created two separate API's for this. One is for uploading the media files and in the second API I am using the media files URL to get those files internally and using them while creating model objects. I can't directly save previous media file URL's in models due to the custom file system design (path should be like /media/instance.id/file-type/unique-file-name). So I thought I'll fetch media files again and save them into models. Finally, I found another workaround for my requirements instead of hitting API again I am getting the file from the previously saved model.

    media = Model.objects.get( )
    new = NewModel.objects.create(file=media.file.open())

[github-actions]

Assuming the original issue was solved, it will be automatically closed now.