API report URL is `http://` link

[bwbroersma]

domains.$domain.report.url starts with http://

Should be https://

Related code:

Internet.nl/interface/batch/util.py

Line 310 in 77dde1c

result["report"] = {"url": f"{site_url}{reverse(url_name, args=args)}"}

which uses:

Internet.nl/interface/batch/util.py

Lines 165 to 170 in 77dde1c

	def get_site_url(request):
	"""
	Compose the url that the user used to connect to the API.
	"""
	return f"{request.scheme}://{request.get_host()}"

But it probably does not have a real request context as a task, so the scheme defaults to http?

Internet.nl/interface/batch/util.py

Lines 244 to 245 in 77dde1c

	@batch_shared_task(bind=True, ignore_result=True)
	def batch_async_generate_results(self, user, batch_request, site_url):

---

If it would be a real request, the following holds:

Because nginx is terminating the TLS, it sets a proxy header X-Forwarded-Proto with $scheme:

Internet.nl/docker/webserver/nginx_templates/app.conf.template

Line 156 in 77dde1c

proxy_set_header X-Forwarded-Proto $scheme;

This requires that SECURE_PROXY_SSL_HEADER is set, which is conditionally set:

Internet.nl/internetnl/settings.py

Lines 602 to 603 in 77dde1c

	if DJANGO_IS_PROXIED:
	SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https")

which is fetched by:

Internet.nl/internetnl/settings.py

Line 48 in 77dde1c

DJANGO_IS_PROXIED = get_boolean_env("DJANGO_IS_PROXIED", False)

and by default set on True:

Internet.nl/docker/compose.yaml

Line 137 in 77dde1c

- DJANGO_IS_PROXIED=True

The relevant Django 4.2.x code is here:
https://github.com/django/django/blob/stable/4.2.x/django/http/request.py#L246-L270

[aequitas]

This is probably because the reports are created on a worker container instead of the app container. For the worker containers the DJANGO_IS_PROXIED is not yet configured:

Internet.nl/docker/compose.yaml

Lines 264 to 305 in 77dde1c

	environment:
	- INTERNET_NL_CHECK_SUPPORT_IPV6
	- INTERNET_NL_CHECK_SUPPORT_DNSSEC
	- INTERNET_NL_CHECK_SUPPORT_MAIL
	- INTERNET_NL_CHECK_SUPPORT_TLS
	- INTERNET_NL_CHECK_SUPPORT_APPSECPRIV
	- INTERNET_NL_CHECK_SUPPORT_RPKI
	- PUBLIC_SUFFIX_LIST_URL
	- ENABLE_BATCH
	- ENABLE_HOF
	- RABBIT_HOST=$IPV4_IP_RABBITMQ_INTERNAL:15672
	- SECRET_KEY
	- GENERATE_SECRET_KEY
	- DB_HOST=$IPV4_IP_POSTGRES_INTERNAL
	- DB_NAME=internetnl_db1
	- DB_USER=internetnl
	- DB_PASSWORD=password
	- CELERY_BROKER_URL=amqp://guest:guest@$IPV4_IP_RABBITMQ_INTERNAL:5672//
	- CELERY_RESULT_BACKEND=redis://$IPV4_IP_REDIS_INTERNAL:6379/0
	- CACHE_LOCATION=redis://$IPV4_IP_REDIS_INTERNAL:6379/0
	- DJANGO_SETTINGS_MODULE=internetnl.settings
	- STATSD_HOST=$IPV4_IP_STATSD_INTERNAL
	- DEBUG
	- DEBUG_LOG
	- DEBUG_LOG_UNBOUND
	- CELERY_LOG_LEVEL
	- INTEGRATION_TESTS
	- INTERNETNL_LOG_LEVEL
	- INTERNETNL_CACHE_TTL
	- ROUTINATOR_URL
	- CONN_TEST_DOMAIN
	- SMTP_EHLO_DOMAIN
	- INTERNETNL_BRANDING
	- INTERNETNL_DOMAINNAME
	- USER_AGENT_URL
	- IPV4_IP_RESOLVER_INTERNAL_VALIDATING
	- SENTRY_DSN
	- SENTRY_ENVIRONMENT
	- SENTRY_SERVER_NAME
	# even though eventlet is used and childs are not really threads or processes, this settings reduces
	# issues with high memory usage, probably because some objects are freed?
	- CELERYD_MAX_TASKS_PER_CHILD=100

[bwbroersma]

Check, but even in the worker, is there a real request with scheme (implying a nginx proxy with X-Forwarded-Proto set to https), I would not think so?

[bwbroersma]

Discusses with @aequitas: probably best to add a configurable scheme, which is http in test.env/develop.env, and https by default.

[bwbroersma]

Also note: a developer release 're-test' is linking to internet.test (maybe this should never be hard-coded to a domain).