Skip to content

Using request.host in expressions for metrics is problematic at ingress gateway #18818

Closed
@douglas-reid

Description

@douglas-reid

Bug description

In #16892, we added request.host to the expression for destination_service labels. This was added as a fallback for cases when destination.service.host would not be set (pass-thru, blackhole, NR).

In theory, this is a nice idea. However, this is problematic at ingress, as the Host header can be set to arbitrary values.

Example of sending such a request:

curl --verbose --header 'Host: %22%3E%3Cscript%3Ealert('Qualys_XSS_Joomla_2.5.3')%3C%2Fscript%3E' <gateway ip>/no-route

This results in metrics with a destination service label containing junk -- some of it potentially harmful.

Example from Prometheus:

image

This is problematic for a number of reasons:

  • allows arbitrary explosion of cardinality of metric, as bad actors could send an unending stream of requests with all different Host headers
  • pollution of the service variables in grafana dashboards
  • potential XSS, etc., when rendering metrics with bad request.host

Expected behavior
Arbitrary NR traffic at ingress should not be able to pollute istio generated metrics with junk destination_service labels.

Either we should set destination.service.host to be "BlackHole" or similar OR we should exclude request.host from the expression for destination_service in configuration for ingressgateway (or a third, better option that someone may have).

Steps to reproduce the bug

Install Istio, have a gateway for wildcard host on port 80. Have a virtual service with an exact match route and nothing else. send traffic to gw not matching the route with a host header with junk info. observe results.

Version (include the output of istioctl version --remote and kubectl version)
1.4.0-beta.2, but any version of istio with the linked PR is affected.

How was Istio installed?
helm template

Environment where bug was observed (cloud vendor, OS, etc)
GKE

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions