Priority for VirtualService when conflict happen

[Yisaer]

Describe the feature request
Hi, I'm wondering is it possible to set priority for each VirtualService when multi VirtualService Match get conflict.

In my case , I have 2 services , A and B. And my virtualservice configuration is following YAML.

When I visit www.example.com/api/xxx ,i noticed that the http request was arranged to the Service-A by istio-ingressgateway.

It seems the request also hit the virtualService Match of Service-A .

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: vsc-A
spec:
  hosts:
    - "www.example.com"
  gateways:
  - example-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        port:
          number: 80
        host: service-A
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: vsc-B
spec:
  hosts:
    - "www.example.com"
  gateways:
  - example-gateway
  http:
  - match:
    - uri:
        prefix: /api
    route:
    - destination:
        port:
          number: 80
        host: service-B

Describe alternatives you've considered

I guess we can add define priority attribute in virtualService in following way.
As 99 means the lowest level and 0 means the highest level, the request in my case will be arranged to the service-B

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: A
spec:
  hosts:
    - "www.example.com"
  gateways:
  - example-gateway
  http:
  - match:
    - uri:
        priority: 99
        prefix: /
    route:
    - destination:
        port:
          number: 80
        host: service-A
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: account-service-external-vsc
spec:
  hosts:
    - "www.example.com"
  gateways:
  - example-gateway
  http:
  - match:
    - uri:
        priority: 0
        prefix: /api
    route:
    - destination:
        port:
          number: 80
        host: service-B

Additional context

[hzxuzhonghu]

@Yisaer You can merge the two virtualservices. Currently istio will match virtualservices by creation timestamp.

[Yisaer]

@hzxuzhonghu Thanks for your reply. After I merge 2 VirtualService in folloing YAML, How can it guarantee the HTTP request for www.example.com/api/xxx will be delivered to the Service-B ,not Service-A?

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: vsc-A
spec:
  hosts:
    - "www.example.com"
  gateways:
  - example-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        port:
          number: 80
        host: service-A
  - match:
    - uri:
        prefix: /api
    route:
    - destination:
        port:
          number: 80
        host: service-B

[hzxuzhonghu]

The priority is by order, you should also change the match rule order

[mehemken]

This forces a significant deploy order for VirtualServices.

I vote no.

What if I need to update a VirtualService that needs to have the highest priority? Then I need to redeploy all the other VirtualServices too?

Can we just have sensible defaults that can be overridden with some priority setting at the VirtualService level?

EDIT
For what it's worth, the defaults are already quite sensible. I just need to let my deploy guy deploy in whatever order he wants. Who am I to tell him how to do his job?

EDIT 2
Actually we're just going to combine all the VirtualServices into a single resource.

EDIT 3
Actually this is really annoying. We really need a better way to define which VirtualService takes priority. I have several VirtualServices that I cannot combine because they handle different hosts. However some of the backend prefixes clash. I'm stuck with one VirtualService that is eating all the requests that need to go to another VirtualService.

EDIT 4
For what it's worth, why can't we just look at the most specific path first? This would fix everything.

[yogin16]

+1 for the above suggestions.

I agree that VirtualService is an improvement on RouteRule to not be giving precedence for every route. (https://github.com/istio/istio/blob/master/samples/bookinfo/networking/ROUTING_RULE_MIGRATION.md)

But for exactly the reasons outlined here: https://istio.io/docs/ops/traffic-management/deploy-guidelines/#multiple-virtual-services-and-destination-rules-for-the-same-host

We could have a case that the different backend services (different teams) do not have a deployment order defined.

Which would lead to issues like #10485 (and this one)

Anyway, we could also have an optional precedence or priority number field in the config to define explicit order? on top of the ordered list of VirtualService.

or EDIT 4 of @mehemken could also work -

For what it's worth, why can't we just look at the most specific path first? This would fix everything.

find the rule which matches the most. (but that would mean matching all rules for every incoming request)

[edwardzjl]

Why is this issue closed? I met a similar situation, is there any solution?

[bittelc]

I would like to reopen this issue. We have a significant issue here with managing distributed VirtualServices. Having the most specific route match take priority, as @mehemken mentioned, would help. Although I can imagine this would also introduce issues (eg. how do you compare specificity between a regex and prefix match rule).

I propose a different solution. Though I need to specify that I'm unfamiliar with the codebase here, so I don't know how much work it would be.

1. Prevent entirely conflicting / overriding matching rules to other VirtualServices
2. Do not prevent conflicting / overriding matching rules within a single VirtualService

Why 1:
Currently, there is no way to enforce prioritization. The priorities are loosely handed to the matching VS rule that was mostly recently created. This makes it an extremely difficult task for operations / infrastructure teams to manage distributed VSs. However, this distributed VS capability is a requirement, and is the actual recommendation from Istio itself!
The best microservice developer experience would be to manage their own VS, and be notified when they attempt to create a VS rule that conflicts with another team's VS rule. Then those two teams work together to come to an agreement.
This takes ownership of prioritizing rules off of the DevOps teams.

Why 2:
There is a somewhat acceptable way to manage prioritization within the same VS at the moment- the controller reads the VS rules from top to bottom. While this isn't exactly ideal, it allows a microservice team to prioritize how their traffic. The microservice team is now already in charge of resolving conflicting routing paths, and has full autonomy to address their own problems. The DevOps teams are again relieved of the job of managing rule prioritization.

[choldrim]

I vote reopen this issue too. Priority feature can make vs more flexible. And all virtual services with same hosts will be merged by create timestamp, the order is out of control.

For example with a problem we are having now:
We have multiple virtual services with a same host, we need to set a default route to this host, but we can't add the default route to any of virtual service cause we can't ensure the orders of merged virtual services.

[dacloutier]

@hzxuzhonghu Is there anyway to reopen this issue to find a working solution for VS route priorities?

[hzxuzhonghu]

I donot think there is anyone working on it

[shakti-das]

We are also facing the exact same issue with multiple VS tied up to the same host. In our case, the catch-all is per port, so we do have a match condition on the port number, but for URI it is still catch-all and we have multiple VS defining URI-based routes for the same port.

[ErikXu]

I am facing a similar scenario. I think the logic is sensible despite order or creation timestamp:

a. Priority: exact > prefix > regex

b. The longer route would always override the shorter one when using prefix or regex uri, eg:
/api/serviceA/action >/api/serviceA

c. The header matched route would override the other one when they both matched in uri, eg:
/api/serviceA/action > /api/serviceA + header: version=1 >/api/serviceA

[ErikXu]

@hzxuzhonghu Is it a graceful way to reload or reorder the virtual service ?

[hzxuzhonghu]

For all the above comments: the http match conditions are complex, code can not handle it well.

type HTTPMatchRequest struct {
	// The name assigned to a match. The match's name will be
	// concatenated with the parent route's name and will be logged in
	// the access logs for requests matching this route.
	Name string `protobuf:"bytes,11,opt,name=name,proto3" json:"name,omitempty"`
	// URI to match
	// values are case-sensitive and formatted as follows:
	//
	// - `exact: "value"` for exact string match
	//
	// - `prefix: "value"` for prefix-based match
	//
	// - `regex: "value"` for RE2 style regex-based match (https://github.com/google/re2/wiki/Syntax).
	//
	// **Note:** Case-insensitive matching could be enabled via the
	// `ignore_uri_case` flag.
	Uri *StringMatch `protobuf:"bytes,1,opt,name=uri,proto3" json:"uri,omitempty"`
	// URI Scheme
	// values are case-sensitive and formatted as follows:
	//
	// - `exact: "value"` for exact string match
	//
	// - `prefix: "value"` for prefix-based match
	//
	// - `regex: "value"` for RE2 style regex-based match (https://github.com/google/re2/wiki/Syntax).
	//
	Scheme *StringMatch `protobuf:"bytes,2,opt,name=scheme,proto3" json:"scheme,omitempty"`
	// HTTP Method
	// values are case-sensitive and formatted as follows:
	//
	// - `exact: "value"` for exact string match
	//
	// - `prefix: "value"` for prefix-based match
	//
	// - `regex: "value"` for RE2 style regex-based match (https://github.com/google/re2/wiki/Syntax).
	//
	Method *StringMatch `protobuf:"bytes,3,opt,name=method,proto3" json:"method,omitempty"`
	// HTTP Authority
	// values are case-sensitive and formatted as follows:
	//
	// - `exact: "value"` for exact string match
	//
	// - `prefix: "value"` for prefix-based match
	//
	// - `regex: "value"` for RE2 style regex-based match (https://github.com/google/re2/wiki/Syntax).
	//
	Authority *StringMatch `protobuf:"bytes,4,opt,name=authority,proto3" json:"authority,omitempty"`
	// The header keys must be lowercase and use hyphen as the separator,
	// e.g. _x-request-id_.
	//
	// Header values are case-sensitive and formatted as follows:
	//
	// - `exact: "value"` for exact string match
	//
	// - `prefix: "value"` for prefix-based match
	//
	// - `regex: "value"` for RE2 style regex-based match (https://github.com/google/re2/wiki/Syntax).
	//
	// If the value is empty and only the name of header is specfied, presence of the header is checked.
	// **Note:** The keys `uri`, `scheme`, `method`, and `authority` will be ignored.
	Headers map[string]*StringMatch `protobuf:"bytes,5,rep,name=headers,proto3" json:"headers,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"`
	// Specifies the ports on the host that is being addressed. Many services
	// only expose a single port or label ports with the protocols they support,
	// in these cases it is not required to explicitly select the port.
	Port uint32 `protobuf:"varint,6,opt,name=port,proto3" json:"port,omitempty"`
	// One or more labels that constrain the applicability of a rule to source (client) workloads
	// with the given labels. If the VirtualService has a list of gateways specified
	// in the top-level `gateways` field, it must include the reserved gateway
	// `mesh` for this field to be applicable.
	SourceLabels map[string]string `protobuf:"bytes,7,rep,name=source_labels,json=sourceLabels,proto3" json:"source_labels,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"`
	// Names of gateways where the rule should be applied. Gateway names
	// in the top-level `gateways` field of the VirtualService (if any) are overridden. The gateway
	// match is independent of sourceLabels.
	Gateways []string `protobuf:"bytes,8,rep,name=gateways,proto3" json:"gateways,omitempty"`
	// Query parameters for matching.
	//
	// Ex:
	//
	// - For a query parameter like "?key=true", the map key would be "key" and
	//   the string match could be defined as `exact: "true"`.
	//
	// - For a query parameter like "?key", the map key would be "key" and the
	//   string match could be defined as `exact: ""`.
	//
	// - For a query parameter like "?key=123", the map key would be "key" and the
	//   string match could be defined as `regex: "\d+$"`. Note that this
	//   configuration will only match values like "123" but not "a123" or "123a".
	//
	// **Note:** `prefix` matching is currently not supported.
	QueryParams map[string]*StringMatch `protobuf:"bytes,9,rep,name=query_params,json=queryParams,proto3" json:"query_params,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"`
	// Flag to specify whether the URI matching should be case-insensitive.
	//
	// **Note:** The case will be ignored only in the case of `exact` and `prefix`
	// URI matches.
	IgnoreUriCase bool `protobuf:"varint,10,opt,name=ignore_uri_case,json=ignoreUriCase,proto3" json:"ignore_uri_case,omitempty"`
	// withoutHeader has the same syntax with the header, but has opposite meaning.
	// If a header is matched with a matching rule among withoutHeader, the traffic becomes not matched one.
	WithoutHeaders map[string]*StringMatch `protobuf:"bytes,12,rep,name=without_headers,json=withoutHeaders,proto3" json:"without_headers,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"`
	// Source namespace constraining the applicability of a rule to workloads in that namespace.
	// If the VirtualService has a list of gateways specified in the top-level `gateways` field,
	// it must include the reserved gateway `mesh` for this field to be applicable.
	SourceNamespace      string   `protobuf:"bytes,13,opt,name=source_namespace,json=sourceNamespace,proto3" json:"source_namespace,omitempty"`
	XXX_NoUnkeyedLiteral struct{} `json:"-"`
	XXX_unrecognized     []byte   `json:"-"`
	XXX_sizecache        int32    `json:"-"`
}

But looks like introducing priority for http route is necessary and is an imperious demand.

[ErikXu]

@hzxuzhonghu I think the rule that first matched route override the others might be a performance consideration.
I could adjust the route order ourself.
But I need a graceful way to reload the virtual service (not delete then recreate).

[hzxuzhonghu]

What do you mean by reload

[ErikXu]

For example, I created a virtual service using kubectl apply -f service-a.yaml.

# service-a.yaml 
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: service-a-vs
spec:
  hosts:
    - "www.example.com"
  gateways:
  - example-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        port:
          number: 80
        host: service-a

Then I have a new route:

# service-b.yaml 
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: service-b-vs
spec:
  hosts:
    - "www.example.com"
  gateways:
  - example-gateway
  http:
  - match:
    - uri:
        prefix: /api
    route:
    - destination:
        port:
          number: 80
        host: service-b

I don't want to use the below commands to recreate.

kubectl delete -f service-a.yaml
kubectl apply -f service-b.yaml 
kubectl apply -f service-a.yaml 

Instead, I want to use the below commands to reload.

kubectl apply -f service-b.yaml 
kubectl apply -f service-a.yaml 

[hzxuzhonghu]

Look my comment above #10464 (comment)

[ErikXu]

@hzxuzhonghu Just for imagination, is it possible to provide a crd like sub virtual service, and also provide an operator watching the new crds then merge the crds and apply?

[ErikXu]

I've implement an operator and a crd "sub virtual service" to handle the priority. Please fell free to check whether it's useful for you.
https://github.com/ErikXu/virtual-service-go

[amitde69]

implementing the priority field is definitely a must.
we have a monolith which we are breaking down into microservices and all requests are sent with the same host.
we cant distribute the routing rules for each microservice since its all being matched to the shorter uri which should actually be the last one matched if no other rules are met.

I've implement an operator and a crd "sub virtual service" to handle the priority. Please fell free to check whether it's useful for you. https://github.com/ErikXu/virtual-service-go

maybe you can dig into the code and contribute this to the original istio solution ?

[houstonheat]

Same as @amitde69 /stale, voting for reopen

[hzxuzhonghu]

Ingress only has path match, while vs provides so many match conditions. They are different.

[hzxuzhonghu]

@howardjohn @ramaraochavali This problem is worth a solution, it is usually a big blocker for people setting up south-north traffic policy. I am totally agree it should align with gateway API merging strategy

[ramaraochavali]

We discussed this some where else and John also commented here - May be we should add api/feature flag in VS to indicate match_merge_semantics_with_gateway_api or some thing like that?

[hzxuzhonghu]

Agree, for any compatibility feature we should have one flag

[chlunde]

We can't do the same thing as gateway-api since it wouldn't be backwards compatible (it's not a new field, just different implicit behavior)

https://istio.io/latest/docs/ops/best-practices/traffic-management/#split-virtual-services

As the current behaviour is undefined, I think it's not a breaking change (or requires a flag) to make it defined?

[Stevenpc3]

You could send headers to do a header match along with the service so you are checking on two areas for the route. Then if service a has a route match of /path and service b has a route match of /path there would be an addition check on the header. I have not tested it yet so I am not aware of any precedence issues. You could even have a service c with no header and I assume that any non header traffic would go there. The header could be { path : service-a } then the VS would match path = /path and header key : path with key : service-a .

[AlanMasciangelo]

@hzxuzhonghu I understand they're different, I only meant the behavior to give precedence to the longest path match makes intuitive sense for any "prefix" type path matching behavior.

[howardjohn]

We can't do the same thing as gateway-api since it wouldn't be backwards compatible (it's not a new field, just different implicit behavior)

https://istio.io/latest/docs/ops/best-practices/traffic-management/#split-virtual-services

As the current behaviour is undefined, I think it's not a breaking change (or requires a flag) to make it defined?

This would be an egregious breaking change that would cause front page news outages. "not well documented" does not imply we can break everyone unfortunately. There are literally millions of virtualservices in production at this point

[dotbalo]

There doesn't seem to be any plans to improve it, right?

[hzxuzhonghu]

There is an improvement recently in #48071

[keithmattix]

#48071 fixes scenarios where there are overlapping wildcards; overlapping specific hosts should still be oldest first

[Stevenpc3]

I realize now that my suggestion earlier will not help with North South traffic. It would only really safely work with internal traffic.

It would make more sense if it was just ordered by path and put in reverse order so the longest match is first like @AlanMasciangelo suggested above.

All the istiod logs output are the same so I assume they all get the same values and could easily add this to a tree. Then in match just march down the tree as far as you can.

Based on current usage of adding a VS it appears to just add the match in order of the VS for prefix. You can tell this because it you simply add multiple matches in a single VS with the longer first it works like below

Match
/Api
Match
/

The problem, as established, is when it's split over several VS there is no guaranteed order, but I would imagine the proxy could order this just fine with the correct data structure like a tree or trie when it gets added.

This change should be backwards compatible as well since any work around would very likely still work and going from undefined behavior (random) to defined (ordered) is not breaking.

Of course there is a case for people that already take advantage of order for some reason, which likely is the way I mentioned above, but folks be crazy sometimes so a feature flag for order by string or whatever the current way is called would be good.

Also, in reading the gateway conflict resolution spec https://gateway-api.sigs.k8s.io/concepts/guidelines/?h=conflict#conflicts
I have concern with the order. The compare of creation timestamp I'm sure is meant for cases where an established server has had a route and doesn't want some new group to take traffic a few months later, but it has the likely unintended effect of blocking the last rule which is the expected order behavior. If you apply multiple VS they will of course vary by nano second at the very least. Which is the likely cause of this first come first serve issue. I would remove the timestamp compare completely or at the very least... Make it last which makes much more sense. If paths are all equal THEN and only then should creation timestamp matter.

[Hronom]

Hey istio maintainers, any news about this? Strange to see this not here, we have same situation where we need to have priority in resolution of it.

[howardjohn]

Hey istio maintainers, any news about this? Strange to see this not here, we have same situation where we need to have priority in resolution of it.

The current approach is to use the HTTPRoute which has this baked in.

[Hronom]

@howardjohn I don't see in HTTPRoute possibility to set priority. This is from VirtualService.

Question here is how to handle situations properly when we have several VirtualService's that serves traffic for the same url.

Example:

1. VisrtualService 1: http://example.com/api -> microservice1
2. VisrtualService 2: http://example.com/api/accounts -> microservice2

Need have possibility to set high priority for prefix /api/accounts and route traffic to microservice2 instead of microservice1.

Can you provide a solution here?

[howardjohn]

@Hronom my apologies, the terms are overloaded. I meant https://gateway-api.sigs.k8s.io/api-types/httproute/?h=httpro#merging, which has a clearly defined merging order

[hzxuzhonghu]

We have added priority to envoyfilter, the same explicit setting could be applied to vs.

[Hronom]

@howardjohn thank you! Unfortunately we are not migrated yet to gateway api, is this works for service to service communication? (not edge to service)

We have added priority to envoyfilter, the same explicit setting could be applied to vs.

@hzxuzhonghu this will be awesome! And I think this resolves this task and satisfy our needs. For now we stuck with vs

Thank you a lot all of you!

[AnshumanTripathi]

We have added priority to envoyfilter, the same explicit setting could be applied to vs.

@hzxuzhonghu can you explain how did this? We are running into this problem ourselves and have migrated to using Gateway API. If we can priority by envoy filters then that is path we can consider.

[hzxuzhonghu]

It is not implemented yet. I am not sure how gateway API can solve this

[howardjohn]

Gateway API solves this by having a precedence order for routes that is decouple from list ordering (https://gateway-api.sigs.k8s.io/api-types/httproute/?h=httpro#merging)

[hzxuzhonghu]

Yeah, that is ideal, in real use case users can still meet conflict. Even not sure how this ordered, not to mention regex semantics.

	// * Method match.
	// * Largest number of header matches.
	// * Largest number of query param matches.

So in a word, k8s HTTPRoute almost have similar issue as istio VS

[chc5]

Can we publicly document this creation timestamp behavior today on https://istio.io/latest/docs/ops/best-practices/traffic-management/#split-virtual-services if we are not planning to modify this behavior?

[StyleTang]

Hi Team, sorry to bother, I want to confirm if I only have one virtualserivce and it have 2 matches.
First match: header + uri ===> aaa
Second match: uri ===> aaa-old

If my request has both header and uri, I can 100% route to aaa(First one)?
If not, any suggestions?

I checked the document, thespec.http.match list order should works?

If ties still exist within an HTTPRoute, matching precedence MUST be granted to the FIRST matching rule (in list order) with a match meeting the above criteria.

Thanks!

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: istio-rollout-vsvc
spec:
  gateways:
  - istio-rollout-gateway
  hosts:
  - istio-rollout.apps.argoproj.io
  http:
  - match:
    - headers:
        sss:
          exact: latest
      uri:
        prefix: /aaa/
    route:
    - destination:
        host: aaa
  - match:
     - uri:
        prefix: /aaa/
    route:
    - destination:
        host: aaa-old

[hzxuzhonghu]

Yes, you are right, first route mateched can be guaranteed to be routed to