Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Guides - Evaluator grammar fixes #193

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions site/content/en/docs/Guides/Evaluator/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ description: >

In a basic flow to generate Matches, the Director calls FetchMatches on Open Match backend for one or more MatchProfiles. Open Match triggers execution of MatchFunctions for each of these requested MatchProfiles concurrently. Each MatchFunction queries Open Match for the Tickets that Match the filtering criteria specified for the Pools in their MatchProfile. The MatchFunction then generates Matches using the Tickets it fetched and returns them as proposals.

If the MatchProfiles perfectly partition the concurrent Ticket Pool (their Pool filters are such that each MatchProfile will only Match a non overlapping set of Tickets), then the Matches generated by the MatchFunction execution do not have any overlap.
If the MatchProfiles perfectly partition the concurrent Ticket Pool (their Pool filters are such that each MatchProfile will only Match a non-overlapping set of Tickets), then the Matches generated by the MatchFunction execution do not have any overlap.

However, this may not always be the case. There are scenarios in which, the Pools specified in concurrent MatchProfiles may have an overlap and thus concurrently executing MatchFunctions could use overlapping Ticket in its proposals.

Here is an example scenario for clarification:
Lets assume a simple case where a Ticket has an attribute 'mmr' that can be a value between 0 and 10.
Let's assume a simple case where a Ticket has an attribute 'mmr' that can be a value between 0 and 10.

### Case1 - Non-overlapping MatchProfiles

Expand All @@ -36,27 +36,27 @@ The Director could request for the following Profiles:

In this case, the MatchFunction executing Profile1 and Profile2 will have some common players they consider for generating Matches (Similarly with Profile2 and Profile3). Thus, the generated Matches may have overlapping Tickets.

This overlap is desired in some real matchmaking scenarios, . Inorder to generate high quality Matches, you may want your Ticket to be considered concurrently for multiple possible MatchProfiles and then be able to compare the quality of these overlapping Matches, picking the highest quality Match and discarding the rest. For instance in the above example, a Ticket with mmr 3 could be a part of a Matches generated by Profile1 and Profile2. Based on the game design and the mmr of other Tickets in these overlapping Matches, one of these two Matches may actually be a better fit for this Ticket - and thus, searching for Matches with overlapping mmr ranges resulted in considering 2 options for this Ticket and picking the better fit.
This overlap is desired in some real matchmaking scenarios. In order to generate high-quality Matches, you may want your Ticket to be considered concurrently for multiple possible MatchProfiles and then be able to compare the quality of these overlapping Matches, picking the highest quality Match and discarding the rest. For instance, in the above example, a Ticket with mmr 3 could be a part of a Matches generated by Profile1 and Profile2. Based on the game design and the mmr of other Tickets in these overlapping Matches, one of these two Matches may actually be a better fit for this Ticket - and thus, searching for Matches with overlapping mmr ranges resulted in considering 2 options for this Ticket and picking the better fit.

The ability to provide this model of executing concurrent MatchFunctions on overlapping Ticket Pool is a core value proposition of Open Match.

## Evaluator

### Why do we need an Evaluator

To allow generation of proposals using overlapping Tickets but still provide unique Matches to the Director, we need to decollide the proposals. To achieve this, Open Match has an Evaluator component that accepts all concurrent Match proposals, evaluates them for collision and returns non-overlapping results back to Open Match. If there are multiple overlapping Matches, the Evaluator evaluates their quality, promotes the highest quality Match as a result and discards the others. The key responsibility of the Evaluator is to ensure that there are no overlapping Tickets in the result Matches returned.
To allow the generation of proposals using overlapping Tickets but still provide unique Matches to the Director, we need to decollide the proposals. To achieve this, Open Match has an Evaluator component that accepts all concurrent Match proposals, evaluates them for collision and returns non-overlapping results back to Open Match. If there are multiple overlapping Matches, the Evaluator evaluates their quality, promotes the highest quality Match as a result, and discards the others. The key responsibility of the Evaluator is to ensure that there are no overlapping Tickets in the result Matches returned.

### How to use an Evaluator

Open Match provides a default score based Evaluator that can be used in most common scenarios but it also provides the user the ability to build your own Evaluator and to customize Open Match to use the user's Evaluator. At a high level, there are three evaluation scenarios for you to consider when using Open Match:
Open Match provides a default score-based Evaluator that can be used in most common scenarios but it also provides the user the ability to build your own Evaluator and to customize Open Match to use the user's Evaluator. At a high level, there are three evaluation scenarios for you to consider when using Open Match:

1. Your MatchProfiles completely partition the Ticket Pool and so you will never have collisions. The default Evaluator handles this scenario and you do not have to make any changes.
2. Your MatchProfiles may use overlapping Ticket Pools - but each MatchFunction can simply generate a quality score for that Match based on certain game specific criteria (latency, mmr etc.). The default Evaluator can be used for this scenario. See [Default Evaluator Tutorial]({{< relref "../../Tutorials/DefaultEvaluator/_index.md" >}})
2. Your MatchProfiles may use overlapping Ticket Pools - but each MatchFunction can simply generate a quality score for that Match based on certain game-specific criteria (latency, mmr, etc.). The default Evaluator can be used for this scenario. See [Default Evaluator Tutorial]({{< relref "../../Tutorials/DefaultEvaluator/_index.md" >}})
3. You have complex evaluation logic that cannot be simplified to a score for a Match but rather involves actually comparing some details of the overlapping Matches. Open Match provides you with the ability to plug in your custom Evaluator to handle this case. See the [Custom Evaluator Tutorial]({{< relref "../../Tutorials/CustomEvaluator/_index.md" >}}) for details.

### How to configure an Evaluator

Inorder to invoke the Evaluator at runtime, Open Match needs to know the location of the Evaluator endpoint. This is provided to Open Match as an override ConfigMap. Here are the field in that config map that specifically configure an Evaluator in Open Match:
In order to invoke the Evaluator at runtime, Open Match needs to know the location of the Evaluator endpoint. This is provided to Open Match as an override ConfigMap. Here is the field in that config map that specifically configures an Evaluator in Open Match:

```yaml
api:
Expand All @@ -69,7 +69,7 @@ Inorder to invoke the Evaluator at runtime, Open Match needs to know the locatio

### Why do we need a Synchronizer

Concurrently executing MatchFunctions fetch Tickets for their MatchProfiles from Open Match at the same time. Thus, the same active Ticket Pool is visible to multiple MatchFunctions (resulting in collision). Although Evaluation handles decolliding the overlapping results, there is a race condition. While the Evaluator is evaluating Match proposals, if MatchFunction execution is permitted, then the MatchFunctions may possible fetch Tickets that were already promoted as results. There is thus a need to stop execution of MatchFunctions (thereby halting new observations from being made) while existing results are being evaluated. This is called Synchronization.
Concurrently executing MatchFunctions fetch Tickets for their MatchProfiles from Open Match at the same time. Thus, the same active Ticket Pool is visible to multiple MatchFunctions (resulting in a collision). Although Evaluation handles decolliding the overlapping results, there is a race condition. While the Evaluator is evaluating Match proposals, if MatchFunction execution is permitted, then the MatchFunctions may possibly fetch Tickets that were already promoted as results. There is thus a need to stop the execution of MatchFunctions (thereby halting new observations from being made) while existing results are being evaluated. This is called Synchronization.

### Overview of Synchronizer States

Expand All @@ -81,7 +81,7 @@ The synchronizer is in Idle state when there is no ongoing MatchFunction executi

#### Registration

Open Match backend registers each FetchMatch request with the Sychronizer before it calls into the MatchFunction. The Synchronizer enters the Registration window (configurable) upon first FetchMatches registration . All the MatchFunctions that get registered in this registration window will be evaluated together.
Open Match backend registers each FetchMatch request with the Sychronizer before it calls into the MatchFunction. The Synchronizer enters the Registration window (configurable) upon first FetchMatches registration. All the MatchFunctions that get registered in this registration window will be evaluated together.

#### ProposalCollection

Expand Down