[core-http] New Pipeline Design RFC

[xirzec]

Redesigning Pipelines

Goals

Today it is not very easy to customize pipelines in core-http as they are presented as a difficult-to-inspect array of policy factories. The new design seeks to:

• Make it easy to compose pipelines by expressing dependencies between policies
• Make it easy to remove policies that are unnecessary (so as to replace them with new custom policies)
• Avoid unnecessary allocations after creating a policy, such that new isn't required to be called when executing a particular policy per operation.

Design

Policies

A PipelinePolicy is an object that implements the following interface:

type SendRequest = (request: Request) => Promise<Response>;

interface PipelinePolicy {
    name: string;
    sendRequest(request: Request, next: SendRequest): Promise<Response>;
}

It is given a string name (policies defined in core will use strings from an exported string enum) so that consumers can express dependencies relative to existing policies. The sendRequest method uses a middleware pattern, which is familiar to JS devs.

Open Question: Is there a way to type responses here for deserialization policies?

Pipelines

A Pipeline composes several policies and provides the ability to make a request that will flow through the policies in a specific order determined by their requirements.

interface AddPipelineOptions  {
    beforePolicy?: string | string[];
    afterPolicy?: string | string[];
    beforePhase?: string | string[];
    afterPhase?: string | string[];
    duringPhase?: string | string[];
}

interface Pipeline {
    addPolicy(policy: PipelinePolicy, options?: AddPipelineOptions);
    removePolicy(options: { name?: string; phase?: string });
    sendRequest(httpClient: HttpClient, request: Request): Promise<Response>;
    getOrderedPolicies(): PipelinePolicy[];
    clone(): Pipeline;
}

Dependencies expressed when calling addPolicy are not sensitive to order. The following code snippets have identical effects:

pipeline.addPolicy(barPolicy);
pipeline.addPolicy(fooPolicy, { afterPolicy: 'barPolicy' });

// The below is the same as the above. 
// Policies don't have to be part of the pipeline in order to be 
// referenced by before/afterPolicy.
pipeline.addPolicy(fooPolicy, { afterPolicy: 'barPolicy' });
pipeline.addPolicy(barPolicy);

Phases are predefined buckets of activity that pre-defined policies inside core will fall into. They are currently a fixed set and not customizable by pipeline consumers.

Current proposed phases:

1. Serialize
2. Retry

Known phase names will be exported as a constant string enum.

Champion Scenarios

Inserting a new policy at a specific place

interface TracingPolicyOptions {
    tracingEnabled: boolean;
}

function tracingPolicy(options: TracingPolicyOptions): PipelinePolicy {
    async function sendRequest(request: Request, next: SendRequest): Promise<Response> {
        if (options.tracingEnabled) {
            // add some headers or something!
        }
        return next(request);
    }
    return {
        name: "TracingPolicy",
        sendRequest
    }
}

// Extend an existing Pipeline
pipeline.addPolicy(
    tracingPolicy({tracingEnabled: true}), 
    {beforePhase: PipelinePhases.Retry}
);

Replacing retry policies

pipeline.removePolicy({phase: PipelinePhases.Retry);
pipeline.addPolicy(customRetryPolicy, {duringPhase: PipelinePhases.Retry})

Other Considerations

• How to create/customize default pipelines in service clients
• How to customize pipelines per client operation

[xirzec]

/cc @chradek @bterlson @ramya-rao-a @daviwil @jeremymeng

[xirzec]

Simplified phases down to just Retry and Serialize.