Automatic detection of JS breaking changes

[deyaaeldeen]

API Guard - JS

Breaking client code degrades our customer experience and will likely hurt Azure adoption. Such breaking changes must be detected before making a release and be handled according to the Azure SDK design guidelines:

Breaking changes can be classified into source and behavioral breaking changes:

• Source breaking changes cause compilation errors next time the client code gets re-compiled.
  • Example: making an optional property in an input model required.
• Behavioral breaking changes alter the program behavior in an unexpected way.
  • Example: throwing an error in a code path that didn't throw before.

This work tracks building apiguard, a tool to detect source breaking changes between two input extracted APIs.

Design

The tool contains the following components:

• parsing: rich JSON parser to parse the two input api.json files into a TypeScript code model
• diffing two code models into a change list model
• breaking changes rules
• APIView plugin that understands the flagged breaking changes to highlight them

Parsing

We use the API Extractor to generate *.api.json files that gets uploaded to https://apiview.dev/ and it is powered by @microsoft/api-extractor-model, a library for reading and writing those *.api.json files. This library can be leveraged to do the desired parsing.

Diffing

TBD

Rules list

TBD

APIView plugin

TBD

[xirzec]

Making an attempt at pinning down some heuristics. I'm ready to be heavily edited/corrected for spec correctness.

An interface/type is considered a "return type" if it:

1. Is returned by a method
2. Is a publicly available property on a client or client-like

An interface/type is considered an "input type" if it:

1. Is used as the type of a parameter to a method or constructor

An interface/type may be used as both a return type and an input type.
A type may be anonymous and not have a name (e.g. foo(): A | B returns an anonymous type)

Now some specific rules:

1. Any rename of a public type or property IS considered breaking.
   • A special case of this is if the rename is only affecting type names and would not be observable to a JS consumer

// Bar could be renamed and not break JS consumers, but it should still be flagged as breaking TS consumers.
interface Bar {
   name: string;
}
function foo() : Bar;

3. Adding an additional required property on an input type IS considered breaking.
4. Changing a required property to be optional on a return type IS considered breaking.
5. Narrowing an input type (e.g. removing a member of a type union) IS considered breaking
6. Widening a return type (e.g. returning a new shape or extending a type union) IS considered breaking.

I think a lot of the complexity here will be spidering through nested types, e.g.

interface Feature {
   name: string;
   // adding a new required property to `Feature` is breaking even though it is not directly used as an input
}
interface Widget {
  features: Feature[];
}
class FactoryClient {
   produce(widget: Widget): ProductionResponse;
}

[deyaaeldeen]

@timovv I think you have a simpler design to address the goal documented above. Is it documented somewhere?