Strongly typed TypedArray values

[aapoalas]

🔍 Search Terms

"TypedArray" "TypedArray value" "indexed value" "value type"

✅ Viability Checklist

• This wouldn't be a breaking change in existing TypeScript/JavaScript code
• This wouldn't change the runtime behavior of existing JavaScript code
• This could be implemented without emitting different JS based on the types of the expressions
• This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
• This isn't a request to add a new utility type: https://github.com/microsoft/TypeScript/wiki/No-New-Utility-Types
• This feature would agree with the rest of our Design Goals: https://github.com/Microsoft/TypeScript/wiki/TypeScript-Design-Goals

⭐ Suggestion

Enable defining the value type of TypedArrays, similarly to how an Array's value type can be defined using T[] or Array<T>. This would probably take the form of an additional generic parameter on TypedArray types with a default of number applied to keep backwards compatibility.

While Arrays can be defined to use an arbitrary T as their value type, TypedArrays always use the plain number. This is runtime-typing-wise correct, of course, but compared to the Array-case it is possible to define an Array that at runtime contains numbers but at the type level contains a subset of numbers, eg. 1 | 2 | 4 or number & { [BRAND]: unknown }.

TypedArrays, lacking this ability, are harder to use in a code base that wants to use strongly-typed numbers through value unions or branded types. (A further feature which is not litigated here is strongly-typed indexing; the same kind of code base would quite possibly want to define an Array or TypedArray to only be indexed by a given number & { [BRAND]: unknown }.)

📃 Motivating Example

A TypedArray is used to store unique identifiers, such as indexes (handles) to a particular global storage array.

const GLOBAL_DATA_ARRAY: unknown[] = [];

declare const GLOBAL_DATA_HANDLE_BRAND: unique symbol;
type GlobalDataHandle = number & { [GLOBAL_DATA_HANDLE_BRAND]: unknown };

const addData = (data: unknown): GlobalDataHandle => {
  const existing = GLOBAL_DATA_ARRAY.indexOf(data);
  if (existing !== -1) {
    return existing as GlobalDataHandle;
  }
  const added = GLOBAL_DATA_ARRAY.length as GlobalDataHandle;
  GLOBAL_DATA_ARRAY.push(data);
  return added;
};

class Foo {
  // currently:
  #storage: Uint8Array | Uint16Array | Uint32Array;
  // in the future:
  // #storage: Uint8Array<ArrayBufferLike, GlobalDataHandle> | Uint16Array<ArrayBufferLike, GlobalDataHandle> | Uint32Array<ArrayBufferLike, GlobalDataHandle>;
  #storageLength: number;

  pushData(data: unknown): number {
    this.ensureStorage(1); // reallocate #storage if it is full
    const index = this.#storageLength;
    this.#storage[index] = addData(1); // okay; GlobalDataHandle is a number.
    return index;
  }

  getDataHandle(index: number): GlobalDataHandle {
    return this.#storage[index]; // error: number is not a GlobalDataHandle.
  }
}

If #storage was a plain Array, this same would work without any real issue, but if the code base wants to take advantage of the (possibly) smaller memory footprint and stronger runtime value type guarantees of TypedArrays, they must add a lot of as GlobalDataHandle assertions; these assertions are then effectively uncheckable and as a result leads to possibly missing some errors when refactoring.

💻 Use Cases

1. Strongly typing TypedArray values to eg. constrain a particular TA to contain "data handles", another to contain "relative indexes", another to contain "data bitsets", and all of these TA value types to be strongly typed so as to be unique from one another and thus not be interchangeable.
2. Currently, TypedArray & T[] can sometimes be used but methods on the type become basically unusable, and even the value types seem to sometimes become just plain number.
3. The "best" thing to do today is to wrap access to the TypedArray into helper functions that as assert the value type: this comes at a small runtime cost as the function does not disappear even though it is entirely trivial in effect.

[jcalz]

This seems like a lot of complexity for a use case that isn't very common. What you can do today is copy the typed array types and make your own generic versions of them, like

interface MyUint8Array<N extends number = number, TArrayBuffer extends ArrayBufferLike = ArrayBufferLike> {
    /** snip **/
    copyWithin(target: number, start: number, end?: number): this;
    every(predicate: (value: N, index: number, array: this) => unknown, thisArg?: any): boolean;
    fill(value: N, start?: number, end?: number): this;
    filter(predicate: (value: N, index: number, array: this) => any, thisArg?: any): MyUint8Array<N, ArrayBuffer>;
    find(predicate: (value: N, index: number, obj: this) => boolean, thisArg?: any): N | undefined;
    /** etc **/
    [index: number]: N;
    toLocaleString(locales: string | string[], options?: Intl.NumberFormatOptions): string;
    [Symbol.iterator](): ArrayIterator<N>;
    entries(): ArrayIterator<[number, N]>;
    keys(): ArrayIterator<number>;
    values(): ArrayIterator<N>;
}

and then make your own constructor type for them, etc., etc. Basically you can go through the TS lib and make the changes you want to see, just to your own copies of the types, with things like var MyUint8Array = Uint8Array as MyUint8ArrayConstructor if you only want to use an assertion once. And then your code would work like

class Foo {
    #storage!: MyUint8Array<GlobalDataHandle> | MyUint16Array<GlobalDataHandle> | MyUint32Array<GlobalDataHandle>;
    getDataHandle(index: number): GlobalDataHandle {
        return this.#storage[index]; 
    }
}

Tedious, yes, but this is effectively what you're asking for in the feature request. And you don't have to wait for TS to do it for you.

Playground link

[aapoalas]

This seems like a lot of complexity for a use case that isn't very common.

Hmm, I would have assumed that an extra type parameter would be fairly easy to add. The use case definitely isn't very common, though that may be partially because it is not available to the average TypeScript user today without a tedious copying of types.

Tedious, yes, but this is effectively what you're asking for in the feature request.

This is indeed what I am asking for, yes.

Specifying a custom type instead is indeed quite tedious, and eg. breaks when using any kind of generic helper APIs that produce TypedArrays from inputs (such as taking a list of column types, byte widths, and a capacity to allocate a table of columns automatically). Of course, an as cast is all you need to fix the issue again but it's not the greatest experience in the world. Having to keep up with any possible updates to the upstream TypedArray methods etc. is another downside. It does have the upside that the key type could be made parametric as well, giving support to strongly keyed and valued TypedArrays, akin to a Record<CustomKey, CustomValue>.

And you don't have to wait for TS to do it for you.

Would a PR to discuss the effective complexity of this be welcome?

[jcalz]

I'm not a TS team member, just an annoying interested community member, so I'm just telling you what you can do in the (likely, imo) event that this is not implemented . If you want to actually modify the types in your local copy of TS instead of using a custom type that would also work. Yes, you would need to keep up with upstream updates. I can't say whether an experimental PR would be welcomed by the TS team. I'll bow out now and wait for them to reply.

[Bertie690]

Chiming in to say this would IMO be a good addition to TS' native types (potentially locked behind a feature flag to avoid backwards compatibility issues).

Our codebase does have the aforementioned hand-rolled typed array interfaces... 500 lines of worth in fact!

/**
 * A partial interface of `Int16Array` where methods that return the array type have been modified to return a more specific type.
 *
 * @privateRemarks
 * Excludes methods that return a new array (e.g. `subarray`, `slice`, `toReversed`, `toSorted`, `with`) as these cannot be resolved by typescript
 * @internal
 */
interface PartialInt16Array<T extends number> extends Int16Array {
  at(index: number): T | undefined; // ES2022
  entries(): ArrayIterator<[number, T]>;
  fill(value: T, start?: number, end?: number): this;
  find(predicate: (value: T, index: number, array: this) => boolean, thisArg?: any): T | undefined;
  findLast(predicate: (value: number, index: number, array: this) => boolean, thisArg?: any): T | undefined;
  forEach(callbackfn: (value: T, index: number, array: this) => void, thisArg?: any): void;
  includes(searchElement: T, fromIndex?: number): boolean;
  indexOf(searchElement: T, fromIndex?: number): number;
  lastIndexOf(searchElement: T, fromIndex?: number): number;
  reduce(callbackfn: (previousValue: T, currentValue: T, currentIndex: number, array: this) => T): T;
  reduceRight(callbackfn: (previousValue: T, currentValue: T, currentIndex: number, array: this) => T): T;
  set(array: ArrayLike<T>, offset?: number): void;
  some(predicate: (value: T, index: number, obj: this) => boolean, thisArg?: any): boolean;
  sort(compareFn?: (a: T, b: T) => number): this;
  values(): ArrayIterator<T>;
  [Symbol.iterator](): ArrayIterator<T>;
  [index: number]: T;
}

Notably, we actively need to include @ts-expect-errors in certain cases due to the overriding of various return types, and simply can't override some methods at all due to type compatibility issues.

Having these types in the standard library would be both nice and remove the need to use error suppression simply to get subtype restrictions.