Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support dotted path notation #154

Closed
nandorojo opened this issue Oct 31, 2020 · 80 comments
Closed

Support dotted path notation #154

nandorojo opened this issue Oct 31, 2020 · 80 comments
Assignees
Labels
feature Add new features
Projects

Comments

@nandorojo
Copy link

nandorojo commented Oct 31, 2020

🍩 Feature Request

Is your feature request related to a problem?

TypeScript 4.1 has template literals, and they're super useful. However, the only solutions out there are quite hacky to use, and result in some bugs.

I want to be able to get a nested object path, as shown in this repo.

Describe the solution you'd like

import { Object } from 'ts-toolbelt'

type I = { hi: { hey: true } }

type Nested = Object.NestedPath<I, 'hi.hey'>

I want to know how do I get the nested object path using dot notation of an object. I'd then like to get the value that matches that object with the path.

I'd like to achieve this:

Screen Shot 2020-10-31 at 11 15 12 AM

This is the idea:

type Path<T, Key extends keyof T = keyof T> =
Key extends string
? T[Key] extends Record<string, any>
  ? | `${Key}.${Path<T[Key], Exclude<keyof T[Key], keyof Array<any>>> & string}`
    | `${Key}.${Exclude<keyof T[Key], keyof Array<any>> & string}`
    | Key
  : never
: never;

type PathValue<T, P extends Path<T>> =
  P extends `${infer Key}.${infer Rest}`
  ? Key extends keyof T
    ? Rest extends Path<T[Key]>
      ? PathValue<T[Key], Rest>
      : never
    : never
  : P extends keyof T
    ? T[P]
    : never;


declare function get<T, P extends Path<T>>(obj: T, path: P): PathValue<T, P>;

However, this code block above seems to be buggy. I occasionally get infinite loop errors from typescript if I use it (but not always), so someone who has a better understanding of TS than I do should probably make it.

Describe alternatives you've considered

I tried the code sample above. However, it is not very robust. Someone put it on twitter, but I think it would make more sense for it to live in TS toolbelt (which is always so helpful.)

Teachability, Documentation, Adoption, Migration Strategy

This is a great article describing the template literals. Surprisingly, if you google "TypeScript template literals," it's hard to find content, even though it's such a cool addition. Do you think you might be able to help @millsp? Thank you so much!

@millsp
Copy link
Owner

millsp commented Oct 31, 2020

¡Hola Nando! ¿Como mola?

We already have something similar here https://millsp.github.io/ts-toolbelt/modules/_object_paths_.html

I can start from there and do this for you in the next week, with a better implementation that won't crash on circular references and that is certainly easier to read and understand/maintain :)

@millsp millsp added the feature Add new features label Oct 31, 2020
@millsp millsp self-assigned this Oct 31, 2020
@nandorojo
Copy link
Author

Todo bien!

Thank you, great to hear. A key part of the new syntax is that it enforces strict paths for nested strings. I use object paths, but I don't think it quite does the same for nested for notation of only certain paths.

Thanks again!

@nandorojo
Copy link
Author

PS I think your link is broken

@millsp
Copy link
Owner

millsp commented Oct 31, 2020

Fixed

@millsp
Copy link
Owner

millsp commented Oct 31, 2020

but I don't think it quite does the same for nested for notation of only certain paths

what do you mean?

@nandorojo
Copy link
Author

Screen Shot 2020-10-31 at 4 10 57 PM

Notice how this allows post.type, but it should only allow post.url, or car.type. Code here.

The correct behavior would be like this:

image

Code here.

@millsp
Copy link
Owner

millsp commented Nov 1, 2020

Yep, it works correctly, though. The problem here is that TypeScript replaced ['post', ''] as a type [string, ...] because it widens primitive types in objects - which can definitely be confusing and impractical in your scenario. Reason why we'd rather use the "dot" notation with strings, since string parameters don't undergo such widening.

The implementation I'll work on will have a common limitation with the implementation you linked above: it cannot handle circular references. When this case happens, it won't crash, instead, it will allow any path and not provide auto-completion. Maybe you see a different way of handling this scenario - let me know.

@millsp
Copy link
Owner

millsp commented Nov 1, 2020

Never mind, I was able to solve this limitation by preventing excess depth. It actually improved the previous implementation - so we can now enjoy circular references to a certain extent.

Screenshot from 2020-11-01 14-07-01

@millsp
Copy link
Owner

millsp commented Nov 1, 2020

I limited the output to 10, that's when ts starts complaining that it's too deep (usually 11). I'm going to think over this in the next week to provide you with the full solution.

@nandorojo
Copy link
Author

Wow very cool! Is there any shot you could share your temporary solution here to try in the meantime?

Thank you again.

@millsp
Copy link
Owner

millsp commented Nov 2, 2020

Giving it to you "as is", let me know if we can improve it. I embeds a few safe-guards already. Try to crash it please 🥇

import {A, I, L, M} from 'ts-toolbelt'

type _Paths<O, Paths extends L.List<A.Key> = [], Limit extends I.Iteration = I.IterationOf<'0'>> =
  10 extends I.Pos<Limit>
  ? Paths
  : O extends M.BuiltInObject
    ? Paths
    : O extends object
      ? Paths | {
          [K in keyof O]: _Paths<O[K], L.Append<Paths, K>, I.Next<Limit>>
        }[keyof O]
      : Paths

export type Paths<O extends object> =
    _Paths<O>

type Joinable = string | number | boolean | bigint

type Join<
  L extends L.List<Joinable>,
  D extends Joinable,
  J extends Joinable = '',
  I extends I.Iteration = I.IterationOf<'0'>> = {
    0: Join<L, D, `${J}${D}${L[I.Pos<I>]}`, I.Next<I>>
    1: J extends `.${infer J}` ? J : never
}[A.Extends<I.Pos<I>, L.Length<L>>]

type StringPaths<O extends object> =
  Paths<O> extends infer P
  ? P extends unknown
    ? Join<A.Cast<P, L.List<string | number>>, '.'>
    : never
  : never
declare function get<O extends object, P extends StringPaths<O>>(obj: O, path: P | String): P;

declare const object: O

get(object, '')

type O = {
  h: {
    b: {
      c: {
        d: {
          e: {
            f: O
          }
        }
      }
    }
  },
  b: {
    b: {
      c: {
        d: {
          e: {
            f: O
          }
        }
      }
    }
  },
  c: {
    b: {
      c: {
        d: {
          e: {
            f: O
          }
        }
      }
    }
  },
  d: {
    b: {
      c: {
        d: {
          e: {
            f: O
          }
        }
      }
    }
  }
}

@millsp
Copy link
Owner

millsp commented Nov 2, 2020

Heya, I woke up fresher this morning. Here's a shorter, cleaner version:

import {I, M} from 'ts-toolbelt'

type _PathsDot<O, Paths extends string = '', Limit extends I.Iteration = I.IterationOf<'0'>> =
  11 extends I.Pos<Limit> ? Paths :
  O extends M.BuiltInObject ? Paths :
  O extends object ? Paths | {
    [K in keyof O]: _PathsDot<O[K], `${Paths}.${K & string}`, I.Next<Limit>>
  }[keyof O]
  : Paths

export type PathsDot<O extends object> =
  _PathsDot<O> extends `.${infer P}` | infer _
  ? P
  : ''
declare function get<O extends object, P extends PathsDot<O>>(obj: O, path: P | String): P;

declare const object: O

get(object, '')

type O = {
  h: {
    b: {
      c: {
        d: {
          e: {
            f: O
          }
        }
      }
    }
  },
  b: {
    b: {
      c: {
        d: {
          e: {
            f: O
          }
        }
      }
    }
  },
  c: {
    b: {
      c: {
        d: {
          e: {
            f: O
          }
        }
      }
    }
  },
  d: {
    b: {
      c: {
        d: {
          e: {
            f: O
          }
        }
      }
    }
  }
}

Notice that I added String into the get definition because I felt like we should be able to pass unknown strings too.

@nandorojo
Copy link
Author

Wow, this looks great, I'll test it and report back. Thank you so much!

@millsp
Copy link
Owner

millsp commented Nov 3, 2020

Thanks, I'll include this in the next release - around the 15th of November. I will also create PathDot so that we can get the type at the end of that path.

@millsp millsp changed the title Support TypeScript 4.1 support dotted path notation Nov 3, 2020
@millsp millsp changed the title support dotted path notation Support dotted path notation Nov 3, 2020
@nandorojo
Copy link
Author

@millsp This seems to be working great! The only thing is my linter is giving me errors when using String. Should I be importing that from ts-toolbelt?

Screen Shot 2020-11-03 at 11 48 51 AM

Let me know if I should just ignore that.

Also, it doesn't seem to support objects nested in arrays at the time.

Screen Shot 2020-11-03 at 11 50 55 AM

Not the end of the world, but figured I'd mention it in case you'd know of a solution.

I will also create PathDot so that we can get the type at the end of that path.

Awesome! I assume this means the illustrative get function would return the actual type, rather than P?

Really great work on this overall. ts-toolbelt is always making my life so much easier. Thanks so much.

@millsp
Copy link
Owner

millsp commented Nov 3, 2020

Also, it doesn't seem to support objects nested in arrays at the time.

I saw that earlier, I updated the snippet :) It should work now.

Awesome! I assume this means the illustrative get function would return the actual type, rather than P

Yes, I just left it like that for now. You can probably use the other implementation for now, though I'm not sure it's 100% type-safe.

Let me know if I should just ignore that.

Yep. Because if I used | string it would swallow all the paths into string. since String is an object, this swallowing behavior does not happen.

Really great work on this overall. ts-toolbelt is always making my life so much easier. Thanks so much.

Amazing! Thanks :) Don't hesitate to share this project with others!

@millsp
Copy link
Owner

millsp commented Nov 3, 2020

I see one case where the paths cannot be resolved is where you have a list of unknown size - because it could be empty (and I cannot generate a path for all possible indexes). The solution could be to take steps of get every time the autocomplete falls short. Not sure if this was very clear, let me know.

@millsp
Copy link
Owner

millsp commented Nov 6, 2020

Great news. While programming today, I actually noticed that TypeScript DOES accept to have strings like prop.[number].prop. This is really amazing, because we don't have to worry anymore, it works with arrays too:

https://gist.github.com/millsp/1eec03fbe64592c70efa4c80515f741f

@millsp millsp closed this as completed Nov 6, 2020
@millsp millsp reopened this Nov 6, 2020
@nandorojo
Copy link
Author

Would this work if I did hi[0] syntax too?

@millsp
Copy link
Owner

millsp commented Nov 15, 2020

No, we would need to implement something. I quickly bootstrapped something and since it increases complexity, you would be limited to 6 nested properties instead of 9.

@millsp
Copy link
Owner

millsp commented Nov 15, 2020

If that's ok, I can do that for you (are you building a lib), or was it purely out of curiosity? @nandorojo

@nandorojo
Copy link
Author

nandorojo commented Nov 21, 2020

A few different places: I'm using it in my actual app, and I'm also working on adding strict types to Drispy, a design system I maintain. I want to safely use fields such as theme.colors.primary throughout my app, including nested fields, arrays, and such. We've been working on this here for theme-ui, which Dripsy relies on.

@nandorojo
Copy link
Author

nandorojo commented Nov 21, 2020

If that's ok, I can do that for you (are you building a lib), or was it purely out of curiosity? @nandorojo

That should still work, I would imagine that 6 nested properties should be sufficient, and we can fallback to any non-nullable field at that point. Thank you again!!

@nandorojo
Copy link
Author

Right - I basically want to pass a type as a generic, and get all paths that correspond to that type.

@millsp
Copy link
Owner

millsp commented Feb 2, 2021

Are you looking for exact types? microsoft/TypeScript#12936 Exact types allow for no more and no less properties than their "model". I have this, unpublished atm.

@millsp
Copy link
Owner

millsp commented Feb 2, 2021

@nandorojo, please, would you be so kind to let me know how the "large object" test went, if you have time to test ofc!

@nandorojo
Copy link
Author

Yeah definitely!

@andreialecu
Copy link

@millsp any chance this supports my use case mentioned in #154 (comment) (omitting array index)?

@millsp
Copy link
Owner

millsp commented Feb 2, 2021

@andreialecu Mongo does not have this yet? It would be awesome if you could get ts-toolbelt as a dependency (guess this is on DefinitelyTyped). It's very easy! We just need to do what I told you early on. We create a type that will allow this MongoIfy:

declare function get<O extends object, P extends string>(
    object: O, path: AutoPath<MongoIfy<O>, P>
): Path<MongoIfy<O>, S.Split<P, '.'>>

type MongoIfy<A> = {
    [K in keyof A]: A[K] extends List
    ? MongoIfy<A[K][number] | A[K]>
    : MongoIfy<A[K]>
}

declare const user: User

type User = {
    name: string
    friends: User[]
}

// works
const friendName = get(user, 'friends.friends.name')
const friendFriendName = get(user, 'friends.40')

// errors
const friendNames = get(user, 'friends.40.names')
const friendFriendNames = get(user, 'friends.40.friends.12.names')

@millsp
Copy link
Owner

millsp commented Feb 2, 2021

I have published the latest bugfixes and tests. I can confirm the there are no performance problems. I tested AutoPath with the Window object type.

@andreialecu
Copy link

andreialecu commented Feb 2, 2021

Clever solution. Except it needs to support both the variant with array indexes and non array indexes at the same time. I'll take a look at modifying your snippet tomorrow to try to add that.

A union will probably work but I'm not sure if it won't harm performance.

To clarify: all 4 lines in your example should work.

Nevermind. I was on my phone and didn't read it properly. Seems it already does it.

Thanks!

@nandorojo
Copy link
Author

@millsp I'm using Formik here, creating a checkbox component:

import React from 'react'
import { Function } from 'ts-toolbelt'
import { useFieldFast } from '../hooks/use-fast-field'

type CheckboxFieldProps<Schema extends object, Path extends string> = {
  name: Function.AutoPath<Schema, Path>
}

export default function CheckboxField<
  Schema extends object,
  Path extends string
>({ name }: CheckboxFieldProps<Schema, Path>) {
  const [{ value }] = useFieldFast<boolean | undefined>(name)

  return <></>
}

type Schema = {
  user: {
    approval: {
      isApproved: boolean
      adminApprovals: boolean[]
    }
  }
}

function Test() {
  return <CheckboxField<Schema> />
}

The problem is, this complains, since I haven't passed a Path generic to CheckboxField's second argument.

Screen Shot 2021-02-02 at 6 33 23 PM

The requirement to pass a second generic argument ahead of time here would kind of defeat the purpose. The problem is, I do indeed need to pass a generic so that the component knows the form type.

I tried making Path optional with Path extends string = string, but then I get no results for name.

Screen Shot 2021-02-02 at 6 34 46 PM

Any idea how I can get the name field to give me the dot path notation for the given schema?

@millsp
Copy link
Owner

millsp commented Feb 2, 2021

Yes, this is a terrible TS limitation. prisma/prisma#3372 (comment) Not sure how to make this work in React

@nandorojo
Copy link
Author

The crux of the issue here appears to be that it cannot infer because one of the generics is required. The only workaround I've found is to do this:

type CheckboxFieldProps<Schema, Path extends string> = {
  name: Function.AutoPath<Schema, Path>
  schema: Schema
}

type User = {
  approval: {
    isApproved: boolean
    adminApprovals: boolean[]
  }
}

// then in the component
<CheckboxField schema={null as User} name="approval.isApproved"  />

But this is really not ideal, since I'm moving beyond types and casting a fake prop. Is there any other option, you think?

@millsp
Copy link
Owner

millsp commented Feb 3, 2021

You are limited by TS here, this impacts all of us on a daily basis. Only workarounds possible.

@millsp
Copy link
Owner

millsp commented Feb 3, 2021

In your case, you'd like the previous implementation - but we know this is not possible because of its performance problems and limitations. I have tried to re-implement a bare minimum non-deferred version, but it fails on large objects. Only the new implementation is production ready. Sorry!

@millsp
Copy link
Owner

millsp commented Feb 3, 2021

import {
  Function,
  Object,
  String,
} from "ts-toolbelt";

declare function get<Obj extends object, Path extends string>(
  object: Obj, path: Function.AutoPath<Obj, Path>
): Object.Path<Obj, String.Split<Path, '.'>>

declare const user: User

type User = {
  name: string
  friends: User[]
}

// works
const friendName = get(user, 'friends.40.name')
const friendFriendName = get(user, 'friends.40.friends.12.name')

// errors
const friendNames = get(user, 'friends.40.names')
const friendFriendNames = get(user, 'friends.40.friends.12.names')

@jessetabak
Copy link

In your case, you'd like the previous implementation - but we know this is not possible because of its performance problems and limitations. I have tried to re-implement a bare minimum non-deferred version, but it fails on large objects. Only the new implementation is production ready. Sorry!

I'm actually using the previous PathsDot implementation (mainly for React props), it already works quite nice for me. Has you're tinkering made any improvement on it? If so, I'd love to see it and try to improve on it myself.

I tested your new AutoPath btw, it's rock solid, amazing.

@millsp
Copy link
Owner

millsp commented Feb 12, 2021

That was my conclusion, sorry. I could not make it reliable enough. While it may work for isolated/small examples, it is not fail-proof at all. There is no way to improve it - it WILL fail when you hit that object that is a bit too big. TypeScript needs to get more agressive optimizations - thus the deferred version is the only answer here. But you can use it at your own risk locally :)

We are bound to this limitation for now, to get it to work with react prisma/prisma#3372 (comment)

@mesqueeb
Copy link
Contributor

mesqueeb commented Feb 26, 2021

I'm trying to achieve something similar with dot nested paths like so:

<T extends Record<string, any>, K extends string> (obj: T, keys: K[]): O.Pick<T, K>
// support dot paths for `K` at the end
// tried:
<T extends Record<string, any>, K extends string> (obj: T, keys: K[]): O.Pick<T, F.AutoPath<T, K>>

But I'm not sure I correctly understand AutoPath

Could you point me in the correct direction? : )

This is what I want to achieve:

const doc = { a: { b: { yes: 0, no: 0 } } }
const res = pick(doc, ['a.b.yes'])

Currently the type of res without the AutoPath is:

res // {}

but I want it to be:

res // { a: { b: { yes: number } } }

@millsp
Copy link
Owner

millsp commented Feb 26, 2021

I think that you're looking for O.P.Pick, which you could combine with Split and AutoPath. We can just change the example above by this. Haven't tested it but since it's standard it should work right away. AutoPath is just a utility that will cause intellisense to show you the next fields as you type:

declare function pick<Obj extends object, Path extends string>(
  object: Obj, path: Function.AutoPath<Obj, Path>
): Object.P.Pick<Obj, String.Split<Path, '.'>>

@nandorojo
Copy link
Author

@millsp this new TS feature might be relevant for this library

microsoft/TypeScript#45711

image

@PKief
Copy link

PKief commented Feb 16, 2022

@millsp I am unfortunately not that familiar with ts-toolbelt so I am not sure if this is a bug. But the example for AutoPath shown in CodeSandbox (and also in my current project) does not show any errors for the two lines under "//errors":

type User = {
    name: string
    friends: User[]
}

// works
const friendName = get(user, 'friends.40.name')
const friendFriendName = get(user, 'friends.40.friends.12.name')

// errors
const friendNames = get(user, 'friends.40.names')
const friendFriendNames = get(user, 'friends.40.friends.12.names')

Link CodeSandbox: https://codesandbox.io/s/x4jly?file=/src/index.ts:3130-3379

Everything that I write after the index of the array is not typed anymore and does not produce any type errors. Is here something missing? I would be very happy if you could take a look at it :)

@nandorojo
Copy link
Author

FWIW I’ve also been using FieldPath from react hook form

@manojdcoder
Copy link

@millsp Facing same issue as @nandorojo, I just copied the same from docs and still no errors were thrown on invalid path.

Playground

@manojdcoder
Copy link

Looks like there is an open issue, version 9.3.0 works.

@nandorojo
Copy link
Author

react-hook-form FieldPath is good too

@100terres
Copy link

I've extracted the Types from react-hook-from. Here it is if anyone is looking for it.

/**
 * Checks whether T1 can be exactly (mutually) assigned to T2
 * @typeParam T1 - type to check
 * @typeParam T2 - type to check against
 * ```
 * IsEqual<string, string> = true
 * IsEqual<'foo', 'foo'> = true
 * IsEqual<string, number> = false
 * IsEqual<string, number> = false
 * IsEqual<string, 'foo'> = false
 * IsEqual<'foo', string> = false
 * IsEqual<'foo' | 'bar', 'foo'> = boolean // 'foo' is assignable, but 'bar' is not (true | false) -> boolean
 * ```
 */
export type IsEqual<T1, T2> = T1 extends T2
  ? (<G>() => G extends T1 ? 1 : 2) extends <G>() => G extends T2 ? 1 : 2
    ? true
    : false
  : false;

export type Primitive =
  | null
  | undefined
  | string
  | number
  | boolean
  | symbol
  | bigint;

export type BrowserNativeObject = Date | FileList | File;

/**
 * Type which given a tuple type returns its own keys, i.e. only its indices.
 * @typeParam T - tuple type
 * @example
 * ```
 * TupleKeys<[number, string]> = '0' | '1'
 * ```
 */
export type TupleKeys<T extends ReadonlyArray<any>> = Exclude<
  keyof T,
  keyof any[]
>;

/**
 * Type to query whether an array type T is a tuple type.
 * @typeParam T - type which may be an array or tuple
 * @example
 * ```
 * IsTuple<[number]> = true
 * IsTuple<number[]> = false
 * ```
 */
export type IsTuple<T extends ReadonlyArray<any>> = number extends T['length']
  ? false
  : true;

/**
 * Type which can be used to index an array or tuple type.
 */
export type ArrayKey = number;

/**
 * Helper function to break apart T1 and check if any are equal to T2
 *
 * See {@link IsEqual}
 */
type AnyIsEqual<T1, T2> = T1 extends T2
  ? IsEqual<T1, T2> extends true
    ? true
    : never
  : never;

/**
 * Helper type for recursively constructing paths through a type.
 * This actually constructs the strings and recurses into nested
 * object types.
 *
 * See {@link Path}
 */
type PathImpl<K extends string | number, V, TraversedTypes> = V extends
  | Primitive
  | BrowserNativeObject
  ? `${K}`
  : // Check so that we don't recurse into the same type
  // by ensuring that the types are mutually assignable
  // mutually required to avoid false positives of subtypes
  true extends AnyIsEqual<TraversedTypes, V>
  ? `${K}`
  : `${K}` | `${K}.${PathInternal<V, TraversedTypes | V>}`;

/**
 * Helper type for recursively constructing paths through a type.
 * This obscures the internal type param TraversedTypes from exported contract.
 *
 * See {@link Path}
 */
type PathInternal<T, TraversedTypes = T> = T extends ReadonlyArray<infer V>
  ? IsTuple<T> extends true
    ? {
        [K in TupleKeys<T>]-?: PathImpl<K & string, T[K], TraversedTypes>;
      }[TupleKeys<T>]
    : PathImpl<ArrayKey, V, TraversedTypes>
  : {
      [K in keyof T]-?: PathImpl<K & string, T[K], TraversedTypes>;
    }[keyof T];

/**
 * Type which eagerly collects all paths through a type
 * @typeParam T - type which should be introspected
 * @example
 * ```
 * Path<{foo: {bar: string}}> = 'foo' | 'foo.bar'
 * ```
 */
// We want to explode the union type and process each individually
// so assignable types don't leak onto the stack from the base.
export type Path<T> = T extends any ? PathInternal<T> : never;


type User = {
    name: string
    friends: {
        test: string;
    }[];
}

type Keys = Path<User>;

const test: Keys = "friends.3.test";

console.log(test);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Add new features
Projects
Board
  
Done
Development

No branches or pull requests

10 participants