Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Map type #101

Open
miracle2k opened this issue Oct 3, 2015 · 76 comments

Comments

Projects
None yet
@miracle2k
Copy link

commented Oct 3, 2015

Should there be a map type?

I have a structure that looks like this:

  Item {
     title {
        de_DE,
        fr,
        pa,
        .....
     }
  }

The point here is that the number of language strings might be arbitrary. I find this hard to model using GraphQLObject, because:

  1. The number of locales is undefined. While the resolver could handle this dynamically, I think it's a problem in Relay that the schema cannot reflect these dynamic fields.
  2. I would like to query all the existing locales, which isn't allowed, it seems ("must have a sub-selection").

If I make my GraphQL server return a JSON object for "title", the Relay client doesn't complain (although maybe shouldComponentUpdate breaks), but I think I'm skirting by here. At the very least, I think I couldn't generate a schema that confirms to the spec.

@OlegIlyenko

This comment has been minimized.

Copy link

commented Oct 3, 2015

I think this is a valid concern as well. I currently thinking of ways to add GraphQL endpoint for our API. We are building project-based multi-tenant service. For example every project has a list of products which have name and description. Just like in your case these are localized strings as well.

One way one can approach this problem is to parametrize the field:

query ProductNames {
  products {
    nameEn: name(locale: "en")
    nameDe: name(locale: "de_DE")
  }
}

Another approach, as you mentioned, would be to generate a schema of-the-fly. In our case it would be possible, since every project has a limited set of locales which are defined in the config of this project.

Where it becomes more tricky is an addition user-defined data types. Users of our API can define new attributes for products (visually through the merchant-center application). These attributes are also typed, so it's possible to generate GraphQL schema for this project, but it has some implications:

  • Generating schema on every request comes with big performance penalty, since we need to load config from a DB in order to do so
  • to compensate for this performance penalty we need to implement some schema caching and cache invalidation logic
  • Schema now becomes tenant-specific. This means that it becomes much harder to integrate with generic tools like GraphiQL and do introspection in general
  • You can't even do an introspection queries without the auth anymore.

I guess one can just put all custom attribute JSON in a string scalar, but I don't think that other developers will appreciate JSON inside of string inside of another JSON :) I feel that generic JSON/Map-like type can provide a very useful middle-ground for these use-cases. It can also help a lot with a migration to the GraphQL. One can quickly start with Map-like structure and after some time develop schema generation and caching mechanisms.

@Sandreu

This comment has been minimized.

Copy link

commented Oct 17, 2015

Hello,

I agree about this, and as @OlegIlyenko said, JSON string inside JSON string seems awkward.
I think Map is one portion of what JSON can provide and I have exposed my point of view in graphql/graphql-js#172
So 👍 to include specs for a "JSON or RawObject or however you want to call it" type.

@xpepermint

This comment has been minimized.

Copy link
Contributor

commented Oct 18, 2015

+1

@clintwood

This comment has been minimized.

Copy link

commented Oct 20, 2015

Hi,

I have a use case where I have 'server based schema' and 'client based schema'. The server based schema is pretty much static and will follow normal project/application changes through time. However the client based schema is specific to client side and generated on the fly for exclusive use by the client/user. It's shape is not ideal for a generic map type as it can become deeply hierarchical.

What I need to do is store the resulting client side JSON blob against the user on the server side. I specifically do not want any validation or type checking done on this JSON blob server side except for checking for valid JSON. At the moment I'm storing it as stringified JSON against an attribute in server side schema which does not seem ideal.

So I'm very much in favor of JSONObject/RawObject/UncheckedObject or whatever as simple JSON object as proposed here: graphql/graphql-js#172.

@jvliwanag

This comment has been minimized.

Copy link

commented Oct 23, 2015

It looks like the use case of @miracle2k can be solved by just using a list. So something like:

  Item {
     titles {
        language
        text
     }
  }

wherein titles is an array. One can always create an input argument if you want to select a subset.

@clintwood 's use case however looks different since there's no known schema and may be hierarchical.

IMO, as long as there's a known structure, a list type could replace a map.

@sonewman

This comment has been minimized.

Copy link

commented Nov 9, 2015

@jvliwanag correct me if I am wrong, but this would mean that the values in the list would have to return as an array. This is OK if someone is developing an API from scratch and has control over defining the response payload.

But if an existing API is being moved over to use GraphQL, which already has a defined contract returning a map of key value pairs (even if the values are always of a defined object type / structure), then it appears this is unavoidable.

I am interested to know how it would be possible to use a list, unfortunately the the list type does not seem to accept anything except an array:
graphql/graphql-js/blob/master/src/execution/execute.js#L679-L683.

Interestingly the error:

 User Error: expected iterable, but did not find one.

Suggests it could be possible to supply a Map or Set, but of course the code says otherwise 😿

@D1plo1d

This comment has been minimized.

Copy link

commented Nov 9, 2015

+1. Maps would be useful in mutations for sending arbitrary key/value pairs.

@mincedmit

This comment has been minimized.

Copy link

commented Dec 11, 2015

+1 as well, imagine this can also allow for embedded documents in query responses if using a record store on the backend?

@leebyron

This comment has been minimized.

Copy link
Collaborator

commented Jan 9, 2016

Support in the JS library itself for arbitrary iterables rather than just Arrays is coming soon.

There are significant tradeoffs to a Map type vs a list of key/value pairs. One issue is paginating over the collection. Lists of values can have clear pagination rules while Maps which often have non-ordered key-value pairs are much more difficult to paginate.

Another issue is usage. Most often Map is used within APIs where one field of the value is being indexed, which is in my opinion is an API anti-pattern as indexing is an issue of storage and an issue of client caching but not an issue of transport. This anti-pattern concerns me. While there are some good uses for Maps in APIs, I fear that the common usage will be for these anti-patterns so I'm suggesting proceeding with caution.

For the original post, I think there are two good ways to solve this API puzzle:

First is what @OlegIlyenko suggests. That you explicitly ask for the languages which you care about fetching. If you need multiple you can use aliases to query for multiple. This may be the right path if you know up front which you want to query.

item {
  title: title(lang: $custom_lang)
  title_en: title(lang: "en")
  title_fr: title(lang: "fr")
}

Second is returning a list of tuples. This may be the right path if you don't know up front which you want, or if you specifically want them all.

item {
  titles {
    language
    text
  }
}

such that you might get the result:

"item": {
  "titles": [
    { "language": "en", text: "Hello" },
    { "language": "fr", text: "Bonjour" }
  ]
}
@akuckartz

This comment has been minimized.

Copy link

commented Jan 9, 2016

Please try to use JSON-LD language maps: http://www.w3.org/TR/json-ld/#index-maps

@dylanahsmith

This comment has been minimized.

Copy link
Contributor

commented Mar 11, 2016

I agree with @leebyron about the solution to the original problem.

In the case of user-defined JSON data, can we just make it clear that custom scalars aren't restricted to how they can be serialized. E.g. with graphql-js you can serialize a custom JSON scalar using arrays, objects, etc.

@jackielii

This comment has been minimized.

Copy link

commented Mar 21, 2016

alternatively:

item {
  titles(languages: ["en", "fr"])
}

result:

"item": {
  "titles": ["Hello", "Bonjour"]
}

You can specify the list of language in a variable

Advantage of this approach:

  1. static query: no need to specify languages before writing the fragment (versus approach 1 in @leebyron 's comment
  2. didn't create a new Object type. (versus approach 2 in @leebyron 's comment)
@Nebulai

This comment has been minimized.

Copy link

commented May 23, 2016

+1 for map support. This "anti-pattern" logic seems like over-thinking it to me. Sure some people will abuse it but that is true for anything..

@LucasIcarus

This comment has been minimized.

Copy link

commented Jun 28, 2016

+1, map support is useful for some situations that the system always return less data than expect, I think.

@juancabrera

This comment has been minimized.

Copy link

commented Aug 5, 2016

+1 Currently I'm dealing with an API that data is arbitrary (as users can create their own contentTypes on the CMS), so there is no way I can create an entryType as I have no idea what fields are in there. Having a mapType will be super useful for these cases.

Update: Just tried this and is working great so far!: https://github.com/taion/graphql-type-json (thank you @taion).

@jarwol

This comment has been minimized.

Copy link

commented Oct 13, 2016

I have the same use case as @juancabrera. Our API delivers content defined and managed through a custom CMS, and a main feature is being flexible and dynamic. In the application, we model the objects as a base set of common fields, and then a "fields" map with all of the dynamic properties that we don't know about on the server.

{
  "contentId": 123456,
  "type": "video",
  "fields": {
    "title": "fake video 1",
    "releaseDate": "...",
    ...
  }
}

We'd like to give our client app devs the ability to query and transform the content in the way they want, but without having to create a strict schema on the server.

@th-ko

This comment has been minimized.

Copy link

commented Oct 21, 2016

+1 for maps. In my use-case I have objects of this shape (in typescript):

interface GenericObject { 
  id: string, 
  label: string,
  types: string[], 
  objectProps: {[key: string]: GenericObject[]}
  dataProps: {[key: string]}: string[]} 
}

Using the list of key-values would mean I am doing a transformation on the server to send the data and then do the reverse transformation on the client to build up that map again.
Using the JSON value type would mean I can not fetch related objs (via objectProps) with their respective fields.
Using arguments would mean I need to know all the properties in advance, which is not possible.

Also the size of data we talk about here is rather small. So paging is not an issue.

@rncry

This comment has been minimized.

Copy link

commented Oct 28, 2016

We have the exact same need as @jarwol

@aweiker

This comment has been minimized.

Copy link

commented Oct 28, 2016

I am in agreement with @leebyron after watching what has happened in other "JSON" type apis at my company. The path of least resistance is the path most often traveled.

This means that if there is a choice of creating a new type that has more semantic meaning to the UI or creating a map that has no tooling support or contract, but is "quick" to create on the server, then a map is going to be used.

@GroofyIT

This comment has been minimized.

Copy link

commented Nov 8, 2016

I would personally opt for 2 seperated types.
As I see it there are 2 use cases of the data:

  1. view, the data is shown to a user in his/her 's preferred language
  2. admin, the data is to be edited (translations added / changed / removed) by an administrative user

This might come across as perhaps anti-pattern, though in my opinion it is not. Since these are 2 completely separated concerns / representations even if the source for both is the same in your db.

Thus:

  1. VIEW: Type is the minimal set, language is given as a variable and resolver is responsible of choosing the correct one to return.
    Changes are high that you want localised data for more types then just "Item", therefore using a variable is also reusable across your schema queries, mutations, ...
type Item {
   title: String
 }

Selected by:

query ($language: String!) {
    Item(language:$language) {
        title
    }
    variables {
        language:'en-US'
    }
}
  1. ADMIN: Type is extensive administrative set. Here you can choose your own preferred medicine:
    Array based
type Language {
    code:String
    text:String
}
type Item {
    title: [Language]
}

Unified languages type
(listing all possible?)

type Languages {
    nl-BE:String
    en-US:String
    fr-FR:String
    nl-FR:String
}

type Item {
    title:Languages
}

Plain object

scalar Object

type Item {
    title: Object
}
@limscoder

This comment has been minimized.

Copy link

commented Feb 10, 2017

We have the same issue as @OlegIlyenko: user defined data types.
Would be interested to hear how other users are tackling this.

@amannn

This comment has been minimized.

Copy link

commented Apr 24, 2017

I have a similar use case. Instead of the value of the Map being a simple string, I'd need GraphQL types as I'd like to use field resolvers there.

Consider a schema like this:

type Book {
  id: ID
  name: String
}

type User {
  name: String

  # This is invalid, but roughly what I'd need
  favouriteBooks: Map<String, Book>
}

A response should look like this:

{
  name: "Jane Doe",
  favouriteBooks: {
    science: {
      id: '1'
    },
    novel: {
      id: '2'
    },
    ...
  }
}

The book category names are dynamic and change often, therefore I'd like to not specify them in the GraphQL response. Note that this is just a contrived example. I'm not looking for a solution to this particular problem.

I understand the value of using a list instead, but it would be great to use this to work with clients / server responses that were designed before the GraphQL layer was in place.

@serle

This comment has been minimized.

Copy link

commented Apr 30, 2017

I would like to support ES6 Map construction directly from json. The ES6 Map constructor accepts and array of Entries where the Entry "type" is a two element array where the elements are of different underlying types one for the key and one for the value e.g. [string object]. I can't do this currently in GraphQL.

@wincent

This comment has been minimized.

Copy link
Contributor

commented May 4, 2017

@amannn: Even "with clients / server responses that were designed before the GraphQL layer was in place", isn't a schema like the following similarly easy to produce/consume?

type Book {
  id: ID!
  name: String
}

type User {
  name: String
  favouriteBooks: [FavoriteBook]
}

type FavoriteBook {
  category: String
  book: Book
}

The obvious use case for having favouriteBooks be a map would be to do O(1) look-up by category name, but if that's what I care about I can create a look-up structure easily enough.

const booksByCategory = {};
books.forEach(({category, book}) => booksByCategory[category] = book);
@gintautassulskus

This comment has been minimized.

Copy link

commented May 10, 2018

@IvanGoncharov thanks, these are very good points. The presentation mentions production testing. The proposal could at least validate the acceptance of the concept. The implementation and testing would follow if need be.

@alkismavridis That looks like a neat solution. The only quirk I see is that [Int:User]requires a hardcoded id field. Maybe a meta field __key would do.

Gotusso, I see your point. You think standardising the approach to avoid custom solutions is a bit of a stretch?

@Gotusso

This comment has been minimized.

Copy link

commented May 10, 2018

No, I think it would be fair enough to have a separate spec and a reference implementation in the spirit of Relay's pagination. That can cover most of the cases, and the community could use it or not depending on their particular needs.

@alkismavridis

This comment has been minimized.

Copy link

commented May 10, 2018

@DylanVann

This comment has been minimized.

Copy link

commented Jun 25, 2018

In Apollo the GraphQL API is cached and can be used sort of like a redux store. Ideally you'd want it to be normalized, but that doesn't seem possible currently.

In one case if I'm using the cache I have to loop over a bunch of items instead of just checking an ID. It's not insurmountable to work around this but it definitely doesn't seem ideal.

@rybon

This comment has been minimized.

Copy link

commented Sep 17, 2018

I feel a Map type, with certain limitations, is the one true missing thing from GraphQL. The most obvious use case would be normalisation. I understand one could use a client-side library like normalizr to solve this problem, but why waste CPU cycles on lots of (mobile) devices when one could solve this server-side? Returning a Map of records indexed by their ID instead of a List wouldn't impact type safety as far as I know. It would also be useful for returning other index structures, such as inverted indices.

I understand it would not be possible to support arbitrary keys, as that would break the ability to verify queries ahead of time / at compile time with the static schema. I think the use case the OP is referring to with arbitrary keys coming for example from a CMS would only be possible if one partly moves the static features of GraphQL to runtime to make them dynamic. Dynamic schemas and queries seem to be beyond the scope of GraphQL though, for understandable reasons. It would probably complicate the nature and implementation of the protocol quite significantly.

@robert-zaremba

This comment has been minimized.

Copy link

commented Sep 17, 2018

As other mentioned - typed maps would solve all the aspects of loosing "type safety".
In our project we also need to make a workaround because of missing a map type.

@leebyron

This comment has been minimized.

Copy link
Collaborator

commented Oct 2, 2018

This issue has been open for a very long time. While I'm still definitely leaning towards Reject for this proposal based on all the concerns above, I'm leaving this as Strawman and Needs Champion in recognition that this issue is not a real proposal and there are only soft suggestions as comments.

If anyone is interested in carrying forward a Map type proposal, they should open a pull request with specific implementation recommendation.

There is a ton of incidental complexity in all of the suggestions - an RFC proposal must account for all of this complexity.

@alkismavridis

This comment has been minimized.

Copy link

commented Oct 2, 2018

Lee Byron, I would like to create a concrete proposal to push this forward. But it is not clear to me what exactly pull request means here.
Are you refering to the js library, or it could be in any other? I am more familiar with graphql-java.
Would a java implementation example be enough for the standard to be expanded?

@leebyron

This comment has been minimized.

Copy link
Collaborator

commented Oct 2, 2018

@alkismavridis I recommend reading https://github.com/facebook/graphql/blob/master/CONTRIBUTING.md which explains what's expected of an RFC proposal.

Any GraphQL library can be a useful testing ground for implementing ideas, however to be accepted a change to GraphQL.js (the reference implementation) is required.

@m4nuC

This comment has been minimized.

Copy link

commented Oct 13, 2018

Hypothetical use case where a Map type could come in handy:

Imagine dealing with a Humans type that has a Genome property which holds a bunch of genes. Each gene has a name and value. The entire possibilities for that Genome are known but can greatly vary from one Human to the other. So it would result in a very tedious and rather hard to maintain union type. There is no interest in running queries for particular genes, but we might want to retrieve the entire genome when getting a Human.

What would be the idiomatic GraphQL way to go about this?

@taion

This comment has been minimized.

Copy link

commented Oct 15, 2018

You'd do something like:

type Human {
  genome: [Gene]
}

type Gene {
  name: String
  value: String
}

i.e. otherwise follow the "list of entries" pattern as above.

@jlouis

This comment has been minimized.

Copy link

commented Oct 15, 2018

To add to @taion's writings:

In Protocol Buffers version 3, there is a handy support for a Map type. This type has an internal representation which follows the above. That is Map<K, V> is represented as [MapEntry] where type MapEntry { key: K, value: V }. This means:

  • It is backwards compatible. Older clients use that format.
  • It is not excessively larger on the wire.

It somewhat suggests that Map types is an artifact of the Client more than it is an artifact of the GraphQL "wire"-representation. Like the Relay NG specification adds special handling of objects with a "Connection" suffix, one could determine a special set of rules for a "Map" suffix, based on the above scheme.

@iNahooNya

This comment has been minimized.

Copy link

commented Oct 25, 2018

+1 for map support.
One side I wants to enlarge our graphQL service cover, in other side our client wants a more effective structure like map. They got map, they will transfer it to old modules easily.

@longzero1888

This comment has been minimized.

Copy link

commented Nov 15, 2018

+1 my team really really really need this!!!

@villesau

This comment has been minimized.

Copy link

commented Mar 20, 2019

If someone is worried about abusing Map type, wouldn't it make much more sense to write a linter for GraphQL which allows you to limit the functionality instead of restricting it by design?

@EgidioCaprino

This comment has been minimized.

Copy link

commented Mar 25, 2019

It would be great if you could use Flow style like:

type Foo {
  [String!]: Boolean
}
@alkismavridis

This comment has been minimized.

Copy link

commented Apr 13, 2019

I think the most straight-forward syntax to define the map would be something like that:

type Bar {
  myField: Foo
  myList: [Foo]
  myMap: [String : Foo]
}
@MFaisalHyder

This comment has been minimized.

Copy link

commented Apr 16, 2019

type User {
name: String
address: String
property: map[ key: String/[String] value: String/[complexType] ]
}
If we declare it this way it will be good as well.

@keynan

This comment has been minimized.

Copy link

commented Jul 10, 2019

+1

// schema {
property: map[ key: String, value: ComplexType ]
}

// query {
  property: map[
    key,
    value: {
      id
    }
  ]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.