Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for custom scalars #2

Open
Akryum opened this issue Jul 27, 2018 · 47 comments
Open

Support for custom scalars #2

Akryum opened this issue Jul 27, 2018 · 47 comments

Comments

@Akryum
Copy link

@Akryum Akryum commented Jul 27, 2018

Migrated from: apollographql/apollo-client#585

@fbartho
Copy link

@fbartho fbartho commented Jul 31, 2018

It'd be great if we had support for custom scalars in Apollo Client. A pretty crucial feature would be for custom scalars to be provided by native code. apollo-link-state users would particularly benefit from this.

The canonical example is Date objects (from ISOString or Unix Timestamp "raw" formats), but it would be valuable to vend other complex objects from a local object pool in certain circumstances beyond just dates.

In the original thread there was a reference that maybe a library-provided semi-static list of common scalars might be an acceptable compromise, but immediately, people would want momentjs dates instead of JS Dates.

@bebbi
Copy link

@bebbi bebbi commented Sep 14, 2018

The thumbs up total of this and OPs of issues linking here are 22+15+37=74.
Most need json parse and date formats.

@brunoreis
Copy link

@brunoreis brunoreis commented Sep 27, 2018

I'm thinking about creating a hoc where I can pass a query, the custom scalar type and a conversion function. I would compose that hoc adding it just after the query hoc so that it would convert data injected from that query.

Can someone point me how to inspect the schema of a specific query to detect what are the paths to the fields that are of that custom scalar type?

@jedwards1211
Copy link

@jedwards1211 jedwards1211 commented Nov 6, 2018

@bebbi why do people need to parse json types on the client side? Unlike dates, json can be serialized just fine by server-side custom scalars and will be received as json by the client.

@jedwards1211
Copy link

@jedwards1211 jedwards1211 commented Nov 6, 2018

@brunoreis you may be able to use graphql's undocumented visitWithTypeInfo, which allows you to traverse the queries while keeping track of the type of each field.

However, it may not be ideal for use on the client because you have to construct a TypeInfo object from an entire GraphQLSchema. I don't know if it would support a partial schema containing only the bare minimum fields necessary to determine what to parse on the client.

Another option is to fetch the type metadata from the server and build a custom index to it; I did this in apollo-magic-refetch. But I had to write my own code to keep track of the types while traversing the query with visit, and there are edge cases I haven't taken care of yet.

I bet there are some hooks in the Apollo caching layer that would allow you to handle custom scalars anywhere in the schema this way, without having to wrap a bunch of your components in an HoC.

@couturecraigj
Copy link

@couturecraigj couturecraigj commented Nov 10, 2018

@jedwards1211 wouldn't that defeat the purpose of a type system? Maybe somebody wants to store mixed data to graphql in certain instances. That is when they would use a JSON type. I know it is not ideal but geo-data for instance can be incrementally adopted through GeoJSON in JSON (mixed) data type that is just stored as a string. Not having to parse it each time the client receives it would be huge.

@jedwards1211
Copy link

@jedwards1211 jedwards1211 commented Nov 10, 2018

@couturecraigj I'm not sure what you're thinking would defeat the purpose if a type system, I was suggesting that there's probably a way to make the apollo cache parse all raw values of a date type (a custom scalar in the schema that serialize to an ISO string on the wire) back into a Date object and store them as such in the cache.

@slorber
Copy link

@slorber slorber commented Nov 14, 2018

Hi,

In addition to custom scalar types, I want to add that it is very important for me that the choosen solution works with codegen tools. If my Date scalar is serialized as String and deserialized or injected as props as JS Date on the client, it's important the codegen tools give me a JS Date type in the end and not a string. This seems complicated to solve, but important to mention.

@bebbi
Copy link

@bebbi bebbi commented Nov 14, 2018

@jedwards1211 In my case for example, I'd like not having to worry about de-serialization, because this seems to create multiple redundant places where I worry about the data model.

@jedwards1211
Copy link

@jedwards1211 jedwards1211 commented Nov 14, 2018

@bebbi I not sure I fully understand you.
Without worrying about de-serialization, any date field in your schema will be received by the client as a string or number (whichever you choose to serialize it as in the custom scalar type on the server).

The only way to convert back to a date on the client is to have de-serialization code.

Of course it would be terrible to have to write code for every single date field or every query that requests a date field -- which is why I'm suggesting that if the client has enough information about the schema to know which fields are dates, it would be possible to write a single piece of code that deserializes any date field encountered in the schema.

@bebbi
Copy link

@bebbi bebbi commented Nov 14, 2018

@jedwards1211 As far as I can understand we're on the same page.
It's json in my case. I've got various queries using fragments using that json type and would like de-serialization of that type to be controlled in one single place.

@jedwards1211
Copy link

@jedwards1211 jedwards1211 commented Nov 14, 2018

@bebbi so when your server loads the JSON it needs to send from the database or a file or whatever, is it a raw string instead of an object?

If you use graphql-type-json for the field and your resolver returns an object instead of a raw string, it will be sent to the client as an object and the client won't need to deserialize at all.

@gullitmiranda
Copy link

@gullitmiranda gullitmiranda commented Nov 14, 2018

@jedwards1211 the https://github.com/taion/graphql-type-json works only to send json to the server, not the opposite.
what we need is to the client parse a json send by the server.

@jedwards1211
Copy link

@jedwards1211 jedwards1211 commented Nov 14, 2018

@gullitmiranda I don't understand why that's been your experience, seems to me something must be misconfigured. Here is proof that it's able to send json to the client in my project (the jsonFieldMappings field below uses graphql-type-json). You can see that the client (GraphiQL) receives it as an object instead of a raw string:

image

@gullitmiranda
Copy link

@gullitmiranda gullitmiranda commented Nov 17, 2018

interesting @jedwards1211 . was not aware of this since I don't use the graphql-js on the server side. but from what I've been researching, this was not a standard that is accepted by the default implementation of the graphql, so many libs do not support or do not intend to support this.

some conversations:

@jedwards1211
Copy link

@jedwards1211 jedwards1211 commented Nov 17, 2018

even if it's not exactly standard, the GraphQL docs do say that

A standard GraphQL POST request should use the application/json content type, and include a JSON-encoded body

Regardless of the method by which the query and variables were sent, the response should be returned in the body of the request in JSON format

In most GraphQL service implementations, there is also a way to specify custom scalar types

So for built-in types, a standard GraphQL server is already serializing to and from JSON, and the most basic possible custom scalar system -- one that just hands you a JSON node representing the scalar in the request or accepts a JSON node from you to represent the variable in the response -- would only necessitate using the identity function for JSON custom scalar serialization/deserialization**.

Which is exactly what a commenter recently mentioned in the issue you linked:

scalar :raw_json do
  parse fn _, _ -> raise "not relevant" end
  serialize fn value -> value end
end

So in other words, any GraphQL server that supports custom scalars is capable of sending a field value that the client will receive as a JSON object instead of a string.

** The only caveat is that it also has to be possible to represent the value as a literal within the GraphQL query itself, not just within the JSON request/response body. But there is a clean mapping between JSON and GraphQL AST nodes, as demonstrated in graphql-type-json's parseLiteral function.

@jedwards1211
Copy link

@jedwards1211 jedwards1211 commented Nov 17, 2018

@slorber do you have enough control over the codegen tools you use to tell them what JS type is associated with a given GraphQL custom scalar type?

@gullitmiranda
Copy link

@gullitmiranda gullitmiranda commented Nov 18, 2018

ye @jedwards1211

Which is exactly what a commenter recently mentioned in the issue you linked:

i see and try to use in outputs and worked. don't work with input's for now, but it's no big deal.

@jedwards1211
Copy link

@jedwards1211 jedwards1211 commented Nov 18, 2018

Cool. I don't know elixir but it looks like inputs may work if you make the parse fn return the argument instead of raising an error.

@gullitmiranda
Copy link

@gullitmiranda gullitmiranda commented Nov 19, 2018

the parse really work, but the absinthe add errors because the typing.

  1) test create template mutation succeeds with valid params (MyAppWeb.Schema.MyContext.TemplateMutationTest)
     test/my_app_web/schema/my_context/mutations/template_mutation_test.exs:12
     match (=) failed
     code:  assert {:ok, data} = run(create_template_mutation(), variables: to_input(params), context: context)
     right: {:error,
             [
               %{
                 locations: [],
                 message: "Argument \"input\" has invalid value $input.\nIn field \"fields\": Expected type \"JSON\", found {key1: \"value\", key2: \"value2\", list1: [1, {keyInList: \"value_in_list\"}]}.\nIn field \"key1\": Unknown field.\nIn field \"key2\": Unknown field.\nIn field \"list1\": Unknown field."
               }
             ]}
     stacktrace:
       test/my_app_web/schema/my_context/mutations/template_mutation_test.exs:54: (test)
@dan-turner
Copy link

@dan-turner dan-turner commented Mar 27, 2019

Any movement on this?

@jamiewinder
Copy link

@jamiewinder jamiewinder commented Jul 4, 2019

I've noticed support for custom scalars is referenced in the GraphQL Tools docs, but as far as I can see it isn't in the pipeline for apollo-client (including 3.0, or at all). Could someone in the know confirm this?

@FredyC
Copy link

@FredyC FredyC commented Oct 4, 2019

Ok, I think I've found fairly good approach, at least to handle ISO dates. With that link, I can easily transform iso date strings to Date instances before they get to components. It is surely slightly fragile as we need to remember to add more fields when they are used. Eventually, I want to make some simple generator based on the schema to remove that barrier.

https://github.com/with-heart/apollo-link-response-resolver

    BusinessHourInterval: {
      openAt: parseISO,
      closeAt: parseISO,
    },

For the reverse direction when I am using Date for the mutation variables, it actually works out of the box and dates are converted. Not sure why such "magic" exists only in one direction, though.

As for the TypeScript, we are using graphql-code-generator and that's simple scalars config to set proper types and works very nicely.

@eturino
Copy link

@eturino eturino commented Nov 30, 2019

I have been working on a custom ApolloLink to deal with this situation.

https://github.com/eturino/apollo-link-scalars

If you pass it a GraphQLSchema and optionally a map of parsing/serializing functions per scalar type, it will parse all scalars on the responses, and serialize all the scalars on the inputs.

It can optionally also validate enums, throwing an error if an invalid enum value is received.

By default, for each scalar, it will apply the parsing/serializing functions defined for that type in the GraphQLSchema. The functions passed in the typesMap argument take precedence.

As a small disclaimer: this is a POC. I just finished testing it on my team's app but I haven't tested performance.

import { withScalars } from "apollo-link-scalars"
import { ApolloLink } from "apollo-link";
import { HttpLink } from "apollo-link-http";
import { schema } from './my-schema'

const link = ApolloLink.from([
  withScalars({ schema }),
  new HttpLink({ uri: "http://example.org/graphql" })
]);

// we can also pass a custom map of functions. These will have priority over the GraphQLTypes parsing and serializing functions from the Schema.
const typesMap = {
  CustomScalar: {
    serialize: (parsed: CustomScalar) => parsed.toString(),
    parseValue: (raw: string | number | null): CustomScalar | null => {
      return raw ? new CustomScalar(raw) : null
    }
  }
};

const link2 = ApolloLink.from([
  withScalars({ schema, typesMap }),
  new HttpLink({ uri: "http://example.org/graphql" })
]);

//cc @FredyC

eturino added a commit to eturino/apollo-link-scalars that referenced this issue Dec 1, 2019
eturino added a commit to eturino/apollo-link-scalars that referenced this issue Dec 9, 2019
…izing interaction

add links to the original apollo-client issue
apollographql/apollo-client#585  as well as the new issue
apollographql/apollo-feature-requests#2
@alfechner
Copy link

@alfechner alfechner commented Feb 12, 2020

There is FieldPolicy in Apollo Client 3.

It allows us to serialize/unserialize values when read/write from/to the cache.

It's on field rather than on type level. Thus one needs to configure the transformation for every field again. However it mitigates the problem of serializing the cache.

@geekox86
Copy link

@geekox86 geekox86 commented Mar 17, 2020

@Akryum Why are you guys not engaged on this high-demand feature?!

@Akryum
Copy link
Author

@Akryum Akryum commented Mar 17, 2020

Read the comment literally before yours before writing there is no progress 😅

@geekox86
Copy link

@geekox86 geekox86 commented Mar 17, 2020

@Akryum I have read it before, but the requested feature (custom scalars) is to have serialize/deserialize on scalar type level not field level.

I really appreciate everything that you are guys doing, but reading through years of going back and forth on this feature seems, at least to me, weird from your side, as your responsiveness on other issues has been and is usually excellent.

@geekox86
Copy link

@geekox86 geekox86 commented Mar 17, 2020

@Akryum and I also took my words back on no plan and no progress as this was only my exaggerations, and I apologize for it.

Though, any progress on having this feature per type?

@FredyC
Copy link

@FredyC FredyC commented Mar 17, 2020

I think it doesn't have a priority because there are viable solutions in userland, see #2 (comment)

@jedwards1211
Copy link

@jedwards1211 jedwards1211 commented Mar 17, 2020

Out of curiosity, why would one ever want to define serialization on a field level? If two fields are the same type, I'm having trouble imagining a use case for serializing them differently.

Er...nevermind, I just realized I have various JSON fields throughout my schema, and different postprocessing on them could theoretically be useful.

@jedwards1211
Copy link

@jedwards1211 jedwards1211 commented Mar 17, 2020

@FredyC FWIW apollo-link-response-resolver is now unmaintained, better to recommend apollo-link-scalars I guess

@jedwards1211
Copy link

@jedwards1211 jedwards1211 commented Apr 1, 2020

@slorber

In addition to custom scalar types, I want to add that it is very important for me that the choosen solution works with codegen tools. If my Date scalar is serialized as String and deserialized or injected as props as JS Date on the client, it's important the codegen tools give me a JS Date type in the end and not a string. This seems complicated to solve, but important to mention.

If codegen tools don't allow you to customize this, they need improvement. To deal with stuff like this I made my own GraphQL -> Flow type generator that allows comments in the schema and documents to customize the type generation very specifically: graphql-typegen

@nateq314
Copy link

@nateq314 nateq314 commented Apr 16, 2020

There is FieldPolicy in Apollo Client 3.

It allows us to serialize/unserialize values when read/write from/to the cache.

It's on field rather than on type level. Thus one needs to configure the transformation for every field again. However it mitigates the problem of serializing the cache.

For reference, the documentation for the FieldPolicy feature that @alfechner mentioned above is here: https://www.apollographql.com/docs/react/v3.0-beta/caching/cache-field-behavior/

@pluqueTheLuxe
Copy link

@pluqueTheLuxe pluqueTheLuxe commented Apr 28, 2020

Read the comment literally before yours before writing there is no progress 😅

@Akryum, I think @geekox86 meant that this feature request is in the apollo-feature-requests backlog since 2018 and in the original backlog since 2016 and stands as #1 feature when you sort by "thumbs up".

Moreover, I think it's not much to say that custom scalar (especially: Date and DateTime) are frustrating to work with in GQL server/client communication

@dora-gt
Copy link

@dora-gt dora-gt commented Jun 9, 2020

https://github.com/eturino/apollo-link-scalars
This is an amazing project. Thank you so much.

@dobrinov
Copy link

@dobrinov dobrinov commented Jul 27, 2020

There is FieldPolicy in Apollo Client 3.

It allows us to serialize/unserialize values when read/write from/to the cache.

It's on field rather than on type level. Thus one needs to configure the transformation for every field again. However it mitigates the problem of serializing the cache.

I tried this with Apollo 3 and it works just fine but it does not handle one use case which https://github.com/eturino/apollo-link-scalars covers. If you use no-cache as fetchPolicy, the read and merge methods of the fieldPolicy won't be executed.

@hwillson hwillson self-assigned this Sep 29, 2020
@coler-j
Copy link

@coler-j coler-j commented Nov 14, 2020

How do you handle typescript types with FieldPolicies though? (for example, this SO ticket)

If your custom scalar maps to string but in your FieldPolicy you convert it to a DateTime then Ts will still think it is a string....

@UchihaYuki
Copy link

@UchihaYuki UchihaYuki commented Nov 25, 2020

I think the work should be done on "@graphql-codegen/typescript-apollo-angular". It can generate code to run parsers and serializers on custom scalars (It already have enough info to do this, since it know the whole schema picture). It will be better if we can just reuse "graphql-scalars". The code on the client side shouldn't be affected much. just some ideas...

@dbbk
Copy link

@dbbk dbbk commented Dec 1, 2020

I have to say it baffles me as to why this is so hard with Apollo Client... the only solution I've been able to find is using https://github.com/eturino/apollo-link-scalars, but I'm quite nervous of bundling my whole schema file in.

My use case is a custom serialize format for Date (I want to format a native JS date to a YYYY-MM-DD string).

@myknbani
Copy link

@myknbani myknbani commented Dec 17, 2020

I'm not sure if it's just a low priority item, or language/reflection differences, but their Android client (Java/Kotlin) has this:

https://www.apollographql.com/docs/android/essentials/custom-scalar-types/

@jamesdh
Copy link

@jamesdh jamesdh commented Dec 21, 2020

@myknbani likewise, the GraphQL-Java server implementation makes defining custom scalars and data mappings very easy to do.

@stephenh
Copy link

@stephenh stephenh commented Dec 22, 2020

Fwiw if you want to use FetchPolicys to do this instead of the link approach (that requires having the schema at runtime), we have a graphql-code-generator plugin that will generate the "for every entity, for every field that is a custom scalar, use this merge policy" object literal to pass to apollo-client / InMemoryCache:

https://github.com/homebound-team/graphql-typescript-scalar-type-policies

@mikepricedev
Copy link

@mikepricedev mikepricedev commented Feb 28, 2021

One might consider custom scalers required to address objects that are not the sum of their parts i.e. an object that only conveys information as a whole.

For example a rational number. The classic signature for a rational number being:

{
  s:-1|1; 
  n:uint; 
  d:uint; 
}

If any of these parts are missing, the object ceases to be a rational number. A first name or a last name still has meaning independent of the full name, but a numerator without the denominator has lost all information.

Expressing the parts of a rational number in a Schema definition does not quite make sense. In the same way we don't define a String as every index of the char(s) that make up the String. We simply call it a scalar String.

@hwillson hwillson removed their assignment Mar 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet