Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat]: add ability to pass variables to StaticQuery #10482

Open
KyleAMathews opened this issue Dec 15, 2018 · 22 comments

Comments

@KyleAMathews
Copy link
Contributor

commented Dec 15, 2018

In order to run queries for Gatsby, we have to know at build-time what the query is and any arguments.

We can do this with page queries as pages can be created with "context" which can be used as arguments for the GraphQL query.

I didn't allow arguments with <StaticQuery> because there wasn't a clear way to allow people to specify build-time arguments that wouldn't confuse people into thinking that arguments would also work at runtime (explaining the distinction between build vs. run time is very hard IME).

There also wasn't need for arguments either as the query is only run once — arguments are of course for when you want to run something multiple times but vary in specified ways how the query runs.

But a recent tweet inspired a thought for how arguments could work: https://twitter.com/wesbos/status/1073338618897448960

The tweet's example was about how to create an image component that queries for image thumbnails for gatsby-image using <StaticQuery>. The idea is that you'd use it by passing a prop in for the image file you want to use e.g. <MyImage src="kyle-mathews.jpg" />.

my-image.jsx might look like:

import React from "react"
import Img from "gatsby-image"
import { StaticQuery } from "gatsby"

export default ({ src }) => {
  <StaticQuery
    query={graphql`
      query HeadingQuery($src: String!) {
         File(relativePath: { eq: $src }) {
           childImageSharp { ...etc }
         }
    `}
    render={data => (
       <Img fixed={data.path.to.query} />
    )}
  />
}

Now this by itself wouldn't work. At build time, we'd have no idea about how or where the component is used and what the arguments are and no way to know when and where to load in the results of the queries (how ever many there'd need to be).

But what could work is that we'd detect that a <StaticQuery> is taking arguments and then scan for every location that component is used. Then we'd rewrite each usage of <MyImage> to import a copy of my-image.jsx that has the arguments inserted e.g. <MyImage src="kyle-mathews.jpg" /> gets turned into <MyImage2344324 src="kyle-mathews.jpg" />

my-image-2344324.jsx looks like:

import React from "react"
import Img from "gatsby-image"
import { StaticQuery } from "gatsby"

export default ({ src }) => {
  <StaticQuery
    query={graphql`
      query HeadingQuery {
         File(relativePath: { eq: "kyle-mathews.jpg" }) {
           childImageSharp { ...etc }
         }
    `}
    render={data => (
       <Img fixed={data.path.to.query} />
    )}
  />
}

The problems with this are we'd still have to enforce that only static values would be supported as args (strings/numbers) and any run-time data would be ignored. Perhaps we could lint for this.

Dynamic runtime lookup of query results might be supportable by creating a lookup table and then running queries for all possible arguments but this obviously could lead to a crap ton of extra queries being run during builds.

We could use this to ship a nice wrapper to gatsby-image e.g. <FixedImage> and <FluidImage> that would work like: <FixedImage src="kyle-mathews.jpg" width={200} height={300} />

@alexluong

This comment has been minimized.

Copy link
Contributor

commented Dec 15, 2018

The problems with this are we'd still have to enforce that only static values would be supported as args (strings/numbers) and any run-time data would be ignored

This mean that we cannot pass user input values as arguments for these types of components, right? I feel like it's going to be super confusing for users to be able to use variables but only static values are supported.

Is there any way we can emphasize that "although you can pass a variable, it needs to be predefined"? I'm thinking something along the line of a Gatsby plugin that accept an array of strings. When passing a variable, it needs to be an element in the array.

@ChristopherBiscardi

This comment has been minimized.

Copy link
Member

commented Dec 15, 2018

This is a similar approach we take to generating wrappers in gatsby-mdx and I think it could be a fantastic approach not just for MDX content, but also potentially as a primitive for theming when the theme's image handling can be shadowed by the user without needing to write additional image queries themselves. Imagine a theme providing using an Image component that renders the result using a file in src/components/blog-post-header-image.js. The rendering logic for the image can be shadowed without the user having to worry about modifying the graphql query from whence it came.

As @alexluong mentions, this could be confusing if you're expecting dynamic values. We could also do something similar to webpack's require.context and if you pass in a glob, we find all of the images that match, get them processed, and you can choose between them at runtime. At which point it's sort of "your responsibility" as the user to make sure you only access images that exist (or fall back if they don't. require.context is reasonable inspiration for providing a json object of potential image values to choose from). In any case, that's "part two" or "sometime later" functionality IMO and we should focus on enabling this for static values to start.

@jgierer12

This comment has been minimized.

Copy link
Member

commented Dec 15, 2018

I like this, with one suggestion: Let users explicitly pass the variables as an object like so:

<StaticQuery query={graphql`
  query HeadingQuery($src: String!) {
    File(relativePath: { eq: $src }) {
      childImageSharp { ...etc }
    }
  `
  }
  variables={{ src }}
  render={data => (
    <Img fixed={data.path.to.query} />
  )}
/>

That way you could remap variable names or use nested objects.

I first thought that the validation could be made as an eslint plugin, but that wouldn't work as we also need to check that the consumer of the component doesn't change the variable. I believe that would only be possible with something like TypeScript.

Another approach (that would also satisfy my first suggestion) is to require a certain name for the query variable prop, eg:

export default ({ queryVariables: { src } }) => {
  <StaticQuery query={graphql`
    query HeadingQuery($src: String!) {
      File(relativePath: { eq: $src }) {
        childImageSharp { ...etc } }
    `}
    render={data => (
      <Img fixed={data.path.to.query} />
    )}
  />
}

That would be easily checkable for an eslint plugin (simply require that everytime a queryVariables prop is passed, it has to be immutable).

@vermario

This comment has been minimized.

Copy link
Contributor

commented Jan 28, 2019

Hello!
Just wanted to point out that this issue really would add a lot of value and would make Gatsby much more useful when (like in our current project) you are trying to give content editors the possibility of creating pages containing custom elements (For example: show here last 5 articles for a particular category etc).

I hope this gives you some feedback in terms of prioritising this feature in your internal development.

@karolyipeter

This comment has been minimized.

Copy link

commented Feb 4, 2019

I ran into this trying to create a very similar wrapper as the one @KyleAMathews uses in the demo, and would be very happy with this as well.

@nduc

This comment has been minimized.

Copy link

commented Feb 9, 2019

My use case is when I use gatsby-background-image and I need to load different file for mobile vs desktop. It would be nice to be able to pass in a file name to the static query, as @KyleAMathews mentioned above.

@gatsbot

This comment has been minimized.

Copy link

commented Mar 2, 2019

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

Thanks for being a part of the Gatsby community! 💪💜

@johno johno added not stale and removed stale? labels Mar 2, 2019

@lxibarra

This comment has been minimized.

Copy link

commented Mar 5, 2019

Plus one here im trying to optimize image loading bu at the same time dont want to write graphql queries for each image.

@guyinpv

This comment has been minimized.

Copy link

commented Mar 17, 2019

I have this use-case as well, for dynamically passing in an image name to the component.

I want to use StaticQuery for the obvious benefit of the image sharp stuff, src for different screens etc.

It seems to me that in a certain sense, the variables are already immutable, since those variables would start life as a prop on the component reference somewhere.

So Gatsby needs to be able to quickly connect the StaticQuery variables to the component props, and then find each reference where the props are set.

This definitely becomes difficult if the props are converted to other variable names before used within the query. So the first idea would be to force a direct correlation between the component props, and the variables being used in the query. This, at least, erases some chicken chasing with variable definitions and so on. I don't know how it could be done, given that a prop value might need some kind of mutation before use in the query, and mutation might necessitate using another variable name.
And then again, maybe somehow in code they have to generate a variable not even based on props at all! Like a calculation based on timestamp or random generation or something.

Anyway, if you can lock in the correlation of query variables to props, you then have the issue of finding everywhere the component is referenced. Or specifically everywhere it is referenced with the given props in use, cause that's all that matters really.

All I can suggest here is that Gatsby keep a kind of lookup table/cache that actually tracks all the components and where they are referenced. This data might be heavy to process, I don't know, but that data would also be useful for debugging, as it could then be used to generate an overview of your site with simple stats like "X component is used Y times across Z pages" and be able to list each page and component reference. That is handy data even if a little heavy to process!

Finally, Gatsby would have no choice, because there simply is no other way to do it, to render additional copies of the StaticQuery with each alternate variable used. This is a given, I don't see any other way to get through this part, it simply has to run through the query multiple times with each different variable set used. So temp copies of the query would have to be generated with each variable replacement.

All that said, I understand the normal way to do this would be query all the potential images from the page itself, and pass those into the component where used. The only reason I don't want to do this is that I'm trying to make the component more self-contained and not mess the page component with tons of queries for everything. If the component has everything it needs within itself, it's so much cleaner.

Anyway I hope you get it solved some day!

@mxmkhv

This comment has been minimized.

Copy link

commented Mar 26, 2019

UPDATE: Another day into this mess I got a way more elegant solution to share.

First off, putting the createPages api into onPreExtractQueries is super fragile and does not really work. Do not try it.

The goal:

  1. fetch data from a headless CMS (custom build of my company, returns a plain JSON)
  2. use the data to fetch the images (included as URLs in the JSON)
  3. create pages dynamically, each page to display it's relevant image with the gatsby-image plugin

The problems:

  1. dynamically creating nodes from image URLs
  2. creating dynamic queries for each page OR pass image object via pageContext

The solution: to problem 1.

Create the image nodes with the createRemoteFileNode helper function, which I found out about here. Not sure if that is a proper use of it, but it works like a charm. Images are now available for querying.

exports.sourceNodes = async ({ actions, createNodeId, node, store, cache }) => {
  const { createNode } = actions;
  
  const fetchData = () => axios.get(myDataURL)
  const res = await fetchData();
  
  const myData = res.data;  //the JSON data from the CMS response

  const myDataNode = {  
    id: myData.id,
    parent: null,
    internal: {
      type: myData,
      contentDigest: crypto.createHash(`md5`).update(JSON.stringify(myData.id)).digest(`hex`),
    },
    children: [], 
    data: myData,   //insert my data
  }

  createNode(myDataNode); // create the node

  const allPosts = myData.posts //get all posts data

  const createImageNodes = async (allPosts ) => {     //create the nodes with an async function
    for (const post of allPosts) {
      await createRemoteFileNode({    //AWAIT creation of nodes!!
        url: post.image,
        store,
        cache,
        createNode,
        createNodeId: id => post.id, //id wanted a function, so I provided one :) 
        name: post.id,   // IMPORTANT - name each image as the post ID to match them later
      })  
  }

  await createImageNodes(allPosts)    //AWAIT the download of the images!!! 
  return
}

IMPORTANT: If I do not await the creation of nodes, the bootstrapping process does not wait for the images to be downloaded, so nodes are not available in createPages query!

In addition:
For the gatsby-image plugin to work, the gatsby-source-filesystem plugin has to have access to the /images folder (although my images are fetched and not stored locally), so the following config was required in gatsby-config.js

    `gatsby-transformer-sharp`,
    `gatsby-plugin-sharp`,
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        name: `images`,
        path: `${__dirname}/src/images`, //this is required for the gatsby-image plugin to work
      },
    },

So, now I have the images in my graphQL schema, but in order to render an image in my dynamically created pages, I have to query for all images and filter out the one I need, similar to what was discussed in the Wes Bos twitter post.

The solution works, but it's not really scalable, so the one true solution for me was to pass the "fixed" or the "fluid" object (returned by a graphQL query for an image), as context to the dynamically created pages, using the createPages API.

The solution: to problem 2.

Awaiting creation of nodes in sourceNodes, makes createPages trigger only after the images are fetched and nodes are created. Otherwise the image nodes are not available at build time, because they are not yet downloaded.

  exports.createPages = ({ graphql, actions }) => {
    const { createPage } = actions
    
    //make one query to fetch myData and one to fetch all images 
    return graphql(`
      {
        myData {
          ...allData
        },
        allImageSharp {  //access all images
          edges {
            node {
              id
              fluid {  // fragments did not work here, I don't know why
                ...GatsbyImageSharpFluid
              }
            }
          }
        }
      }
    `
  ).then(result => {
      const allPosts = result.myData.posts //get all posts
      const imgData = result.data.allImageSharp.edges //the array of all image nodes
      
      const fluidImageObjects = [] //create an empty array

      imgData.forEach((image) => { //push all the fluid objects to the empty array
        fluidImageObjects.push(image.node.fluid)
      })
      
      allPosts.forEach((post) => {  //iterate over each post object and create a page

        //match the relevant fluid image object, this is for jpg only, but you can slice('/').pop() the url and get the image name also
        const imageFluid= fluidImageObjects.find(image => image.originalName === `${post.id}.jpg`)
        
       createPage({
          path: `/pages/${post.title}`,
          component: path.resolve(`./src/templates/PostTemplate.js`),
          context: { 
            data: post, 
            image: imageFluid   //pass the fluid object as context
          }
        })
      })
    })
  }

Then simply add this to any page, created by createPages

<Img fluid={pageContext.imageFluid}/>

And you get the post image.

@KyleAMathews, what do you think about this approach? It there a better way to accomplish this?

@DSchau DSchau added this to To do in OSS Roadmap via automation Mar 26, 2019

@DSchau DSchau moved this from To do to Prioritized in OSS Roadmap Mar 26, 2019

@baba43

This comment has been minimized.

Copy link

commented Apr 8, 2019

It's really sad that optimizing images using Gatsby results in so much overhead, unreadable und unreusable code. We've been so happy after being able to simply import images from everywhere with Webpack and other bundlers and now we're stuck with a solution that is so much worse when building static pages.

While I totally understand that Gatsby has a specific way to handle data and stuff, we should not accept that the most basic things have to be done so complicated.

@KyleAMathews

This comment has been minimized.

Copy link
Contributor Author

commented Apr 8, 2019

@baba43 — no one is accepting anything :-P that's why this issue exists.

We'd appreciate your help fixing it. Gatsby is a community built project so needs strong developers like yourself to help improve things.

@guyinpv

This comment has been minimized.

Copy link

commented Apr 13, 2019

It occurs to me that perhaps instead of having to duplicate and generate new queries for every possible variable used within , you could assemble all the possible variables into a list in the query itself. Kind of like WHERE something IN [array,of,all,vars,found]

Then the compiler just has to recognize the query as a set, and loop over it to generate each copy of the component. It only has to somehow know which component render belongs to which index of the returned set!

Just spitballing on that little idea.

@hiddentao

This comment has been minimized.

Copy link
Contributor

commented Apr 22, 2019

For dynamically querying images, I'm currently simply fetching all nodes and then filtering within the results:

import safeGet from 'lodash.get'
import React, { useMemo } from "react"
import { graphql, useStaticQuery } from "gatsby"
import Img from "gatsby-image"

const Image = ({ src, ...props }) => {
  const data = useStaticQuery(graphql`
    query {
      allFile( filter: { internal: { mediaType: { regex: "/image/" } } } ) {
        nodes {
          relativePath
          childImageSharp {
            fluid(maxWidth: 300) {
              ...GatsbyImageSharpFluid_noBase64
            }
          }
        }
      }
    }
  `)

  const match = useMemo(() => (
    data.allFile.nodes.find(({ relativePath }) => src === relativePath)
  ), [ data, src ])

  const fluid = safeGet(match, 'childImageSharp.fluid')

  return fluid ? (
    <Img
      fluid={fluid}
      Tag='div'
      {...props}
    />
  ) : null
}

export default Image
@nextlevelshit

This comment has been minimized.

Copy link
Contributor

commented Apr 24, 2019

Regarding the solution of @hiddentao I had to transform it to fit into my setup. I was to lazy to investigate what's exactly the point; might be the environment, versions etc. But heres an apadapted solution. Thanks for your input @hiddentao:

import React, { useMemo } from 'react'
import { graphql, useStaticQuery } from 'gatsby'
import Img from 'gatsby-image'

const Image = ({ src, ...props }) => {
  const data = useStaticQuery(graphql`
    query {
      allFile( filter: { internal: { mediaType: { regex: "images/" } } } ) {
        edges {
          node {
            relativePath
            childImageSharp {
              fluid(maxWidth: 300) {
                ...GatsbyImageSharpFluid
              }
            }
          }
        }
      }
    }
  `)

  const match = useMemo(() => (
    data.allFile.edges.find(({ node }) => src === node.relativePath)
  ), [ data, src ])

  return (
    <Img
      fluid={match.node.childImageSharp.fluid}
      {...props}
    />
  )
}

export default Image
@LeviSchuck

This comment has been minimized.

Copy link

commented Apr 24, 2019

I was doing something similar as hiddentao and nextlevelshit
I have since switched to pre-processing my markdown into an external graphql service to do more complicated mappings, the markdown makes references to entities via shortcodes (parsed with remark-shortcodes), which are then included as references in the graphql service.

Entity:

export default (document) => ({id, type}) => {
  switch (type) {
    case "image": {
      var images = document.referencedImages.filter(i => i.id === id);
      return (<Image data={images[0]} />)
    }
    default: {
      return (<div style={{color: "red"}}>Unhandled entity {type}:{id}</div>)
    }
  }
}

MarkdownDocument:

 html = remark()
        .use(shortcodes, {startBlock: "{{>", endBlock: "<}}"})
        .use(remark2rehype, {
          handlers: {
            shortcode: shortcodeHandler, /* This converts {{> Entity type="image" id="123" <}} to <Entity type="image" id="123 /> in HAST */
          },
        })
        .use(rehype2react, {
          createElement: React.createElement,
          components: {
            /* This converts the <Entity .. /> in HAST to a react one proper
                But I close over the document so it can access the referencedImages */
            Entity: Entity(document),
          },
        })
        .processSync(content)
        .contents;
...
return (<div>{html}</div>)

Template:

export default ({pageContext}) => {
    const thing = pageContext.thing;
    return (<Layout>
        <MDDocument document={thing.markdownDocument} full={true} />
    </Layout>)
}

gatsby-node

exports.createPages = ({ graphql, actions }) => {
  const { createPage } = actions;
  return graphql(`{
    externalgraphql {
      relatedThingHere {
        partialId
        markdownDocument {
          content
          excerpt
          referencedImages {
            id
            url
          }
        }
      }
    }
  }`).then(result => {
    result.data.externalgraphql.relatedThingHere.forEach(thing => {
      createPage({
        path: `/related-things/${thing.partialId}`,
        component: path.resolve(`./src/templates/template.js`),
        context: {
          thing
        },
      })
    });
  });
};

and in gatsby-config

{
      resolve: "gatsby-source-graphql",
      options: {
        // This type will contain remote schema Query type
        typeName: "ExternalQuery",
        // This is field under which it's accessible
        fieldName: "externalgraphql",
        // Url to query from
        url: "https://....",
      },
    },
@adamwade2384

This comment has been minimized.

Copy link

commented May 4, 2019

I'm currently working through a related issue with the useStaticQuery hook. I want to pass a variable to the hook without pulling all nodes and filtering client side as I'm concerned with performance, and unsure of how many nodes will be added to Contentful by content editors. Has there been any progress on this feature?

@me4502

This comment has been minimized.

Copy link
Member

commented May 7, 2019

Is there any update on this? I'm working on adding translations to components, and being able to query for strings based on a language tag in the component would be ideal here, rather than passing the strings from the page into the component.

raulrpearson added a commit to raulrpearson/gatsby-starter-mdx-basic that referenced this issue May 11, 2019

Add first working version of img override
This version transforms ![alt](src) in MDX into an Img component from
gatsby-image with all its goodness.

Taking some code and inspiration from:
gatsbyjs/gatsby#10482 (comment)

Which seems the best implementation until static query arguments are a
thing.

The beauty is that the component is used to override img tags with MDXProvider,
but can also be used directly. Notice that index.mdx is not modified in this
commit but still renders as before (even though the Image component was
modified).

There's still a problem: Img inserts a div that ends up inside a p. Will look
into fixing that next.
@landsman

This comment was marked as spam.

Copy link

commented May 17, 2019

I have exactly the same need

@denkhaus

This comment was marked as spam.

Copy link

commented May 17, 2019

the same here

lourd added a commit to lourd/descioli-design that referenced this issue May 19, 2019

Refactors to MDX and TypeScript
- Swaps out gatsby-remark-plugin for gatsby-mdx.
- Removes custom remark plugins, replacing them with components. Lost
the multi-resolution images for portfolio posts because it was too
much of a pain. Allowing static variables in static queries will make
this much simpler. See gatsbyjs/gatsby#10482
- Moves everything to TypeScript, including changes component files to
.tsx. Just gives better compiler feedback. Can still live variables
implicitly `any` all over the place, but can add types where desired.
- Removes propt-types
- Refactors use of StaticQuery component to useStaticQuery hook 👍👍
- Uploads a boatload of dependencies, including gatsby v1 to v2
- Combines src/lib/components into src/components

lourd added a commit to lourd/descioli-design that referenced this issue May 19, 2019

Refactors to MDX and TypeScript
- Swaps out gatsby-remark-plugin for gatsby-mdx.
- Removes custom remark plugins, replacing them with components. Lost
the multi-resolution images for portfolio posts because it was too
much of a pain. Allowing static variables in static queries will make
this much simpler. See gatsbyjs/gatsby#10482
- Moves everything to TypeScript, including changing component files to
tsx. Just gives better compiler feedback. Can still leave variables
implicitly `any` all over the place, but can add types where desired.
- Upgrades from gatsby v1 to v2
- Uploads a boatload of other dependencies, and removes a few unused
- Refactors use of StaticQuery component to useStaticQuery hook 👍👍
- Combines src/lib/components into src/components

lourd added a commit to lourd/descioli-design that referenced this issue May 19, 2019

Upgrades to Gatsby v2, switches to MDX & TypeScript
- Swaps out gatsby-remark-plugin for gatsby-mdx.
- Removes custom remark plugins, replacing them with components. Lost
the multi-resolution images for portfolio posts because it was too
much of a pain. Allowing static variables in static queries will make
this much simpler. See gatsbyjs/gatsby#10482
- Moves everything to TypeScript, including changing component files to
tsx. Just gives better compiler feedback. Can still leave variables
implicitly `any` all over the place, but can add types where desired.
- Upgrades from gatsby v1 to v2
- Uploads a boatload of other dependencies, and removes a few unused
- Refactors use of StaticQuery component to useStaticQuery hook 👍👍
- Combines src/lib/components into src/components
@superfunkminister

This comment has been minimized.

Copy link

commented May 20, 2019

+1

@DSchau DSchau changed the title Thinking around how to pass arguments to <StaticQuery> [feat]: add ability to pass variables to StaticQuery May 21, 2019

@hkr86

This comment has been minimized.

Copy link

commented May 21, 2019

Fetching all nodes and then filter on client does not scale very well! I've done that and my builds is now several MB large!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.