Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question: How do I query based on gatsby-source-filesystem name? #1634

Closed
saviomuc opened this Issue Jul 27, 2017 · 16 comments

Comments

Projects
None yet
9 participants
@saviomuc
Copy link

saviomuc commented Jul 27, 2017

I have multiple folders for different page types. Ie. type1 , type2, etc. which contain markdown files.

pages
   --type1
   --type2

The files in each folder are transformed using gatsby-transformer-remark.
I want to query only content for type1, type2, etc.
For instance I want a type1 list page which contains a list of all type1 pages.

gatsby-config.js

   {
      resolve: 'gatsby-source-filesystem',
      options: {
        path: `${__dirname}/src/pages/type1`,
        name: 'type1',
      },
    },
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        path: `${__dirname}/src/pages/type2`,
        name: 'type2',
      },
    },

My assumption was that you could query based on the name property.

Something like:

{
  someQueryForGatsbySourceFilesystem(
    filter: {
      name: {
        eq: "type1"
      }
    }
  ) {
    edges {
      node {
        childMarkdownRemark {
          html
        }
      }
    }
  }
}

I tried all different queries like allSite and allFile but could not find a solution.
I also tried

  allSitePlugin(filter: {
    pluginOptions: {
      name: {
        eq: "type1"
      }
    }
  })

which doesnt seem to return something which I can use to render content:

"Cannot query field \"childMarkdownRemark\" on type \"SitePlugin\".",

How would you do it? Use allFile and filter based on the path name (i.e. a regex which matches the folder type1 and markdown extension)?

 allFile(
  filter: {
    absolutePath:{regex:"/(type1)\/.*\\.md$/"}
  }
) {...}
@saviomuc

This comment has been minimized.

Copy link
Author

saviomuc commented Jul 27, 2017

I figured you could also filter the path using allMarkdownRemark:

{
   allMarkdownRemark(
    sort: { order: DESC, fields: [frontmatter___date]},
    filter: {fileAbsolutePath: {regex: "/(type1)/.*\\.md$/"}}
  ) {
      edges {
        node {
          excerpt(pruneLength: 250)
          id
          frontmatter {
            title
            date(formatString: "MMMM DD, YYYY")
            path
          }
        }
      }
    }
}
@KyleAMathews

This comment has been minimized.

Copy link
Contributor

KyleAMathews commented Jul 27, 2017

File nodes have the source instance name set as sourceInstanceName so you can use that for queries.

Using regex is a great solution as well which we use a lot on gatsbyjs.org for example.

@KyleAMathews

This comment has been minimized.

Copy link
Contributor

KyleAMathews commented Jul 27, 2017

BTW, if you're not using it already, GraphiQL is great for exploring what data is available.

@nwshane

This comment has been minimized.

Copy link
Contributor

nwshane commented Sep 12, 2017

@KyleAMathews Is there a way to use sourceInstanceName in the allMarkdownRemark query? I was hoping that I could get the sourceInstanceName of the file node for each markdown node like this, so that I can separate my blog posts from my projects without using regex on the the file paths.

{
  allMarkdownRemark {
    edges {
      node {
        sourceInstanceName
      }
    }
  }
}

But the GraphQL debugger shows that sourceInstanceName is not available.

How do you recommend filtering markdown pages by source type, such as projects and blog posts? My current solution for this is to get the fileAbsolutePath of each markdown node:

{
  allMarkdownRemark {
    edges {
      node {
        fileAbsolutePath
      }
    }
  }
}

And then to test whether it includes certain strings, like /pages/blog/ or /pages/projects/. But this will break if the absolute path to the gatsby site directory itself contains one of these strings. What do you recommend as the best practice in this case? (Sorry if I missed something in the documentation.)

@KyleAMathews

This comment has been minimized.

Copy link
Contributor

KyleAMathews commented Sep 14, 2017

@nwshane just query allFile and filter there. You can filter for markdown files by internal.mediaType.

@chmac

This comment has been minimized.

Copy link
Contributor

chmac commented Feb 18, 2018

@KyleAMathews How does one combine a query on allFile with a sort on frontmatter? Is that possible?

@lukejanicke

This comment has been minimized.

Copy link

lukejanicke commented Feb 25, 2018

Trying to figure out something similar.

I can filter by subfolder. But I can’t see frontmatter in allFile.

screen shot 2018-02-25 at 8 18 51 p m

UPDATE:

This worked for me.

I have folders posts and pages inside the folder content and my gatsby-config.js points to content.

export const query = graphql`
  query IndexQuery {
    allMarkdownRemark(
      filter: { fileAbsolutePath: {regex : "\/posts/"} },
      sort: {fields: [frontmatter___date], order: DESC},
    ) {
      totalCount
      edges {
        node {
          id
          frontmatter {
            title
            slug
            date(formatString: "DD MMMM YYYY")
            category
          }
          excerpt
        }
      }
    }
  }
`;
@chmac

This comment has been minimized.

Copy link
Contributor

chmac commented Feb 25, 2018

@lukejanicke Since my last update I dived into the node system a little more. The allFile nodes get parsed by the markdown handler and then new allMarkdownRemark nodes get created. So as you found, it’s not possible to sort by frontmatter in allFile because it’s only generated once the markdown parser kicks in. The regex works, but it’s a bit fragile for my taste. My solution was to copy the collection name (specified in the filesystem source plugin config) from the file node into the markdown node via an onCreateNode hook. Don’t have the code in front me to post an example, but it was very quick and then I have a field inside allMarkdownRemark that I can filter by.

@gbouteiller

This comment has been minimized.

Copy link

gbouteiller commented Mar 7, 2018

Another possibility as @KyleAMathews advised ( in my case, the source name is projects )

allFile(filter: {internal: {mediaType: {eq: "text/markdown"}}, sourceInstanceName: {eq: "projects"}}) {
  edges {
    node {
      childMarkdownRemark {
        frontmatter {
            ...
        }
      }
    }
  }
}
@chmac

This comment has been minimized.

Copy link
Contributor

chmac commented Mar 13, 2018

@gbouteiller True, that will get the frontmatter and allow one to filter by sourceInstanceName. The trouble is, you cannot sort by anything inside frontmatter using this approach. It seems like folks are talking about blog posts here, and they are typically sorted by date. The date is typically stored inside the frontmatter.

There is no way to do sort by something from frontmatter and filter by sourceInstanceName by default. The sourceInstanceName is not copied from the File node into the markdownRemark node. The only option is to use a regex or to copy some data as per my earlier post.

If sorting is not required, which might be true in many cases, then your approach makes a lot of sense. I hadn't realised the childMarkdownRemark was available as a field on file, that's super useful to know, thanks for sharing.

@tsiq-swyx

This comment has been minimized.

Copy link
Contributor

tsiq-swyx commented May 8, 2018

had this issue to and was kind of disappointed that the "name" field doesnt work well with the markdownRemark. is there a small win here that we can do to pass this information on?

EDIT: after messing around a bit i realized that the id field exposes the path and you can filter based on that:

query AllQuery {
    DSes: allMarkdownRemark(
      limit: 3
      filter: { id: { regex: "/_design-systems/" } }
    ) {
      edges {
        node {
          frontmatter {
            title
            date
            company
            link
            image
            description
          }
          fields {
            slug
          }
        }
      }
    }

    Articles: allMarkdownRemark(
      limit: 3
      filter: { id: { regex: "/_articles/" } }
    ) {
      edges {
        node {
          frontmatter {
            title
            date
            company
            link
            image
            description
          }
          fields {
            slug
          }
        }
      }
    }
  }

the name field from source filesystem still doesnt really do anything for gatsby-transformer-remark, i would say it improves the dx a bit to flow that through as a queryable/filterable field since gatsby-transformer-remark never exists without gatsby-source-filesystem.

@chmac

This comment has been minimized.

Copy link
Contributor

chmac commented May 14, 2018

@tsiq-swyx I'm using something like this to pass the collection field from File to MarkdownRemark.

exports.onCreateNode = ({ node, boundActionCreators, getNode }) => {
  const { createNodeField } = boundActionCreators

  if (_.get(node, 'internal.type') === `MarkdownRemark`) {
    // Get the parent node
    const parent = getNode(_.get(node, 'parent'))

    // Create a field on this node for the "collection" of the parent
    // NOTE: This is necessary so we can filter `allMarkdownRemark` by
    // `collection` otherwise there is no way to filter for only markdown
    // documents of type `post`.
    createNodeField({
      node,
      name: 'collection',
      value: _.get(parent, 'sourceInstanceName'),
    })

The regex approach does work, but it's fragile.

@tsiq-swyx

This comment has been minimized.

Copy link
Contributor

tsiq-swyx commented May 15, 2018

@chmac this is very clever, and it works, thank you for sharing your solution. @KyleAMathews sorry for tagging you (lmk if theres someone else to tag) but is it possible for gatsby-source-filesystem to work this way (passing sourceInstanceName to a node field or some other field) by default?

@chmac

This comment has been minimized.

Copy link
Contributor

chmac commented May 15, 2018

@tsiq-swyx I'm not 100% sure, but I'd imagine it might be a fairly simple pull request! ;-)

@tsiq-swyx

This comment has been minimized.

Copy link
Contributor

tsiq-swyx commented May 15, 2018

i think the gatsby maintainers have strong opinions about these things so i think its worth discussing before sending in random unsolicited PRs :)

@dumblole

This comment has been minimized.

Copy link
Member

dumblole commented Sep 23, 2018

Okay i got it working, but my createPages export only uses one template

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.