Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'relative path' field for uploaded images #325

Open
MarcCoet opened this issue Mar 27, 2017 · 51 comments

Comments

Projects
@MarcCoet
Copy link

commented Mar 27, 2017

Hi,
I am starting to use Netlify CMS with Gatsby 1.0.
I need a relative path to fetch pictures with the graphQL layer. So in my frontmatter I would like to have a field like:
fieldName: ../images/myImage.jpg
Of course the last part ("myImage.jpg") would be different for each entry depending on the file I upload.

Is there a way to auto generate a string field in the frontmatter that would concatenate a string of my choice (for example '../images/') with the name of a picture I may upload in an image field of the same entry ?

I also tried to set:
public_folder: "../images"
This would theoretically change all my image fields in all collections to get that leading string in the file path but when I create a new file with the CMS I get an auto opening '/'... So in my frontmatter:
image: /../images/myImage.jpg
instead of
image: ../images/myImage.jpg
Of course this breaks my groove as well.

What is the best way to get around this issue?

@erquhart

This comment has been minimized.

Copy link
Member

commented Mar 29, 2017

@MarcCoet a simple, non-breaking approach would be to accept a public_folder_relative setting in the config - set it to true to avoid the prepended slash. If you're up for giving this a shot, implementation would probably look like this:

  1. Update the resolvePath helper to accept a third option allowRelative.
  2. Update the two places in that function where slashes are prepended to only do so if the allowRelative param is not truthy.
  3. Update the assetProxy function to pass in the new config value as a third param to resolvePath.
@erquhart

This comment has been minimized.

Copy link
Member

commented Jun 28, 2017

This issue should be at least helped by #350.

@brandonmp

This comment has been minimized.

Copy link

commented Sep 2, 2017

@MarcCoet FWIW a quick workaround is to just edit the markdown of your post to the correct relative path from within netlify cms (ie, just toggle the 'Markdown' switch and edit the path)

It'll break the preview, but so far it appears to push fine. here's my config:

...
media_folder: content/blog-posts/media
public_folder: ../media/
collections:
  - name: blog
    label: Blog
    folder: content/blog-posts/posts
...
@erquhart

This comment has been minimized.

Copy link
Member

commented Dec 8, 2017

I still think this use case should be supported, but it merits further discussion.

@fk

This comment has been minimized.

Copy link

commented Dec 11, 2017

Hey! 👋 I took a first quick shot at what @erquhart outlined in #325 (comment) over the weekend.

Previews don't work yet (haven't looked into the code much) but this does enough to let a friend of mine that I'm helping with his portfolio site author his stuff – relative paths are correctly inserted in Markdown. My context is exactly like discussed in #843 (using Gatsby).

I would love to provide a PR for this eventually but most probably will need some guidance.

@erquhart

This comment has been minimized.

Copy link
Member

commented Dec 19, 2017

@fk that's awesome!

In the time since I wrote that comment, I've come to avoid "switches" that affect other settings in the config. A better approach that's still non-breaking would be to just check if the path begins with one or two dots and a slash and don't prepend a slash if so. We'd want to test it out a bit for unexpected consequences, though. Are you up for giving that a shot?

@fk

This comment has been minimized.

Copy link

commented Dec 19, 2017

Hey @erquhart! Yes! Up for anything! ;-) :-D

I also have been thinking a little about how to approach this feature in the meantime. I didn't have much time, so I only came as far as to look into assetProxy in regards on how to restore media previews in the editor and preview pane.

I also started to wonder if approaching this change from what I believe is a quite specific, "Gatsby" point of view, is the right thing to do to actually land a PR that is beneficial to anyone else – in other words: "What are other use cases for a relative public path (which couldn't be resolved by just changing the location of the media folder)?" is a question I figured I should ask here before going any further.

A quick recap of my specific use case: For Gatsby to be able to process URLs in Markdown (process images, copy linked files), it requires them to be relative (I still have to check back with @KyleAMathews on that, but I think "it's a Webpack thing") – basically what surfaced in #843.
My initial approach was to add the public_folder_relative option as you suggested (and then somehow make previews work with that), and to put all my markdown content at the same "distance" to the media folder – e.g. if my public_folder_relative was ../../img, all my Markdown content would have to live two folders down from img per convention.

Ideally I thought netlify-cms could resolve the path to be inserted in Markdown relative to the Markdown source and the location of the media folder. But somehow I feel like I'm missing something obvious still…

I've come to avoid "switches" that affect other settings in the config. A better approach that's still non-breaking would be to just check if the path begins with one or two dots and a slash and don't prepend a slash if so.

Yes, totally can follow your thoughts. FWIW, I've been seeing sindresorhus/is-relative-url being used in Gatsby.

We'd want to test it out a bit for unexpected consequences, though. Are you up for giving that a shot?

Yes! How about I adjust my fork to look at the path/remove the new public_folder_relative option and report back?

@erquhart

This comment has been minimized.

Copy link
Member

commented Dec 24, 2017

@fk apologies for the delay - that would be awesome. The problem and solution can really approached quite abstractly: for myriad reasons, folks sometimes want to set their media folder via relative path. I can't think of other specific use cases off the top of my head.

@tech4him1

This comment has been minimized.

Copy link
Collaborator

commented Dec 25, 2017

Hugo is going to be supporting image processing soon (before Jan 1!), and if I understand correctly, it will only work with relatively placed images. We probably want to prioritize this a little more.

@tech4him1

This comment has been minimized.

Copy link
Collaborator

commented Dec 25, 2017

@erquhart I don't quite understand how the media library will handle relatively placed media. Will it just walk through all the directories to find files that match the media_folder string (via regex?)?

Also, what do we do if an image is added to one entry, and then it is added to a different post via the media library? Do we just add the path relatively to the old location, or would we copy the actual image to the second location as well?

@ilyabo

This comment has been minimized.

Copy link

commented Dec 30, 2017

I found another workaround. Maybe it will be useful for somebody else. The path added to the markdown files by Netlify CMS can be adjusted and set (as Gatsby requires) to the relative path from the particular node (markdown file) the image is used in. Adding the following code to gatsby-node.js did the trick for me:

exports.onCreateNode = ({
  node,
  getNode,
  loadNodeContent,
  boundActionCreators,
}) => {
  const { frontmatter } = node
  if (frontmatter) {
    const { image } = frontmatter
    if (image) {
      if (image.indexOf('/img') === 0) {
        frontmatter.image = path.relative(
          path.dirname(node.fileAbsolutePath),
          path.join(__dirname, '/static/', image)
        )
      }
    }
  }
}

This assumes that config.yml has the media file paths set as follows:

media_folder: "static/img"
public_folder: "/img"

Also, the name of the markdown fields this would apply for is hardcoded above as image.

@KyleAMathews

This comment has been minimized.

Copy link
Contributor

commented Dec 30, 2017

You shouldn't mutate node data. The data should be treated as immutable (I'm actually surprised that worked as I thought we were already passing a copy of data to APIs to prevent exactly this sort of thing. Instead use https://www.gatsbyjs.org/docs/bound-action-creators/#createNodeField

@ilyabo

This comment has been minimized.

Copy link

commented Dec 30, 2017

Ok, makes sense. I actually tried to use the spread operator to avoid mutation, but somehow it didn't work. Will try with createNodeField

@zionis137

This comment has been minimized.

Copy link

commented Dec 30, 2017

how would the graphQL query look like with createNodeField?

This works:


markdownRemark{
     frontmatter {
        image
      }
      fields {
        imageRel
      }
}

But nicer would be:

markdownRemark{
     frontmatter {
        image {
          relativePath 
            // and then here the childImageSharp stuff
        }
      }
}

which fails with: GraphQL Error Field "image" must not have a selection since type "String" has no subfields.

@tech4him1

This comment has been minimized.

Copy link
Collaborator

commented Apr 10, 2018

@erquhart @talves Is it worth adding @danielmahon's plugin to gatsby-plugin-netlify-cms in some way?

@danielmahon

This comment has been minimized.

Copy link
Contributor

commented Apr 10, 2018

@tech4him1 Not sure if that would work as it is. The plugin basically has to "intercept" the node returned from gatsby-transformer-remark before it gets to gatsby-remark-images.

It bascially just sets the node.url to path.relative(node.parent.dir, file.url).

So, given a node like /src/pages/blog/my-first-post.md, with an image src of /uploads/fat-racoon.jpg, and the referenced image in /static/uploads it changes the node.url to .../.../../static/uploads/fat-racoon.jpg. Which lets gatsby-remark-images resolve it correctly when it does path.join(node.parent.dir, node.url).

You can "maybe" see it in action here:
https://github.com/danielmahon/gatsby-starter-procyon/tree/netlifycms.

I say maybe because I'm still working on that starter, expecially the netlifycms branch.
So get ready to step in some 💩.

@brendan-hurley

This comment has been minimized.

Copy link

commented Apr 10, 2018

@danielmahon tried your package but nothing appeared to happen. Upon debugging it, for whatever reason the select(markdownAST, "image") and select(markdownAST, "html") return nothing, despite there being plenty of html content. Wondering if you or anyone had any ideas why that might be? For what it's worth, my config is pretty much set up the same as your example.

@danielmahon

This comment has been minimized.

Copy link
Contributor

commented Apr 10, 2018

@brendan-hurley Hmm, not sure. markdownAST is passed to the plugin array via gatsby-transformer-remark. Those calls are mirroring how gatsby-remark-images grabs the images and html nodes so as long as you have the plugin in the correct place (right before gatsby-remark-images) then it should work . Double check your config. gatsby-remark-relative-images should be inside the plugins array of gatsby-transformer-remark and before gatsby-remark-images. If markdownAST isnt returning anything then something else is going on upstream with the remark parsing.

I've tested this with md ![](/path/to/fat-racoon.jpg), and html <img src="/path/to/fat-racoon.jpg"/>. Also works with nested img tags.

...
    {
      resolve: 'gatsby-transformer-remark',
      options: {
        plugins: [
          {
            resolve: 'gatsby-remark-relative-images',
            options: {
              name: 'uploads',
            },
          },
          {
            resolve: 'gatsby-remark-images',
            options: {
              // It's important to specify the maxWidth (in pixels) of
              // the content container as this plugin uses this as the
              // base for generating different widths of each image.
              maxWidth: 720,
            },
          },
        ],
      },
    },
...
@brendan-hurley

This comment has been minimized.

Copy link

commented Apr 10, 2018

@danielmahon Aye I have it set up as such:

plugins: [
    'gatsby-plugin-react-next',
    'gatsby-plugin-react-helmet',
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        path: `${__dirname}/static/content`,
        name: 'content',
      },
    },
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        path: `${__dirname}/static/assets`,
        name: 'assets',
      },
    },
    'gatsby-transformer-sharp',
    { 
      resolve: 'gatsby-transformer-remark',
      options: {
        plugins: [
          {
            resolve: `gatsby-remark-relative-images`,
            options: {
              name: 'assets',
            },
          },
          {
            resolve: `gatsby-remark-images`,
            options: {
              maxWidth: 1200,
            },
          },
        ],
      },
    },
    '@jacobmischka/gatsby-plugin-react-svg',
    {
      resolve: 'gatsby-plugin-postcss-sass',
      options: {
        postCssPlugins: [
          pixrem(),
          autoprefixer({
            browsers: ['last 2 versions']
          }),
        ],
        precision: 8,
      },
    },
    {
      resolve: 'gatsby-plugin-netlify-cms',
    },
    'gatsby-plugin-sharp',
    'gatsby-plugin-netlify',
  ],
@danielmahon

This comment has been minimized.

Copy link
Contributor

commented Apr 10, 2018

@brendan-hurley Open an issue and include a sample markdown string you're trying to parse, and I'll take a look. Trying not to hijack this thread 😉

@ahoward

This comment has been minimized.

Copy link

commented Apr 12, 2018

fwiw, a 'public_folder' config containing an absolute url is also munged with a leading /WHACK -> -- after having much discussion in gitter, and reading (grepping really...) the source i think the global, and correct, fix is to never touch 'public_folder' , eg. never prepend a /...

understanding that this could break existing customers but, also, from a naming perspective my suggestion is similar to others in that i'd add another key, eg 'media_prefix' and prefer that when expanding links in content. this value should be a promise that would always resolve a media item whether static or dynamic. it would have a very, very simple contract: media_prefix + '/' + media_filename is valid. what that might be so is the customer's problem - not netlify's

@PaulEibensteiner

This comment has been minimized.

Copy link

commented Jun 1, 2018

Hi everyone!
I had the problem of using Hugos image processing while at the same time maintaining the easy image adding that netlify cms provides.
I used the method talves provided to store all media in a folder in the content directory. Then I created a shortcode which resizes the image and inserts the link. Now I created a custom editor widget for the netlify cms which inserts the shortcode by using the media library and edits the paths, so preview and output work.
I can provide all the files if anyone's interested
(This is my first post in a github discussion - I hope I'm in the right place ˆˆ)

@erquhart

This comment has been minimized.

Copy link
Member

commented Jun 1, 2018

Hi @PaulEibensteiner - welcome, you're in the right place! And your solution sounds awesome! Can you put your code in a gist and provide a link?

@talves

This comment has been minimized.

Copy link
Collaborator

commented Jun 1, 2018

@PaulEibensteiner great job. I was planning to eventually get to creating a widget for a Hugo shortcode, but have been really busy with some other fun stuff that will be coming soon.

@PaulEibensteiner

This comment has been minimized.

Copy link

commented Jun 1, 2018

Sure!
So first you have to create the directory "images" in your content directory, and add the _index.md with following front matter:

---
title: Media Folder
headless: true
---

Then the shortcode in the layouts/shortcodes directory:
https://gist.github.com/PaulEibensteiner/c88e14439abc1111f7388520ade2cf36

Now the custom editor widget: I have a 1-year knowledge of self taught javascript so this might be a complete mess - also I didn't quite understand the documentation about building a custom widget. So I just added the following code at the bottom of my static/admin/index.html (after the <script src="https://unpkg.com/netlify-cms@^1.0.0/dist/cms.js"></script> tag):
https://gist.github.com/PaulEibensteiner/c2f1d107deef3a0d7d30fa25ef0a198e
I know that the method to remove and add the path sequence for the folder is really messy. Ideal would probably be to access the public_folder and media_folder configuration and change the path accordingly, I just don't have a clue how to to that - and it works so whatever...
(You could also remove the path like others did in the shortcode via go...but there you wouldn't even have the option to change it based on the folder name I guess)

  • now you only have to move all your images to the new location and change all links to the new shortcodes...and it should work

ps: I might have messed up the naming of elements because I already changed some stuff in the original project but omitted it here for 'simplicity' - I hope it works. Pls let me know if anything is flawed or if my explanations are confusing
Have fun coding - Hugo's pretty awesome ˆˆ

@talves

This comment has been minimized.

Copy link
Collaborator

commented Jun 1, 2018

@PaulEibensteiner This is good stuff. Don't worry about the widget, I will eventually formalize yours into an npm module if you don't get to it first. Again, great job. Also, yes Hugo is pretty Awesome!

@PaulEibensteiner

This comment has been minimized.

Copy link

commented Jun 1, 2018

@talves Thank you, yeah, I don't really know about npm and that stuff ˆˆ But that would be awesome...
Quick question off-topic: Are there custom widgets provided by other users somewhere on the internet? I could imagine there are a lot of examples (like the yt widget in the docs) which a lot of people would find useful? Googled and didn't find anything...

@talves

This comment has been minimized.

Copy link
Collaborator

commented Jun 1, 2018

We have been discussing creating a place for people to publish information about their custom widgets on the NetlifyCMS site somewhere. You are right, and a plan to have a full information format will help quite a bit. I am sure we will get there. We just need someone in the community to take it on when they get time. I might tackle it with some other stuff I am doing, but it is lower on the priority at the moment.

I created an issue for discussion here, rather than us keep hijacking this issue.

@bmingles

This comment has been minimized.

Copy link

commented Oct 13, 2018

@zionis137 Responding to an old comment of yours #325 (comment)

If you use createNodeField to convert frontmatter.image to a relative path in gatsby-node.js:

createNodeField({
  node,
  name: 'imageRel',
  value: someConversionToFileSystemPath(node.frontmatter.image)
});

then you should be able to query your image node like this:

markdownRemark{
    frontmatter {
        image // this will still be original string that couldn't resolve to file system path
    }
    fields {
        imageRel { // this will contain a file node
            relativePath 
            // and then here the childImageSharp stuff
        }
    }
}

As I understand it, any paths that Gatsby can identify as a file system path will be converted to file nodes in the query results in which case you can access relativePath, and childImageSharp, etc.

@johnclark456

This comment has been minimized.

Copy link

commented Oct 13, 2018

Thanks, that works great, I used a trick like this to let me leave the images in src/static whilst letting me process them in remark-images. Doing that means the netlify CMS previews work as expected as well.

@jslyonnais

This comment has been minimized.

Copy link

commented Oct 17, 2018

I've found this working damn great!
Image Loading with Gatsby v2 and Netlify CMS v2
Can also find the link to the dude repos in the article if you wanna see the full config.

@joshuarule

This comment has been minimized.

Copy link

commented Oct 18, 2018

Has anyone tried this with an animated gif file? I previously used @danielmahon plugin and had that working...but when my client just decided they wanted a gif...I'm now getting errors that childImageSharp is null. Does it behave differently with a gif vs jpg? I had looked at this...but on first try didn't have much luck https://www.gatsbyjs.org/packages/gatsby-remark-copy-linked-files/?=copy-

@danielmahon

This comment has been minimized.

Copy link
Contributor

commented Oct 19, 2018

@joshuarule check the ‘gatbsy-image’ docs but I think GIFs and SVGs aren’t supported...maybe?

@johnclark456

This comment has been minimized.

Copy link

commented Oct 19, 2018

@joshuarule I'm assuming this is still the case? gatsbyjs/gatsby#7317 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.