New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Host images using a 3rd party service #432

Closed
bdougie opened this Issue May 26, 2017 · 31 comments

Comments

@bdougie
Contributor

bdougie commented May 26, 2017

- Do you want to request a feature or report a bug?
Feature

- What is the current behavior?
Currently you can choose a media folder and upload images there directly using the CMS. This is great1

- What is the expected behavior?
It would be nice to be able to directly upload images to a 3rd party service like S3 or similar in order to save on build size and such. I would love to discuss the possibilities on this sometime and making this issue to remind myself to bring it up.

screenshot 2017-05-26 09 19 55

@erquhart

This comment has been minimized.

Member

erquhart commented Jun 28, 2017

This could happen in tandem with #350.

@erquhart erquhart added this to the 1.0 milestone Jun 28, 2017

@g12n

This comment has been minimized.

g12n commented Jul 4, 2017

This would be a blast! Espeacially when Working with image processing services like imgix.
Optimizing images could be handled within Templates.

@nytyp nytyp added the area: ux label Jul 11, 2017

@erquhart erquhart modified the milestones: 0.5.0, 1.0 Aug 30, 2017

@erquhart erquhart removed this from the 0.5.0 milestone Sep 7, 2017

@dopry

This comment has been minimized.

Contributor

dopry commented Sep 14, 2017

@bdougie, @g12n, what 3rd party services do you currently use? Which ones would you like to see supported?

@bdougie

This comment has been minimized.

Contributor

bdougie commented Sep 14, 2017

I only have experience with cloudinary and the one I had in mind. I believe it uses S3 under the hood.

@g12n

This comment has been minimized.

g12n commented Sep 15, 2017

imgIX currently supports 3 Kinds of sources:

Amazon S3: Connects to an existing Amazon S3 bucket with its own credentials.
Web Folder: Proxies content from a given domain (or a subfolder on the domain).
Web Proxy: Can proxy any valid image URL.

S3 Bucket work good for image Storage. Additional services like imgIX would be a great addition, because they really help processing different image sizes (responsive images) and formats (webp).

@jessegavin

This comment has been minimized.

jessegavin commented Oct 4, 2017

I am looking to port an image centric website over to use netlifycms. All our assets are stored in S3 and hosted through imgix currently (though we're looking to set up a custom image server using imageflow in the near future).

If netlifycms had a flexible storage adapter which works with common 3rd party file storage solutions that would be awesome!

@matthewselby

This comment has been minimized.

matthewselby commented Oct 18, 2017

Related to the Feature Request here, I have started using the Media Cloud plugin on my WP sites to offload my media library and serve images over imgIX. It has been amazing to include this on my WP sites and I would love to see Custom Asset management/pipelines be as seamless on Netlify CMS.

While most of the conversation above has been about image assets, there are many other asset types that we can expect our clients to be uploading such as video, audio, zip folders, pdf, etc. From all my own clients that need a CMS, they upload files that are not images more often than they upload images. Specifically, a few of them have podcasts and rarely upload files that are not audio. My current workaround for CMS sites without asset piping options is to have the client use a different 3rd party service to interact with a storage service and then have them paste the file URL into a string field...only one client has no-complaints accepted this.

So at least for me, asset pipelines/offloading in a CMS is a must.

That all being said...here are some of my requests for sources:

I currently use Backblaze B2 but I have been keeping an eye on Wasabi and would be on them now if I was still on S3.

@talves

This comment has been minimized.

Collaborator

talves commented Oct 18, 2017

@matthewselby I added Cloudinary to your list 👍

@matthewselby

This comment has been minimized.

matthewselby commented Oct 19, 2017

@talves Thanks! That's right, Cloudinary has their own asset hosting.

They also support Remote Images to serve and do their magic with images that are hosted elsewhere. Any Cloudinary folks know if there is a downside to using Remote Images over Cloudinary hosted images?

@agush22

This comment has been minimized.

agush22 commented Feb 15, 2018

This would be an awesome feature 👍

@erquhart erquhart added the hot label Feb 22, 2018

@r1b

This comment has been minimized.

r1b commented Feb 23, 2018

I help maintain an image-heavy gallery site - we have reached the point with the volume and size of images in our repo where it no longer makes sense to store them in git! :)

I am very excited about this feature - interim we will probably move to a solution where we manage the images externally and only store links / references in the CMS, similar to what was suggested above.

@01ivr3

This comment has been minimized.

01ivr3 commented Mar 7, 2018

This could also be a good workaround for not being able to store images to page collections, which prevents NetlifyCMS from working with Hugo's new native image processing ref: #325

@01ivr3

This comment has been minimized.

01ivr3 commented Mar 7, 2018

+1 for Cloudinary and Imgix

@crgeary

This comment has been minimized.

crgeary commented Mar 8, 2018

+1 for Cloudinary & S3.


@01ivr3 if you set the uploads directory for assets to /content/uploads you can then use those images as page resources, and then are able to crop/resize etc :)

You will also need an index.md file set to headless: true in the directory. Then something similar to this will work:

{{ $src := .Params.image }}

{{ with .Scope.Site.GetPage "page" "uploads/index.md" }}
    {{ $match := .Resources.GetMatch (strings.TrimPrefix "/uploads/" $src) }}
    {{ $image := $match.Fill "200x200 center" }}
    <img src="{{ $image.RelPermalink }}">
{{ end }}
@01ivr3

This comment has been minimized.

01ivr3 commented Mar 22, 2018

@crgeary huge thanks! Your solution works perfectly

@jnowland

This comment has been minimized.

jnowland commented Mar 29, 2018

Keeping large binaries out of a git repo seems like a smart idea for large photo heavy projects.

@erquhart erquhart added the roadmap label Apr 3, 2018

@owenhoskins

This comment has been minimized.

owenhoskins commented Apr 3, 2018

@jnowland is correct -- I've just hit some kind of limit with my git repo.

The repo is a little over 1gb with about 5000 image assets and the Media Libraries Image Upload functionality via the github API started returning a 500 Server Error sometime after passing 3.5k image assets.

I was hoping to get away with storing assets in the repo while a solution for offloading them could be developed but it looks like offloading has become my only option!

So I am going to attempt to tackle this, probably with an S3 bucket and imgix or similar so that I can create responsive graphql queries with GatsbyJS like https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-source-contentful/src/fragments.js.

Any guidance or help on getting this kick started would be greatly appreciated.

@knpwrs

This comment has been minimized.

knpwrs commented Apr 3, 2018

I would like to expand the scope of this a little bit from just images to arbitrary blobs. My use-case is storing audio and video on an external service in addition to images.

@erquhart

This comment has been minimized.

Member

erquhart commented Apr 3, 2018

@Benaiah I'm thinking this needs to happen on top of the upcoming backend API work, can you weigh in here?

@Benaiah

This comment has been minimized.

Contributor

Benaiah commented Apr 3, 2018

@erquhart right - the backend API redesign will be moving the integrations and backends to a single API, which will include the API currently used by integrations to implement this feature for Smashing. Once that's done, this can be implemented for any storage backend (and there's nothing I'm aware of that would limit it solely to images). It looks to me like Cloudinary is the one with the most immediate interest, but it should be pretty straightforward to implement others once we have the system working.

@brandonkal

This comment has been minimized.

brandonkal commented Apr 12, 2018

This would be awesome

@jnowland jnowland referenced this issue Apr 26, 2018

Open

Netlify CMS 2.0 #1280

1 of 10 tasks complete
@haileypate

This comment has been minimized.

haileypate commented Apr 27, 2018

Lots of use cases wrapped up in this thread!

I'm picking up on these high-level use cases for us to consider as discussion about image management continues:

  • allow for hosting of media library assets by 3rd party services
  • allowing site developers to leverage 3rd party image processing services which do things like:
    • automate the process of deriving "responsive image" collateral (for example, using an API to derive different sizes of an image file for use at certain breakpoints)
    • optimize an image for performance (for example, use a 3rd party API to compress uploaded files)
  • helping users upload the right photo for a particular post

hmm.. what else might i add? :D

@agush22

This comment has been minimized.

agush22 commented Apr 28, 2018

Hey thread, please give a thumbs up to this comment, so it gains more visibility for the 2.0 Roadmap. Cheers! 👍

@haileypate haileypate referenced this issue Apr 30, 2018

Open

Image Handling Umbrella Issue #1321

1 of 1 task complete
@cpjolicoeur

This comment has been minimized.

cpjolicoeur commented May 4, 2018

I'd love this feature as well. It affects those of us who host the CMS backend in a private repository. Media uploaded via the CMS currently won't be publically viewable in the final builds of our static site, as the media is inaccessible in a private repository. Would love to be able to upload directly to AWS S3 instead as the media backend.

@erquhart

This comment has been minimized.

Member

erquhart commented May 5, 2018

@cpjolicoeur that shouldn't be a problem currently - if you're storing assets in your repo, you should be publishing those assets in your build so they're publicly accessible. Your site's image src attributes shouldn't be pointing to your repo.

Sent with GitHawk

@cpjolicoeur

This comment has been minimized.

cpjolicoeur commented May 7, 2018

@erquhart agreed. I realized this shortly after posting this comment and updated our deploy to do that. Forgot to come back here and follow-up/delete the comment however.

Thanks for the response.

@erquhart

This comment has been minimized.

Member

erquhart commented Jun 12, 2018

This will be supported by #1171. That issue will provide the necessary API(s), and this issue will bring in one or more actual extensions to integrate with specific services.

@dev7ch

This comment has been minimized.

dev7ch commented Aug 4, 2018

While searching about how to implement Google Drive Hosted public images into a JAM Stack i came here.

Due @haileypate already mentioned it´s a multi topic issue, i am not sure if this is the right place to ask but my case is somehow related to 3rd Images.

Currently, I am using static Nuxt (Vue) with contents based on markdown files and netlify cms to manage the md files.

Atm I see it´s possible to handle the Drive Rest API and images with vue, nuxt, axios, etc nicely but it would be super cool to have a visual admin for that in the wonderful netlify headless admin app.

Drive Rest Web implementation examples:

https://github.com/gsuitedevs/browser-samples/blob/master/drive/quickstart/index.html
https://developers.google.com/drive/api/v3/quickstart/js

Is there a way to connect to read images from Drive Rest API in admin UI and implement them into the .md files as resources?

What do you think about the amount of effort which is needed to make that happen?

@mittalyashu

This comment has been minimized.

Contributor

mittalyashu commented Aug 4, 2018

Integration with Imgur

@erquhart

This comment has been minimized.

Member

erquhart commented Aug 8, 2018

Check out this blurry gif:

netlify-cms-cloudinary

This approach is based on a new API for registering an external widget library. I'm only aware of two services that have media library widgets, Cloudinary and Uploadcare. I'll likely put together integrations for both if we go this route.

Thoughts? Know of any other services with media library or upload widgets?

@erquhart erquhart referenced this issue Aug 11, 2018

Merged

External media libraries #1602

1 of 1 task complete
@erquhart

This comment has been minimized.

Member

erquhart commented Aug 11, 2018

Opened #1602 to satisfy this issue, check it out!

2.0 Release automation moved this from Needs Planning to Complete Aug 30, 2018

@erquhart erquhart added this to Complete in Priority Issues Aug 30, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment