Host images using a 3rd party service

[bdougie]

- Do you want to request a feature or report a bug?
Feature

- What is the current behavior?
Currently you can choose a media folder and upload images there directly using the CMS. This is great1

- What is the expected behavior?
It would be nice to be able to directly upload images to a 3rd party service like S3 or similar in order to save on build size and such. I would love to discuss the possibilities on this sometime and making this issue to remind myself to bring it up.

[erquhart]

This could happen in tandem with #350.

[g12n]

This would be a blast! Espeacially when Working with image processing services like imgix.
Optimizing images could be handled within Templates.

[dopry]

@bdougie, @g12n, what 3rd party services do you currently use? Which ones would you like to see supported?

[bdougie]

I only have experience with cloudinary and the one I had in mind. I believe it uses S3 under the hood.

[g12n]

imgIX currently supports 3 Kinds of sources:

Amazon S3: Connects to an existing Amazon S3 bucket with its own credentials.
Web Folder: Proxies content from a given domain (or a subfolder on the domain).
Web Proxy: Can proxy any valid image URL.

S3 Bucket work good for image Storage. Additional services like imgIX would be a great addition, because they really help processing different image sizes (responsive images) and formats (webp).

[jessegavin]

I am looking to port an image centric website over to use netlifycms. All our assets are stored in S3 and hosted through imgix currently (though we're looking to set up a custom image server using imageflow in the near future).

If netlifycms had a flexible storage adapter which works with common 3rd party file storage solutions that would be awesome!

[matthewselby]

Related to the Feature Request here, I have started using the Media Cloud plugin on my WP sites to offload my media library and serve images over imgIX. It has been amazing to include this on my WP sites and I would love to see Custom Asset management/pipelines be as seamless on Netlify CMS.

While most of the conversation above has been about image assets, there are many other asset types that we can expect our clients to be uploading such as video, audio, zip folders, pdf, etc. From all my own clients that need a CMS, they upload files that are not images more often than they upload images. Specifically, a few of them have podcasts and rarely upload files that are not audio. My current workaround for CMS sites without asset piping options is to have the client use a different 3rd party service to interact with a storage service and then have them paste the file URL into a string field...only one client has no-complaints accepted this.

So at least for me, asset pipelines/offloading in a CMS is a must.

That all being said...here are some of my requests for sources:

• S3
• Backblaze B2
• Wasabi
• GCE
• Cloudinary

I currently use Backblaze B2 but I have been keeping an eye on Wasabi and would be on them now if I was still on S3.

[talves]

@matthewselby I added Cloudinary to your list 👍

[matthewselby]

@talves Thanks! That's right, Cloudinary has their own asset hosting.

They also support Remote Images to serve and do their magic with images that are hosted elsewhere. Any Cloudinary folks know if there is a downside to using Remote Images over Cloudinary hosted images?

[agush22]

This would be an awesome feature 👍

[r1b]

I help maintain an image-heavy gallery site - we have reached the point with the volume and size of images in our repo where it no longer makes sense to store them in git! :)

I am very excited about this feature - interim we will probably move to a solution where we manage the images externally and only store links / references in the CMS, similar to what was suggested above.

[01ivr3]

This could also be a good workaround for not being able to store images to page collections, which prevents NetlifyCMS from working with Hugo's new native image processing ref: #325

[01ivr3]

+1 for Cloudinary and Imgix

[crgeary]

+1 for Cloudinary & S3.

---

@01ivr3 if you set the uploads directory for assets to /content/uploads you can then use those images as page resources, and then are able to crop/resize etc :)

You will also need an index.md file set to headless: true in the directory. Then something similar to this will work:

{{ $src := .Params.image }}

{{ with .Scope.Site.GetPage "page" "uploads/index.md" }}
    {{ $match := .Resources.GetMatch (strings.TrimPrefix "/uploads/" $src) }}
    {{ $image := $match.Fill "200x200 center" }}
    <img src="{{ $image.RelPermalink }}">
{{ end }}

[01ivr3]

@crgeary huge thanks! Your solution works perfectly

[jnowland]

Keeping large binaries out of a git repo seems like a smart idea for large photo heavy projects.

[owenhoskins]

@jnowland is correct -- I've just hit some kind of limit with my git repo.

The repo is a little over 1gb with about 5000 image assets and the Media Libraries Image Upload functionality via the github API started returning a 500 Server Error sometime after passing 3.5k image assets.

I was hoping to get away with storing assets in the repo while a solution for offloading them could be developed but it looks like offloading has become my only option!

So I am going to attempt to tackle this, probably with an S3 bucket and imgix or similar so that I can create responsive graphql queries with GatsbyJS like https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-source-contentful/src/fragments.js.

Any guidance or help on getting this kick started would be greatly appreciated.

[knpwrs]

I would like to expand the scope of this a little bit from just images to arbitrary blobs. My use-case is storing audio and video on an external service in addition to images.

[erquhart]

@Benaiah I'm thinking this needs to happen on top of the upcoming backend API work, can you weigh in here?

[Benaiah]

@erquhart right - the backend API redesign will be moving the integrations and backends to a single API, which will include the API currently used by integrations to implement this feature for Smashing. Once that's done, this can be implemented for any storage backend (and there's nothing I'm aware of that would limit it solely to images). It looks to me like Cloudinary is the one with the most immediate interest, but it should be pretty straightforward to implement others once we have the system working.

[brandonkal]

This would be awesome

[haileypate]

Lots of use cases wrapped up in this thread!

I'm picking up on these high-level use cases for us to consider as discussion about image management continues:

• allow for hosting of media library assets by 3rd party services
• allowing site developers to leverage 3rd party image processing services which do things like:
  • automate the process of deriving "responsive image" collateral (for example, using an API to derive different sizes of an image file for use at certain breakpoints)
  • optimize an image for performance (for example, use a 3rd party API to compress uploaded files)
• helping users upload the right photo for a particular post

hmm.. what else might i add? :D

[agush22]

Hey thread, please give a thumbs up to this comment, so it gains more visibility for the 2.0 Roadmap. Cheers! 👍

[cpjolicoeur]

I'd love this feature as well. It affects those of us who host the CMS backend in a private repository. Media uploaded via the CMS currently won't be publically viewable in the final builds of our static site, as the media is inaccessible in a private repository. Would love to be able to upload directly to AWS S3 instead as the media backend.

[erquhart]

@cpjolicoeur that shouldn't be a problem currently - if you're storing assets in your repo, you should be publishing those assets in your build so they're publicly accessible. Your site's image src attributes shouldn't be pointing to your repo.

_{Sent with GitHawk}

[cpjolicoeur]

@erquhart agreed. I realized this shortly after posting this comment and updated our deploy to do that. Forgot to come back here and follow-up/delete the comment however.

Thanks for the response.

[erquhart]

This will be supported by #1171. That issue will provide the necessary API(s), and this issue will bring in one or more actual extensions to integrate with specific services.

[dev7ch]

While searching about how to implement Google Drive Hosted public images into a JAM Stack i came here.

Due @haileypate already mentioned it´s a multi topic issue, i am not sure if this is the right place to ask but my case is somehow related to 3rd Images.

Currently, I am using static Nuxt (Vue) with contents based on markdown files and netlify cms to manage the md files.

Atm I see it´s possible to handle the Drive Rest API and images with vue, nuxt, axios, etc nicely but it would be super cool to have a visual admin for that in the wonderful netlify headless admin app.

Drive Rest Web implementation examples:

https://github.com/gsuitedevs/browser-samples/blob/master/drive/quickstart/index.html
https://developers.google.com/drive/api/v3/quickstart/js

Is there a way to connect to read images from Drive Rest API in admin UI and implement them into the .md files as resources?

What do you think about the amount of effort which is needed to make that happen?

[mittalyashu]

Integration with Imgur

[erquhart]

Check out this blurry gif:

This approach is based on a new API for registering an external widget library. I'm only aware of two services that have media library widgets, Cloudinary and Uploadcare. I'll likely put together integrations for both if we go this route.

Thoughts? Know of any other services with media library or upload widgets?

[erquhart]

Opened #1602 to satisfy this issue, check it out!