demo images broken / where should we host them?

[genekogan]

all the demos are broken because i removed the MNIST/CIFAR images from the repo to make it smaller. CIFAR alone is around 120MB. would it make sense to host the images somewhere else to keep the repo smaller (and likewise with other large content)? if so, where's a reliable place to do so? if we do that we save the space but the demos can't run on localhost without an internet connection.

[frnsys]

s3 might be a good choice; they have a s3 free tier option which would hold us over for a bit:

5 GB of Amazon S3 standard storage, 20,000 Get Requests, and 2,000 Put Requests

maybe even something like dropbox could work for long-term free storage?

I'm not familiar enough with jekyll, but maybe there is some way to specify a IMAGE_HOST config option when building which we can specify as S3 or w/e as the default (for when its built on github) and manually specify as localhost when running locally?

[genekogan]

yeah, we could do a flag in jekyll. there can be a lot of these actually -- like global variables. that might be helpful. i can set that up.

is there any reason to use s3 and not just go straight to dropbox?

[frnsys]

not that I can think of!

[genekogan]

i tried setting it up on dropbox but had a lot of trouble. i created public links to the image files and would grab the full image url and it would work fine at first but would then unexpectedly give me 403 errors. they are not reliable.
s3 gives you 20k get requests, so we can try them, but do we have any other alternatives? i could perhaps host them separately on genekogan.github.io which i'm not using for anything at the moment.

[frnsys]

bummer. S3 is very cheap so I'm down for that. If it's just images though we could even use an image hosting service like imgur?

[TiborUdvari]

Don't know if this is still an issue, but there is Github Large File System or LFS. I don't know how well it can interop with GH Pages but it would eliminate an external dependency.

If not maybe go the Linux way and setup torrents? It has obvious downsides so I don't know if it's worth the trouble.

[kyeshmz]

I think there is away that we can start using Google photos with a shared album for the repo.
We would then make them all shared, and make the links for the files, here is someone who has already started doing that themselves.

https://discuss.pixls.us/t/my-static-photo-gallery-on-github/2164/14

[genekogan]

these are both good ideas. i think it's fine storing the images as part of gh-pages, as it is now, but mainly for things like datasets and models. i was also thinking google drive would work, although the only thing with those (and possible g photos as well?) is that there's no straightforward url to files so you can easily put them into scripts (like wget or curl), you have to usually pull up a browser to download them. github large file system seems like it may work as well.

[kyeshmz]

Though it seems really hacky right now, I think that using https://github.com/ddo02/GooglePhotosDirectLink with https://stackoverflow.com/questions/39508631/how-to-get-direct-permanent-link-to-public-image-on-google-photos could work.

This makes it so that the google photos album into a google archive album, which has static links.
GPhotos direct links are dynamic right now due to their development