Specify your own filename for a screenshot

[rufuspollock]

Allow users to specify a short name to save their screenshot at (a bit like bit.ly but for screenshots). E.g. you could do ?filename=... and then you could get that screenshot forever at:

webshot-service/f/{filename}.png

[rossjones]

Almost certainly going to clash, might be easier to somehow specify you want it cached and then get given a URL that is a base 62 encoded int of some description, or a UUID or something?

[rufuspollock]

Hmm, but isn't this just like bit.ly - sure there can be clashes but that's life ...

[rossjones]

I think I'd rather have a URL that had a UUID in it than use it on a site and have someone 'accidentally' overwrite it with a picture of kittens.

[rufuspollock]

@rossjones you wouldn't be able to overwrite - like bit.ly this would be write once ...

@simong what do you reckon? how easy is this to implement do you think?

[simong]

It could work on a first-come first-served basis, similar to dns names?

We'd need to work out how to store the actual files and stick a small database behind the app. I don't think it'll be terribly hard. We just need to figure out a way to easily do this on both okfn as dev. I'll try to come up with a prototype in the next week(s).

[rufuspollock]

@simong agree re first-come, first-served (let's do the simplest thing possible). I suggest we use s3 as the backend for the present (no need for a proper database) plus we need to store images ...

[simong]

Example:
url: https://openspending.org
name: openspending.jpg

The file would be accessible at
https://webshot.okfnlabs.org/f/openspending.jpg

which would actually be stored at:

    var hash = md5('openspending.org'); // ex: f1ab4edaec312de12c...
    S3 Url: https://s3.amazon.com/okfn-webshot/f1/ab/4e/da/openspending.org

(The hashing is in place so we don't end up with thousands of files in a single folder)
If the file doesn't exist S3 will return a 404, for which we can setup a custom 404 page.

If we would ever be worried about taking up too much space we could make use of S3 their object expiration [1].

WDYT?

[1] http://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectExpiration.html

[rufuspollock]

@simong sorry to miss this at the time. Proposal looks really good. If you need s3 space we can get s3 bucket setup for you.

Small suggestions:

• we do need to check for conflicts (i.e. we should not be allowed to overwrite an existing filename).

• suggest we have an app config option to set the bucket and config option for any root path (so we could locate data in s3 bucket at

  s3.amazon.com/okfn-webshot/my/base/path/...

Also implementation detail: suggest

GET webshot.okfnlabs.org/f/openspending.jpg

redirect to s3 bucket rather than proxying to save on b/w and hassle

Would be awesome to get this implemented!

[rufuspollock]

@simong I was just using webshot again (I use it a lot!) and really missed this feature. I think this would be super useful to have.

Following my last comment my only thoughts were:

• Is it worth doing the hash - s3 does not have a problem with files in a "directory" (since it does not have directories)
• Should we consider putting in support for users (even if we don't have login yet) e.g. if username were set files would be stored at `/{username}/{filename}
• Last option would be support for allowing '/' in specified file name (so you could fake subdirectories that way)

[rufuspollock]

@simong any time to work further on this - I use this service all the time and I would love to have this ability to save feature.

I'm also wondering if we really care about namespace issues about allowing users to login with e.g. github - it is so easy to implement (but that may be overkill!)

[simong]

Hi all, I finally had some time available to move this ahead. Currently my s3-store branch has the following:

• There's an /api/store endpoint that allows you to make a POST request with all the same parameters as the /api/generate endpoint with the addition of a name parameter. That parameter allows you to retrieve the image later at /f/<name>.
• When calling /api/store we check S3 for conflicts. If the name is already used, we respond with a 400.
• Images are stored in S3, credentials (aws access key, secret key, bucket, region and directory prefix) can be defined through process environment variables. This should work nicely with Heroku
• /f/<name> redirect to a signed S3 URL, so the app isn't serving anything

Some questions:

• Should we keep this at a separate endpoint or just integrate it in the existing one?
• Depending on the above, any suggestions for the UI
• Who do I talk to to get some S3 credentials so I can deploy it? :)

[pietercolpaert]

@simong some HTTP nitpicking: if you let your client choose the name, you should use PUT. If your server can choose the name, you should use POST.

[rufuspollock]

@simong first this is fantastic - great work :-) Thoughts

• Naming: 2 options. We may want to do one, both or neither of these. However, it would massively reduce conflicts which is nice. I sort of like the date idea because you'd automatically get a sense of age (though you might also get that s3 headers)
  • What about prefixing with the date yyyymmdd format - this would much reduce conflicts on naming.
  • We could also prefix with a random 3-4 digits from [azA-Z-_0-9] - that would also massively reduce collisions.
• Really nice on the design
  • I suggest we just integrate with existing one and just add a "name" parameter
  • Re UI i think we just want to add an "Save this image" checkbox which then shows a name field
  • S3 credentials - i will get you those with assistance of @danfowler

---

@pietercolpaert personally I'm not too zealous on PUT vs POST - i know that if we are being nicely RESTful we would do it but happy not to be perfect ;-)

[simong]

It feels like you'd lose some of the value of providing a custom name if we're prefixing timestamps/random strings? Each url would look like /f/mything-Xfe or /f/2015/09/03/mything. It isn't something you'd be able to easily remember so you'd have to copy/paste it everywhere. Maybe that's OK though?

[rufuspollock]

@simong i think people will usually need to copy and paste anyway but what's nice is you can just see the url and know what the thing is which is really nice. I'm thinking looking at it that the date one is probably the nicer one.

[rufuspollock]

@simong any update here?

/cc @danfowler something for the next newsletter and generally to advertise!

[danfowler]

@simong @rgrp Reserving a place for this in the upcoming newsletter 😄

[danfowler]

@simong do you need anything from me?

[rufuspollock]

@simong any update here - this would be so awesome.

Couple of final thoughts:

• I think prefixing with the yyyy/mm/dd is nice and allows us to track usage (?)
• can we make sure we add .png to the end of file names