Dockerize

[scjody]

Part of https://github.com/plotly/streambed/issues/9865

Ready for review.

[etpinard]

Right. Generating an image will hang using plotly.js latest as the renderer assumes plotly/plotly.js#1939.

Originally, I was thinking of putting all the docker stuff in streambed. I believe this repo here will one day become open source and be just an npm module.

By design, the start server script in ths repo won't do all the things we want on prod. For example, it simply prints logs to the terminal as opposed to logging them to a file.

In brief, I'm thinking streambed/image_server/ will consume this module with its own start sever script with all the logging logic and environment variables unique to streambed. I can write this script in like ~1 hour tomorrow, if you want.

If that conflicts with anything on your side @scjody, I'm happy to discuss it.

[scjody]

There isn't really much Docker stuff. The Dockerfile is actually really trivial other than the stuff I did to base the image on Ubuntu 14.04 rather than the node-xvfb image.

We could put it in streambed but then it would be harder to keep it in sync with changes in this repo, it doesn't belong with anything else in that repo, and it would make deployment more complicated. For other projects (database connector and Dash On Premise) we have the Dockerfile in the project repo, and database connector is open source.

For logging, printing to STDOUT and STDERR is fine for GKE - it collects logs itself. Ideally:

• Print one line per request to STDOUT summarizing the request and its status (similar to what nginx or apache does).
• Print errors, debugging info, etc. to STDERR.

What else do we need to do that's specific to Plotly Cloud?

[etpinard]

For logging, printing to STDOUT and STDERR is fine for GKE - it collects logs itself

Wow. That's a game changer.

What else do we need to do that's specific to Plotly Cloud?

Now that I believe image-exporter is set to be fully open source -- including PDF and EPS plotly graph exports (cc @jackparmer @bpostlethwaite ) -- this means there would be no need for private export components. So yeah, there's not much Plotly-Cloud-specific things to worry about. Adding the docker stuff to this repo is making more and more sense.

That said, we'll need to take special care of the mapbox access token. Looking at the current image_server code, nothing else comes to mind.

[scjody]

That said, we'll need to take special care of the mapbox access token.

I think GKE can pass that in as part of the container's environment. I'll look into that.

[scjody]

Mapbox access token support is done and documented in 6472479

[scjody]

@bpostlethwaite Please review

[etpinard]

These are different screen dimensions than in streambed.

Is this on purpose @scjody ?

[scjody]

Actually we currently run at 1280x1024x24 in production: https://github.com/plotly/streambed/blob/master/deployment/roles/streambed_repo/templates/xvfb_wrapper#L30

I don't think it makes a difference but I'm willing to change it to whatever you think is best.

[etpinard]

I was just curious. I have no idea if this makes a difference or not.

[scjody]

I don't think it does. @bpostlethwaite any ideas?

[bpostlethwaite]

It might cap the maximum screen size but I think Plotly.js has no troubles rendering "off screen" anyway. We used the "off screen" trick for in-browser rendering since forever.

That said, not super sure either.

[scjody]

I just tested image export on stage and it will happily create (for example) a 3000x3000 image despite the current screen size. So I really don't think it matters.

[bpostlethwaite]

thanks!

[bpostlethwaite]

Looks good - it's a lot to digest but the architecture seems sound. 💃

Side note: It's interesting that ansible yaml files are becoming dumb wrappers around local actions. Basically Kube and GCloud with Jinja2 templating and python data structures. I realize this is necessary so that these deployments fit into our current deployment framework but it portends a review at some point once the majority of services are deployed this way.