Support container-based workflow with no precompilation/update required after building image

[jlperla]

This is an addendum to #90. I think an important use-case for thinking about the precompilation when using containers for reproducible research and CI. Maybe the design of the package and precompilation system already support this use-case.

In particular,

• I would love to be able to create a docker image for my department which has precompiled versions of all of the main libraries required by economists.
• Then, we could write easy scripts for users to run custom scripts for individual projects which use that parent image, get the sourcecode for the computation, etc.
• The key would be that in the process, no precompilation outside of the users own code would be required.

[stugorf]

@jlperla Have you had success using a Project.toml file in a Docker container. I am having this issue: https://stackoverflow.com/q/53548263/10442044

[jlperla]

@arnavs May be able to point you towards some of what we did with the QuantEcon docker image.

[arnavs]

@stugorf You may have seen the SO, but the issue there is that each RUN command is being executed separately. So activate context from one julia -e line is not used in the next.

It's also good practice to try and combine these things, because each Docker line translates to a distinct layer, which may be overkill.

We have a Docker image with a few different TOML environments here. Would be happy to walk you through it if you think it'd be helpful.

[KristofferC]

Problem seems to have been solved in that stack overflow question. (thx @fredrikekre)

[stugorf]

Thank you Arnav. I'll look at the Docker image you provided the link to! Regards,

…

--David

On Fri, Nov 30, 2018 at 9:56 AM Arnav Sood ***@***.***> wrote: @stugorf <https://github.com/stugorf> You may have seen the SO, but the issue there is that each RUN command is being executed separately. So activate context from one julia -e line is not used in the next. It's also good practice to try and combine these things, because each Docker line translates to a distinct layer, which may be overkill. We have a Docker image with a few different TOML environments here <https://github.com/arnavs/docker-stacks/blob/julia-after-resize/pims-julia>. Would be happy to walk you through it if you think it'd be helpful. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#92 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFQ5eeDDFMKxsD9pjwZ0EniceUociwWqks5u0XE7gaJpZM4ROxCF> .

[IanButterworth]

Maybe this could be closed given the development of https://github.com/JuliaLang/PackageCompiler.jl

But perhaps it's more about handling that process more automatically through Pkg, so further work to be done?