-
Notifications
You must be signed in to change notification settings - Fork 31
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Build fully featured Docker container #56
Comments
Pushed first draft of Docker image. The image can run Stata and has a conda environment with dependencies installed. To run this, a new user may have to make a few tweaks to their Docker configuration settings. Namely
In its current state, this container has much of what we need. Next steps:
|
I managed to replicate @szahedian last results. I effectively needed to increase Docker's allowable memory usage to 10 GB and allow access to Some comments on the next steps:
Proposed next steps:
|
@snairdesai and I made progress on this issue. Following #59, we updated the Dockerfile to support both Python and R dependencies. Similar to what was reported in the local MacOS case, the computation times improved considerably (the environment was created in roughly 6 minutes when building the Docker container). We also added the installation of git-lfs in its respective layer in the Dockerfile. I opened PR #60 to bring these changes to the development branch (@snairdesai and I are not collaborators in Next steps:
|
Thanks! Next steps sound good. I added @jc-cisneros and @snairdesai as collaborators. |
#56 PR Merge changes from jccisneros/template fork to issue56-full_docker branch in gentzkow/template
Noting that @jc-cisneros and I met with the Hunt RAs this morning to discuss Docker integration into the standard workflow. They are now getting familiar with the We are also in the final stages of debugging Docker usage for WindowsOS, and hope to have an updated proposal for these users shortly. |
Container is now run as a non-root user.
I am happy to announce that the container runs successfully on both MacOS and WindowsOS. I will describe the latest updates and list the next steps. Updates:
Next steps:
|
@jc-cisneros Thanks! This is exciting. A question I'd be curious to get your thoughts on as we think about whether we want to use Docker as our default dev environment vs. only for releasing replication archives etc.: To what extent would you expect that this will add another layer of failure points / debugging? Something we've run into again and again is that changes that are intended to make life easier can also create lots of new problems. The business with solving Conda environments is a good example of this. Problems are especially costly if only a small share of users know how to debug them. So I'm wondering how often we're going to see cases where some RA somewhere is working away and gets a Docker related error and then we have a very hard time figuring out how to debug that. |
(i) We think We also think that regardless of whether we decide to use We anticipate that within this development pipeline, building images in The process of building images does have a learning curve, and is not straightforward. We hope that the skeleton we have built in We've been considering how to step through these issues. Here's what we are planning: When it comes to running containers from built images (relevant for public release):
When it comes to building images (relevant for internal lab development):
Happy to add/remove any steps from this testing pipeline if helpful. |
Excellent. Thanks!
… Message ID: ***@***.***>
|
Update: Today we verbally described the purpose/features of Docker to a Heidi RA in the lab, and had them run the |
Thanks. Can you give me an idea of how the template could take 7 minutes to run? I would have thought all the scripts should be basically instantaneous. |
Apologies, that message was unclear - we are including the time it takes from a first-time user to sit down with the Docker instructions up until the time when they successfully run The benefit of Docker is that individuals do not have to download any of our standard application requirements (i.e., When using Docker, We just timed how long it takes for users who have already built the image once. The entire process (starting from cloning |
Got it. Thanks! Is the ~30 seconds for |
It's roughly the same as the time it takes in the standard |
@gentzkow: I'm closing out this issue for the sake of housekeeping. @jc-cisneros and I do hope to revisit |
Summary + DeliverablesIn this issue (#56) @szahedian @jc-cisneros and I experimented with A summary comment of the issue's original purpose can be found here. This The commits in this branch (
|
Following #54 (comment), in this issue we will
The main improvement over #43 is the addition of Stata, using ideas from AEA Stata for Docker.
The text was updated successfully, but these errors were encountered: