Tutorial for running containers as a demo or for evaluation #10273
Conversation
Differences from dev version: - localstack and minio removed - env vars filled in based on current .env The goal is to have a single file to download, rather than a compose file and an .env file.
Also update tags section under "app image" (now live).
| Development and maintenance of the `image's code <https://github.com/IQSS/dataverse/tree/develop/src/main/docker>`_ | ||
| happens there (again, by the community). Community-supported image tags are based on the two most important | ||
| upstream branches: |
There was a problem hiding this comment.
I copied this from base-image.rst and left the "by the community" stuff in there but I'd say we can remove it (both here and there). These days IQSS is helping to maintain these images.
| Intro | ||
| ----- | ||
|
|
||
| See :doc:`../dev-usage`. |
There was a problem hiding this comment.
We could consider moving dev-usage.rst to be underneath "running" but I haven't tried yet.
|
|
||
| Please be aware that for now, the "dev" persona is used to bootstrap Dataverse, which means that admin APIs are wide open (to allow developers to test them; see :ref:`securing-your-installation` for more on API blocking), the "create user" key is set to a default value, etc. You can inspect the dev person `on GitHub <https://github.com/IQSS/dataverse/blob/master/modules/container-configbaker/scripts/bootstrap/dev/init.sh>`_ (look for ``--insecure``). | ||
|
|
||
| We plan to ship a "demo" persona but it is not ready yet. See also :ref:`configbaker-personas`. |
There was a problem hiding this comment.
I've never created a persona. I hope doesn't take much time to create and document how to use it.
There was a problem hiding this comment.
TODO: link to this page from admin/metadatacustomization.rst
docker/compose/demo/compose.yml
Outdated
| dev_dataverse: | ||
| container_name: "dev_dataverse" |
There was a problem hiding this comment.
I kept the container names with dev_ prepended. Happy to change this.
| DATAVERSE_DB_HOST: postgres | ||
| DATAVERSE_DB_PASSWORD: secret | ||
| DATAVERSE_DB_USER: dataverse | ||
| DATAVERSE_FEATURE_API_BEARER_AUTH: "1" |
There was a problem hiding this comment.
I left the bearer auth stuff in just in case someone wants to demo the new frontend as well. Should I note this under the security section?
| image: gdcc/dataverse:alpha | ||
| restart: on-failure | ||
| user: payara | ||
| environment: |
There was a problem hiding this comment.
In order to only have a single file to download, I copied the values out of .env into the compose file.
| Deleting the Data Directory | ||
| +++++++++++++++++++++++++++ | ||
|
|
||
| Data related to the Dataverse containers is placed in a directory called ``docker-dev-volumes`` next to the ``compose.yml`` file. If you are finished with your demo or evaluation or you want to start fresh, simply delete this directory. |
There was a problem hiding this comment.
Should we continue to call the directory "docker-dev-volumes"? For a demo something like "dataverse-data" might make more sense.
| In the compose file, try increasing the timeout in the bootstrap container by adding something like this: | ||
|
|
||
| .. code-block:: bash | ||
|
|
||
| environment: | ||
| - TIMEOUT=10m |
There was a problem hiding this comment.
If we're worried about this, should we preemptively put this in the compose file?
There was a problem hiding this comment.
Yes, and document it prominently. I had some failed startups and it took me a while to realize that the bootstrap did not run due the short timeout.
docker/compose/demo/compose.yml
Outdated
| #volumes: | ||
| # - ./docker-dev-volumes/smtp/data:/mail |
There was a problem hiding this comment.
Hmm, I'm not sure why this is commented out. It's also commented out in the "dev" version. I've never worried about persisting this data.
| Introduction | ||
| ============ | ||
|
|
||
| Dataverse in containers! |
There was a problem hiding this comment.
We could steal my old bricks image and put it here (or a variant thereof)
| This guide is *not* about installation on technology like Docker Swarm, Kubernetes, Rancher or other | ||
| solutions to run containers in production. There is the `Dataverse on K8s project <https://k8s-docs.gdcc.io>`_ for this | ||
| purpose, as mentioned in the :doc:`/developers/containers` section of the Developer Guide. |
There was a problem hiding this comment.
Maybe we should keep a variant of this to make it clearer what is out of scope.
There was a problem hiding this comment.
Ok, so maybe clarify the relationship to k8s, rancher, swarm, etc.
| Helping with the Containerization Effort | ||
| ---------------------------------------- | ||
|
|
||
| In 2023 the Containerization Working Group started meeting regularly. All are welcome to join! We talk in #containers at https://chat.dataverse.org and have a regular video call. For details, please visit https://ct.gdcc.io |
There was a problem hiding this comment.
| In 2023 the Containerization Working Group started meeting regularly. All are welcome to join! We talk in #containers at https://chat.dataverse.org and have a regular video call. For details, please visit https://ct.gdcc.io | |
| In 2023 the Containerization Working Group started meeting regularly. All are welcome to join! We talk in #containers at https://chat.dataverse.org and have a regular video call. For details, please visit https://ct.gdcc.io. |
Also, explain how to create a persona and some basic config.
This comment has been minimized.
This comment has been minimized.
Besides those remarks/requests its a really good guide! Many thanks! |
Using --insecure at first and then doing securing APIs, etc later (like non --insecure does) seems like the best option for now. It allows us to simplify the tutorial and set up an unblock key for later use.
|
I had a great long discussion with @poikilotherm @johannes-darms @donsizemore and others about this pull request during the container meeting this morning, which was recorded. Thank you! The change I made afterwards was to use @poikilotherm I know we talked about adding There's other feedback I could or should address such as talking about |
This comment has been minimized.
This comment has been minimized.
|
Hey! I encountered a small issue while following the Deleting Data and Starting Over section detailed in the guide. After stopping the server, deleting the data directory as instructed, and rerunning This output suggests that the To resolve this, I stopped the process and restored the original data folder, ensuring only its contents were cleared. This approach rectified the error, allowing the server to launch successfully. It seems like there might be a step missing in the guide or an issue with the |
|
📦 Pushed preview images as 🚢 See on GHCR. Use by referencing with full name as printed above, mind the registry name. |
|
@Saixel I'm not sure what happened, but I'm wondering if what you ran into was simply the Docker instance failing to properly deploy when composed from scratch; not because there are any steps missing in the guide. There's enough going on on the inside that there may be some timing issues - something taking just a little too long to initialize, etc. - for the whole thing to fail once in a while. |
|
I'm happy to just merge this, unless anyone objects? |
|
@landreev it's fine with me if you merge. I threw a release not in there. Are you ok with how we're running containers in the foreground? I could document |
|
It's easier to stop when it's running in the foreground, and you get a better idea of what's going on. In other words, I don't think it's strictly necessary, anyone who may need -d for any practical purposes will likely figure it out on their own. But up to you. |
|
@landreev yeah, I feel the same way. People will figure out -d if they want that. Please feel free to merge away. |
|
@Saixel Sorry for dismissing your report - it appears that this may in fact be either a bug, or a behavior that we didn't document properly. |
What this PR does / why we need it:
People want to try running Dataverse in containers for demo or evaluation purposes. This pull requests is mostly documentation, which can be previewed at https://dataverse-guide--10273.org.readthedocs.build/en/10273/container/index.html
It also includes a stripped-down Docker compose file that's more suited to demos than development.
Please note that it currently still uses the "dev" bootstrapping persona, which means that it's just as insecure as a developer's environment with open admin APIs, etc. I wrote a section on security about this.
I also stubbed out related pages under "running Dataverse in Docker" and reorganized here and there in the guide (the Container Guide).
Which issue(s) this PR closes:
Special notes for your reviewer:
Please see above and the comments I left on specific lines under "files changed".
Suggestions on how to test this:
Try the quickstart under the demo page, at least: https://dataverse-guide--10273.org.readthedocs.build/en/10273/container/running/demo.html#quickstart
Does this PR introduce a user interface change? If mockups are available, please link/include them here:
No.
Is there a release notes update needed for this change?:
Yes, included.