Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Helm charts for Hyrax and FCRepo #4496

Merged
merged 11 commits into from
Oct 30, 2020
Merged

Helm charts for Hyrax and FCRepo #4496

merged 11 commits into from
Oct 30, 2020

Conversation

no-reply
Copy link
Contributor

@no-reply no-reply commented Sep 3, 2020

http://dassie.dld.library.ucsb.edu is deployed from this chart.

This Helm chart provides configurable deployments for Hyrax applications
to Kubernetes clusters. It seeks to be a complete but flexible
production-ready setup for Hyrax applications. By default it deploys:

  • A Hyrax-based Rails application
  • Fedora Commons v4.7
  • Postgresql
  • Solr (in a cloud configuration, including Apache Zookeeper)
  • Redis

A base Hyrax deployment

Because Hyrax is a Rails Engine---not a stand-alone application---
deploying it requires us to have a specific application. This chart assumes that
the user has a container image based on samveralabs/hyrax (see:
CONTAINERS.md) that includes their application. Point the chart at
your image by setting the image.repository and image.tag values.

By default, the chart deploys [images][dassie-images] for Hyrax's development
application, dassie.

For application configuration, we take our queues from [12-factor][twelve]
methodology. Applications using environment variables to manage their
configuration can be easily reconfigured across different releases using this
chart; e.g. the same chart can be used to deploy sandbox, staging, and
production environments.

The chart populates the following environment variables:

|-------------------|--------------------------------|------------------------|

Variable Description Condition
DB_HOST Postgresql hostname postgresql.enabled
DB_PORT Postgresql service port postgresql.enabled
MEMCACHED_HOST Memcached host memcached.enabled
RACK_ENV app environment ('production') n/a
RAILS_ENV app environment ('production') n/a
REDIS_HOST Redis service host redis.enabled
FCREPO_HOST Fedora Commons host fcrepo.enabled
FCREPO_PORT Fedora Commons port fcrepo.enabled
FCREPO_REST_PATH Fedora Commons REST endpoint fcrepo.enabled
SOLR_HOST Solr service host solr.enabled
SOLR_PORT Solr service port solr.enabled
SOLR_URL Solr service full URL solr.enabled
----------------- - -------------------------------- ------------------------

For DevOps:

For those interested in trying out or contributing to this Chart, it's helpful
to setup a simple cluster locally. Various projects exist to make this easy; we
recommend k3d or minikube.

For example, with k3d:

k3d cluster create dev-cluster --api-port 6550 -p 80:80@loadbalancer --agents 3

@samvera/hyrax-code-reviewers

@dunn
Copy link
Contributor

dunn commented Sep 3, 2020

at some point i imagine we'll want a separate charts repo for shared components like fcrepo (and possibly solr helm/charts#22863 (comment)); in that case would we want to move application charts like hyrax as well? how hard would that be?

@orangewolf
Copy link
Member

at some point i imagine we'll want a separate charts repo for shared components like fcrepo (and possibly solr helm/charts#22863 (comment)); in that case would we want to move application charts like hyrax as well? how hard would that be?

👍 for moving the fcrepo out, but I think having the chart for the application live with the code is a very devops way of handling it. That way simple things like "where is this value used" pick up the infrastructure as code along with the application code.

@no-reply no-reply force-pushed the cn-helm branch 2 times, most recently from b5a3e29 to cd3cd76 Compare September 3, 2020 22:20
@no-reply
Copy link
Contributor Author

no-reply commented Sep 3, 2020

💯 x 👍 on extracting the fcrepo chart into some community charts repository.

i mentioned to @orangewolf that my default mode has become putting everything in the same version control repository (owing to positive experiences with the Surfliner monorepo).

having lock-step updates across charts has been important up to this point, but maybe it's already a good time to look at extracting?

@orangewolf
Copy link
Member

@no-reply you know I want to see it separate and published, but I'm wondering if this should go in as is and extract as a next action, just so this work isn't held up at all.

@no-reply no-reply changed the title Cn helm Helm charts for Hyrax and FCRepo Sep 4, 2020
@no-reply no-reply marked this pull request as ready for review September 4, 2020 00:27
@no-reply
Copy link
Contributor Author

no-reply commented Sep 4, 2020

i'm finding that the fedora db setup init container isn't as reliable as i'd hope.

sometimes it complains that local user with ID 1001 does not exist, and refuses to start fedora. i've tried reproducing by changing the script to sleep 120, shelling in and running manually, but can't get the negative behavior that way.

it might be worth considering whether a smaller container than bitnami/postgresql is called for for that purpose, anyway?

@scottrigby
Copy link

Hello! 👋

If you want to extract it, here are some steps to help (we'll be publishing a post on this soon, but in the meantime): prometheus-community/helm-charts#3 (in an example from another migrated helm repo, @torstenwalter has combined those steps with a checklist for all the other steps needed in this nice summary issue: grafana/helm-charts#3).

You can see this working demo of Helm GitHub Actions for CI (chart testing), and CD (for releasing chart version packages). The README has detailed steps to emulate them in your own Helm repo: https://github.com/helm/charts-repo-actions-demo

Also you'll want to add your helm repo to the CNCF Artifact Hub (the successor to Helm Hub) for discoverability 😄

Hope that helps!

@no-reply
Copy link
Contributor Author

no-reply commented Sep 4, 2020

@scottrigby thanks for this input! it's very helpful indeed!

db/seeds.rb Show resolved Hide resolved
no-reply pushed a commit that referenced this pull request Oct 29, 2020
ports the parts of #4496 that relate to container build. installs UV via rails
asset precompile.

this makes `.dassie` pass the one test that depends on UV's presence. it's also
needed generally for UV in the docker development and test environments.
no-reply pushed a commit that referenced this pull request Oct 29, 2020
ports the parts of #4496 that relate to container build. installs UV via rails
asset precompile.

this makes `.dassie` pass the one test that depends on UV's presence. it's also
needed generally for UV in the docker development and test environments.
no-reply pushed a commit that referenced this pull request Oct 29, 2020
ports the parts of #4496 that relate to container build. installs UV via rails
asset precompile.

this makes `.dassie` pass the one test that depends on UV's presence. it's also
needed generally for UV in the docker development and test environments.
jeremyf pushed a commit that referenced this pull request Oct 29, 2020
ports the parts of #4496 that relate to container build. installs UV via rails
asset precompile.

this makes `.dassie` pass the one test that depends on UV's presence. it's also
needed generally for UV in the docker development and test environments.
tamsin johnson added 9 commits October 29, 2020 10:11
ran `mkdir chart; cd chart; helm new hyrax` with:

```sh
$ helm version
version.BuildInfo{Version:"v3.2.4", GitCommit:"0ad800ef43d3b826f31a5ad8dfbb4fe05d143688", GitTreeState:"clean", GoVersion:"go1.13.12"}
```

this chart is being developed to allow deployment of the Hyrax application using
containers based on the `hyrax` image (see: `Dockerfile`).
the description text is taken from hyrax.samvera.org.
move this configuration from the `.env` to the container image---it's the
container image that this setting is specific to, not the engine or its
`docker-compose`.
switch postgresql.enabled to `true` by default and configure the database
connection. in the current state, the Hyrax rails app pod hangs waiting for
fcrepo to connect; we need a sub-chart for fedora.
we need a sub-chart for fcrepo, we're hoping to extract this to samvera labs
before too long, but getting it in here will help us keep it stable.

Co-authored-by: @orangewolf
support servicePort configuration for fcrepo

Fedora won't let us use port 80 explictly: we either need to use another service
port (e.g. 8080, as here), or configure the application to use no port at all.

since the second option involves changing application code and configuration,
using 8080 seems easier.
adds values.yaml files for two actual hyrax/dassie deployments;

makes some small changes to image generation to ensure precompiled assets work
correctly in image building and production.

bump the chart versions.
Copy link
Member

@orangewolf orangewolf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went end to end on deploying this both locally with k3d and to our k8s cloud. It works slick!

command:
- sh
- -c
- export PGPASSWORD=$DATABASE_PASSWORD PGHOST=$DATABASE_HOST PGUSER=$DATABASE_USER
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can these be set in an environment block below?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably. i'm going to suggest we merge and tweak from there.

this is now deploying successfully and i'm keen on being able to work both on improving the chart and on getting some fashion of CD in place. nurax's current issues are worrisome since only a few people can work on its infra.

@no-reply no-reply merged commit eb3a909 into master Oct 30, 2020
@no-reply no-reply deleted the cn-helm branch October 30, 2020 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants