[stable/anchore-engine] dependency updates

[ts-mini]

What this PR does / why we need it:

The current versions are VERY far behind and this allows more customization of the postgresql and redis subcharts for production level configuration

Special notes for your reviewer:

Checklist

[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

• DCO signed
• Chart Version bumped
• Variables are documented in the README.md
• Title of the PR starts with chart name (e.g. [stable/mychartname])

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ts-mini
To complete the pull request process, please assign btodhunter
You can assign the PR to them by writing /assign @btodhunter in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• stable/anchore-engine/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

Hi @ts-mini. Thanks for your PR.

I'm waiting for a helm member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[zanhsieh]

/ok-to-test

[zanhsieh]

@ts-mini

I1107 07:03:52.378] 2019-11-07 06:54:15.402 GMT [218] FATAL:  password authentication failed for user "anchoreengine"
I1107 07:03:52.378] 2019-11-07 06:54:15.402 GMT [218] DETAIL:  Role "anchoreengine" does not exist.
I1107 07:03:52.379] 	Connection matched pg_hba.conf line 1: "host     all             all             0.0.0.0/0               md5"

[zanhsieh]

/retest

[Btodhunter]

@ts-mini Thanks for the contribution! The update to the Redis dependency shouldn't be an issue.

However, upgrading the postgresql chart creates a challenge for existing users. Due to the huge changes between v1.0.0 & v2.0.0 of that chart, users would be required to do a pgdump/restore to upgrade - check out the postgresql chart readme for upgrading. This is something we've been wanting to do for awhile, but have not yet determined a great solution on how to cleanly handle upgrades for existing users.

Do you have any suggestions?

[Btodhunter]

/assign btodhunter

[zhill]

At the least a change to the db layer like this will require a major version bump in the anchore chart version, since it is no longer backwards compatible. Is there are way to use the new postgres chart but override the image to a pg9.6+ image for existing users so there is an upgrade path without a pgdump/restore?

[ts-mini]

I'd agree that it warrants a major version bump of the chart. I've bumped it to 2.0.0.
Furthermore, I've successfully tested the above changes with 9.6 in the values, we could update the default values.yaml with:

postgresql:
  image:
    tag: 9.6

and

anchore-feeds-db:
  image:
    tag: 9.6

@zhill thoughts? (this is in this PR)

[Btodhunter]

@ts-mini This works great for a new deployment, thanks! Unfortunately, during my testing I noticed that upgrading the chart from v1.3.6 to v2.0.0 just blows away the existing database. Upgrades still require the pg_dump/pg_restore process outlined in the postgresql chart readme.

I think we need to thoroughly document the upgrade procedure in our README or come up with a solution that handles database upgrades automatically for the user. I'm currently trying to come up with a solution to automate the upgrade process. I'll keep this thread updated with any progress I make.

@zhill thoughts?

[zhill]

@Btodhunter was that result what you saw when using the 9.6 image or the default pg10 image? There may be a way to manage the transition carefully. Given the chart would be a major version bump, I think it's acceptable to require users to either update their values.yaml to use the pg9.6 version to avoid the db update or else do a db upgrade. Major versions are expected to be breaking so as long as properly messaged, I think its okay. Ideally, I'd like to ensure that if a user does a 1.3.x -> 2.0.0 upgrade that doesn't have the 9.6 tag that it would fail the upgrade and allow rollback rather than blowing away the db. Maybe with a pre-upgrade hook?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

[stale]

This issue is being automatically closed due to inactivity.

[cityofships]

This looks like an important piece of work which should not be abandoned. Currently the dependencies won't deploy on K8s v1.16+ due to API deprecation - similar issue as described in helm/helm#6969.