ci(docker): Build Docker images once, scan them for vulnerabilities during CI

[khvn26]

Thanks for submitting a PR! Please check the boxes below:

• I have run pre-commit to check linting
• I have added information to docs/ if required so people know about the feature!
• I have filled in the "Changes" section below?
• I have filled in the "How did you test this code" section below?
• I have used a Conventional Commit title for this Pull Request

Changes

This does a few things:

• The PR workflow now builds a number of Docker images (unified, frontend only, backend only, frontend for E2E) once per a PR event. Previously, the E2E image was being built for every dockerised E2E invocation. This should decrease our Depot costs.
• The Docker images are pushed to the GitHub registry for future reuse.
• The merge to main workflow also builds and pushes an image, currently just for E2E.
• E2E jobs are refactored to use the built API and E2E images.
• E2E jobs are now parametrised. A great deal of duplicate code is gone. Better naming for "Unified" jobs (see below).
• Unified, frontend only and backend only images are verified for vulnerabilites.

During the making of this PR, I've discovered a number of discussion points and action listed below.

Things to consider / TODOs

1. Ideally we should build all the images we ship in the merge to main workflow, and then use those for all subsequent deployments. This work will include migrating the private-cloud build steps from GH actions to its own Dockerfile. This will also allow us to build private-cloud images for specific PRs, which should improve our integration testing and allow us to detect problems earlier.
2. C̶u̶r̶r̶e̶n̶t̶ ̶E̶2̶E̶ ̶d̶i̶s̶t̶i̶n̶c̶t̶i̶o̶n̶ ̶b̶e̶t̶w̶e̶e̶n̶ ̶"̶L̶o̶c̶a̶l̶"̶ ̶a̶n̶d̶ ̶"̶U̶n̶i̶f̶i̶e̶d̶"̶ ̶m̶a̶k̶e̶s̶ ̶q̶u̶i̶t̶e̶ ̶l̶i̶t̶t̶l̶e̶ ̶s̶e̶n̶s̶e̶.̶ ̶M̶y̶ ̶i̶n̶i̶t̶i̶a̶l̶ ̶i̶m̶p̶r̶e̶s̶s̶i̶o̶n̶ ̶w̶a̶s̶ ̶w̶e̶ ̶w̶a̶n̶t̶e̶d̶ ̶t̶o̶ ̶t̶e̶s̶t̶ ̶s̶e̶p̶a̶r̶a̶t̶e̶ ̶A̶P̶I̶/̶f̶r̶o̶n̶t̶e̶n̶d̶ ̶v̶s̶ ̶a̶ ̶c̶o̶m̶b̶i̶n̶e̶d̶ ̶i̶m̶a̶g̶e̶,̶ ̶i̶.̶e̶.̶ ̶t̶h̶e̶ ̶o̶n̶e̶ ̶w̶e̶ ̶s̶h̶i̶p̶ ̶a̶s̶ ̶̶f̶l̶a̶g̶s̶m̶i̶t̶h̶/̶f̶l̶a̶g̶s̶m̶i̶t̶h̶̶.̶ ̶H̶o̶w̶e̶v̶e̶r̶,̶ ̶w̶h̶a̶t̶ ̶i̶s̶ ̶n̶o̶w̶ ̶c̶a̶l̶l̶e̶d̶ ̶"̶E̶2̶E̶ ̶U̶n̶i̶f̶i̶e̶d̶"̶ ̶a̶c̶t̶u̶a̶l̶l̶y̶ ̶r̶u̶n̶s̶ ̶s̶e̶p̶a̶r̶a̶t̶e̶ ̶A̶P̶I̶ ̶a̶n̶d̶ ̶f̶r̶o̶n̶t̶e̶n̶d̶ ̶s̶e̶r̶v̶i̶c̶e̶s̶ ̶s̶t̶i̶l̶l̶.̶ ̶C̶u̶r̶r̶e̶n̶t̶l̶y̶,̶ ̶t̶h̶e̶ ̶o̶n̶l̶y̶ ̶d̶i̶f̶f̶e̶r̶e̶n̶c̶e̶ ̶b̶e̶t̶w̶e̶e̶n̶ ̶'̶l̶o̶c̶a̶l̶'̶ ̶a̶n̶d̶ ̶'̶u̶n̶i̶f̶i̶e̶d̶'̶ ̶i̶s̶ ̶w̶h̶e̶t̶h̶e̶r̶ ̶f̶r̶o̶n̶t̶e̶n̶d̶ ̶i̶s̶ ̶b̶e̶i̶n̶g̶ ̶r̶u̶n̶ ̶i̶n̶ ̶D̶o̶c̶k̶e̶r̶.̶ ̶@̶m̶a̶t̶t̶h̶e̶w̶e̶l̶w̶e̶l̶l̶ ̶@̶n̶o̶v̶a̶k̶z̶a̶b̶a̶l̶l̶a̶ ̶W̶e̶ ̶n̶e̶e̶d̶ ̶a̶ ̶d̶e̶c̶i̶s̶i̶o̶n̶ ̶o̶n̶ ̶t̶h̶i̶s̶.̶ This is now resolved in favour of Docker-based E2E runs.
3. I don't like that frontend/Dockerfile.e2e build mostly duplicates frontend/Dockerfile one. I'd like to try a multi-stage Docker build for E2E, i.e. add a FROM ghcr.io/flagsmith/flagsmith-frontend:${FRONTEND_VERSION} as frontend stage and pull whatever we need on top of the base E2E image. I couldn't tackle this on my own so tagging @kyle-ssg for advice on this.

How did you test this code?

This is a CI change.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 29, 2024 4:22pm
flagsmith-frontend-preview	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 29, 2024 4:22pm
flagsmith-frontend-staging	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 29, 2024 4:22pm

[github-actions]

Uffizzi Preview deployment-52093 was deleted.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.42%. Comparing base (eef02fb) to head (7a654a5).
Report is 23 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3997      +/-   ##
==========================================
+ Coverage   96.22%   96.42%   +0.19%     
==========================================
  Files        1142     1146       +4     
  Lines       36760    37386     +626     
==========================================
+ Hits        35371    36048     +677     
+ Misses       1389     1338      -51     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[matthewelwell]

Added some minor comments but overall looks great 👌

[matthewelwell]

I think it would be good to provide more information here. What format is this expected to be in?

[khvn26]

Example value added.

[matthewelwell]

Suggested change

	slug:
	image-slug:

[matthewelwell]

I've also never heard the term slug used for an image. Why not use e.g. image-name here?

[khvn26]

Renamed to image-name.

[matthewelwell]

Current E2E distinction between "Local" and "Unified" makes quite little sense. My initial impression was we wanted to test separate API/frontend vs a combined image, i.e. the one we ship as flagsmith/flagsmith. However, what is now called "E2E Unified" actually runs separate API and frontend services still. Currently, the only difference between 'local' and 'unified' is whether frontend is being run in Docker. @matthewelwell @novakzaballa We need a decision on this.

Yes, this doesn't make an awful lot of sense. I think we can just consolidate this to only test via docker since this is the product that we provide to our customers.