21.04 TrueCharts2.0 (Breaking Changes) #216

Ornias1993 · 2021-03-03T15:58:49Z

Description

This PR contains a multitude of breaking changes for next release of TrueNAS SCALE.

It adds a lot of documentation, both developer and user side
It completely refactors Services (and thus also removes a few bugs in its path)
It completely refactors Ingress (and thus also removes a few bugs in its path)
It removes Cert-Manager and fully moves to IX-Certificates
Add significant amounts of additional testing
Adds auto-updater CI support
Adds the option to define the user by securityContext
Updates the Readme to include featured projects we support

Fixes #

Type of change

Feature/App addition
Bugfix
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Refactor of current code

How Has This Been Tested?

Each PR has been both manually and automatically tested before being merged into this one.

Goal of this PR is final and complete review + additional manualt(!) testing by all codeowners.

All code owners need to test if the following features work and do manual code review.

I've made a short checklist codeowners and testers can use to verify all functionality works as planned:
General Checks

Check storage works with ix_volumes (hostPath disabled)
Check if default storage options work with hostPath
Check additional (list with + on it) storage works
Check storage gets the right permissions
Check if you can change the PUID/PGID and those change storage accordingly

Install with with Nodeport

Check if Portal Button works
Check if NodePort connection works at all (with or without portal)
Check if other ports work with NodePort (if present)

Install with Ingress (+traefik) and ClusterIP

Check if Portal Button works
Check if manually connecting using Ingress (domain name) works for UI (with or without portal)
Check using Ingress for other all services supporting Ingress

Notes:

Checklist:

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests to this description that prove my fix is effective or that my feature works

Apps verified/non-root/own-container to be working correctly:
non-root: ☑️
own-container-non-root: ✅

Stable:

Incubator:

* Refactor All Services - services (dict, including main service) - additionalServices (list) * Add Documentation and standardised questions.yaml layout for services * Update all existing Apps to 2.0.0 * Fix whiteline error * fix addons

* Refactor Ingress and Add Tests * New HTTP Ingress UI Layout * Cleanup plain TCP and all UDP ingress UI's * Bulk Traefik Changes: - update traefik - move storage to common chart - Remove reflector - Remove Cert-Manager - Add http IngressRoute to common - remove ingress(Route) from traefik - Cleanup Ingress(Route) to be includable in other charts * Update common included with all Apps * Update traefik and addserviceName, servicePort and serviceKind to questions.yaml * Polish: Auto find port and servicename for ingress, allow override * Add basic documentation and examples * Cleanup * small bugfix * fix mini mistake in qbittorrent * Use default torrent port within qbittorrent

* Refactor questions.yaml, ix_values.yaml and values.yaml * Add Docs

* Fix FAQ link, add issue tracker link (#226) The previous faq url points to a 404 page. Also added a link to the Github issue tracker. * Simple Emby chart based on the existing Jellyfin chart * Set current UID and GID environment variables * Reset the version to match the other charts * Clean up incorrect version * Fix versions and maintainer in Chart.yaml# Please enter the commit message for your changes. Lines starting * Switch to the linuxserver.io Docker image -- it's more closely matched to other Truecharts * Add end of file \n * Add back upstream_version (commented out) * Update CODEOWNERS * Update Emby for Truecharts 2.0.0 * Update charts/emby/2.0.0/ix_values.yaml Co-authored-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl> * Update charts/emby/2.0.0/test_values.yaml Co-authored-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl> * Delete Chart.lock Co-authored-by: Sebastien Dupont <dupont.sebastien@gmail.com> Co-authored-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl>

* move jackett to k8s-container * move sonarr to k8s container * move radarr to k8s container * move qbittorrent to k8s container * move tautulli to k8s container * move sabnzbd to k8s container * nzbget * lidarr * bazarr * nzbhydra * jellyfin * cleanup gpu mounting on common * update common * add gpu selector to jellyfin * add GPU selector to Emby * enable securityContext.runAsNonRoot

@stavros-k

* Setup dynamic portal config map. - Includes tests * Add portal docs * Update charts/calibre-web/2.0.0/questions.yaml Co-authored-by: Troy Prelog <35702532+tprelog@users.noreply.github.com> * Fix feedback from @stavros-k * Update charts/jackett/2.0.0/test_values.yaml * Update charts/jackett/2.0.0/test_values.yaml * Update charts/zwavejs2mqtt/2.0.0/questions.yaml Co-authored-by: Troy Prelog <35702532+tprelog@users.noreply.github.com> * Update charts/home-assistant/2.0.0/questions.yaml Co-authored-by: Troy Prelog <35702532+tprelog@users.noreply.github.com> * Update charts/esphome/2.0.0/questions.yaml Co-authored-by: Troy Prelog <35702532+tprelog@users.noreply.github.com> * Update charts/handbrake/2.0.0/questions.yaml Co-authored-by: Stavros Kois <47820033+stavros-k@users.noreply.github.com> Co-authored-by: Troy Prelog <35702532+tprelog@users.noreply.github.com> Co-authored-by: Stavros Kois <47820033+stavros-k@users.noreply.github.com>

… it.

* Setup externalService processing and enhance Traefik * Simplify Ingress * Simplify External Services * Adjust Service and Portal settings accordingly * Fix some newly introduced bugs * Adapt docs

Includes mix fixes like: - Fixing the documentation - Making sure device and storage mounts work correctly - Fixing broken things that where supposed to work according to the PR highlights - Moving some additional things to TrueCharts based containers - Hiding, changing and reordening some bad UI elements - Adding support for supplementalGroup permissions Co-authored-by: Dan Sheridan <dan.sheridan@postman.org.uk> Co-authored-by: Stavros Kois <47820033+stavros-k@users.noreply.github.com> Co-authored-by: Justin Clift <justin@postgresql.org>

Ornias1993 · 2021-03-12T16:54:44Z

Staging Branch is now codefrozen. This means that only bugfixes (and no new feature) will be allowed into Staging.

All Codeowners, Please review this PR.
At a minimum:

Review all functions (Portal, Ingress, Service, Nodeport, Storage etc) of the Apps you maintain.

Preferably:

Review as much as you can.

Please take your time, a lot of things changed!

Please specify when sending an "Approval" review, which parts you reviewed.
If you want to request a lot of changes, please send a PR to the "Staging" branch... This makes it easier to review your requested changes as a whole.

For those wondering: Documentation is also updated and available in the /docs folder.

Tagged maintainers:
@stavros-k @tprelog @whiskerz007 @djs52

Ornias1993 · 2021-04-02T19:33:13Z

truenas/webui#5269 is merged.
Hence you can now test TrueChart Staging like normal again on the next nightly release! :)
(no need for the docker trick above)

stavros-k · 2021-04-02T21:19:51Z

Hey, maybe change emby icon to https://emby.media/resources/logowhite_1881.png because right now is broken.
It's directly form their website.

No point making whole PR for this :)

Ornias1993 · 2021-04-02T22:19:13Z

@stavros-k Please ignore icons till launch, some have to be pointed towards master and don't exist there (yet)...

It's also preferable to use logo's without names.
But feel free to discuss it with the emby maintainer @djs52 ;-)

stavros-k · 2021-04-04T13:50:39Z

Hey, if i select any certs (either in reverse proxy section or in the external services) in traefik install, i get the following error,
Without certs installs fine.

Error: [EFAULT] Failed to install chart release: Error: failed to parse /tmp/tmpd9j9phz0: error converting YAML to JSON: yaml: invalid map key: []interface {}{"directory", interface {}(nil)}

Ornias1993 · 2021-04-04T14:14:21Z

@stavros-k THanks for your report! :)

Though next time: Please start filing normal bugreports. Because we are in codefreeze so I do not actually plan to add any more bugfixes to this PR itself. (thats also why I deleted your additional screenshots, because they just get lost in the discussions on this PR)

I've fixed one of the two bugs in certificates I found.

Working on the other one!

Ornias1993 · 2021-04-04T14:44:38Z

@stavros-k looking into it with iX, it's an upstream issue it seems. I expect this to be fixed before 21.04 release :)

Ornias1993 · 2021-04-05T10:39:44Z

@stavros-k

edit
The issue should be fixed on next Nightly being released :)

Ornias1993 · 2021-04-05T15:00:39Z

I've made the hard decision to move all Apps that don't have an active maintainer, to the "incubator" train.

As we can not guarantee maintenance without a maintainer, plus technically they where added as literally an incubator for the project by me to begin with :)

stavros-k · 2021-04-05T15:14:52Z

I've made the hard decision to move all Apps that don't have an active maintainer, to the "incubator" train.

Seems fair, hopefully with the release of scale, more people will hope in and then we will see more maintainers!

Ornias1993 · 2021-04-05T15:18:07Z

I've made the hard decision to move all Apps that don't have an active maintainer, to the "incubator" train.

Seems fair, hopefully with the release of scale, more people will hope in and then we will see more maintainers!

Indeed, it also helps me "let go" of apps I myself don't use...
I came to the conclusion while working on getting some setup as non-root. When it struck me "I don't use this and this is taking my time from working on core-issues and bug".

Ornias1993 · 2021-04-06T19:34:10Z

Some minor TODO's:

Make sure the scripts can work with "incubator" charts
Add Incubator and it's description to the docs
Fix Traefik portal button not always displaying
Fix (mage) links still pointing towards "charts" instead of "stable" or "incubator" respectively
Add/Move images to docs/website instead of in-chart.

Ornias1993 · 2021-04-06T21:04:40Z

@tprelog WHAT THE HECK happened to homeassistant?
Did you actually test it, because it could definately impossibly have worked previously.

I've pushed some cleanup, but can you please completely reverify all your apps again?

edit:
Actually I know what happened, some weirdness when copy-pasting and eding multiple charts.
But that was way before the testing phase, so I don't get how this was tested, but it definately couldn't have worked...

edit2:

Thanks to the activity from @grover31 I noticed and right away pushed some fixed to Home-Assistant.
I verified it at least starts now :)

Ornias1993 · 2021-04-07T10:57:04Z

remaining TODO's:

Full tests of his charts by @tprelog
Fix Traefik portal button not always displaying (correctly)

Ornias1993 · 2021-04-09T08:43:36Z

i've readded Deluge, because while it isn't that well updated...
It's fine as an incubator App and it was already finished anyway!

Ornias1993 · 2021-04-09T10:38:33Z

I've created a special legacy_21.02ALPHA branch containing the last master supporting 21.02ALPHA.

Hence, we are going to merge this into master before the release of 21.04ALPHA. Possibly today or tomorrow.
This way we can ensure all CI and development can be polished before actual launch of 21.04ALPHA.

It shouldn't instantly break people's existing installed Apps and they should still be able to access the old code using the special legacy branch.

- Fix traefik Portal Button - Touchup Security Policies - Update Support Policies - Re-Add Deluge as incubator App - Add Canary to Security Policies - Add all-contributors basic config

Ornias1993 · 2021-04-09T11:48:54Z

I hereby merge this into master. 🥇

Please be aware: We might still need to tweak a thing or two before 21.04ALPHA release, but I think the quality of this release is already many times beter than what we currently have sitting on Master.

I want to thank everyone involved for their Work and their patience! 💯

21.04 TrueCharts2.0 (Breaking Changes)

truecharts-admin · 2023-07-09T00:31:42Z

This PR is locked to prevent necro-posting on closed PRs. Please create a issue or contact staff on discord if you want to further discuss this

[Common] Refactor Services (#212)

d22b481

* Refactor All Services - services (dict, including main service) - additionalServices (list) * Add Documentation and standardised questions.yaml layout for services * Update all existing Apps to 2.0.0 * Fix whiteline error * fix addons

Ornias1993 added bug Something isn't working documentation Improvements or additions to documentation enhancement New feature or request labels Mar 3, 2021

Ornias1993 self-assigned this Mar 3, 2021

Ornias1993 mentioned this pull request Mar 3, 2021

[Common] Refactor and Document Services #211

Closed

Ornias1993 force-pushed the staging branch from 6ce336d to 3631622 Compare March 3, 2021 21:24

Ornias1993 added 3 commits March 5, 2021 22:31

Move docs from .github to project root (#223)

5373c61

Move to new ix_values system (#233)

680e359

* Refactor questions.yaml, ix_values.yaml and values.yaml * Add Docs

Ornias1993 force-pushed the master branch from afc7bcb to b7bed1c Compare March 7, 2021 14:26

Ornias1993 added 2 commits March 7, 2021 17:46

Cleanup versioning and add auto-updater (#245)

0481612

Run as PUID/PGID by default and add startAsRoot option (#246)

4e6ceee

Ornias1993 removed their assignment Mar 8, 2021

djs52 and others added 2 commits March 8, 2021 20:01

Ornias1993 self-assigned this Mar 11, 2021

Ornias1993 mentioned this pull request Mar 11, 2021

Dynamicly determine portal settings #253

Merged

10 tasks

Ornias1993 and others added 5 commits March 11, 2021 17:41

Move to truecharts containers (#258)

4b16ef4

Add a clear warning about auto-permissions, risks and when not to use…

fcc43db

… it.

ExternalService and Entrypoint-Disable Support (#256)

1257de6

* Setup externalService processing and enhance Traefik * Simplify Ingress * Simplify External Services * Adjust Service and Portal settings accordingly * Fix some newly introduced bugs * Adapt docs

Ornias1993 force-pushed the staging branch from ad940b6 to 9374e69 Compare March 12, 2021 16:21

Ornias1993 marked this pull request as ready for review March 12, 2021 16:49

Ornias1993 requested review from djs52, stavros-k and tprelog March 12, 2021 16:55

Ornias1993 changed the title ~~[WIP] NEXT-Release (BREAKING CHANGES!)~~ 21.04 TrueCharts2.0 (Breaking Changes) Mar 12, 2021

Ornias1993 force-pushed the staging branch from a1ccf80 to 0d553b5 Compare April 2, 2021 15:17

truecharts deleted a comment from stavros-k Apr 4, 2021

Ornias1993 mentioned this pull request Apr 5, 2021

imported cert not seen by Traefik cert selection box #288

Closed

Ornias1993 requested a review from tprelog April 6, 2021 21:04

Ornias1993 force-pushed the staging branch 2 times, most recently from 5f11d73 to 14674fb Compare April 9, 2021 08:39

Ornias1993 force-pushed the staging branch from e1257e3 to 2279c70 Compare April 9, 2021 10:34

Post Code-Freeze Review, Bugsfixes and Documentation

cde1863

- Fix traefik Portal Button - Touchup Security Policies - Update Support Policies - Re-Add Deluge as incubator App - Add Canary to Security Policies - Add all-contributors basic config

Ornias1993 force-pushed the staging branch 2 times, most recently from 7143ab7 to cde1863 Compare April 9, 2021 11:37

Ornias1993 merged commit eb9bc1b into master Apr 9, 2021

ellienieuwdorp pushed a commit to ellienieuwdorp/charts that referenced this pull request Apr 10, 2021

Merge pull request truecharts#216 from truecharts/staging

17424df

21.04 TrueCharts2.0 (Breaking Changes)

truecharts locked as resolved and limited conversation to collaborators Jul 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

21.04 TrueCharts2.0 (Breaking Changes) #216

21.04 TrueCharts2.0 (Breaking Changes) #216

Ornias1993 commented Mar 3, 2021 •

edited

Ornias1993 commented Mar 12, 2021

Ornias1993 commented Apr 2, 2021

stavros-k commented Apr 2, 2021

Ornias1993 commented Apr 2, 2021 •

edited

stavros-k commented Apr 4, 2021

Ornias1993 commented Apr 4, 2021 •

edited

Ornias1993 commented Apr 4, 2021

Ornias1993 commented Apr 5, 2021 •

edited

Ornias1993 commented Apr 5, 2021 •

edited

stavros-k commented Apr 5, 2021

Ornias1993 commented Apr 5, 2021

Ornias1993 commented Apr 6, 2021

Ornias1993 commented Apr 6, 2021 •

edited

Ornias1993 commented Apr 7, 2021 •

edited

Ornias1993 commented Apr 9, 2021 •

edited

Ornias1993 commented Apr 9, 2021

Ornias1993 commented Apr 9, 2021

truecharts-admin commented Jul 9, 2023

21.04 TrueCharts2.0 (Breaking Changes) #216

21.04 TrueCharts2.0 (Breaking Changes) #216

Conversation

Ornias1993 commented Mar 3, 2021 • edited

Ornias1993 commented Mar 12, 2021

Ornias1993 commented Apr 2, 2021

stavros-k commented Apr 2, 2021

Ornias1993 commented Apr 2, 2021 • edited

stavros-k commented Apr 4, 2021

Ornias1993 commented Apr 4, 2021 • edited

Ornias1993 commented Apr 4, 2021

Ornias1993 commented Apr 5, 2021 • edited

Ornias1993 commented Apr 5, 2021 • edited

stavros-k commented Apr 5, 2021

Ornias1993 commented Apr 5, 2021

Ornias1993 commented Apr 6, 2021

Ornias1993 commented Apr 6, 2021 • edited

Ornias1993 commented Apr 7, 2021 • edited

Ornias1993 commented Apr 9, 2021 • edited

Ornias1993 commented Apr 9, 2021

Ornias1993 commented Apr 9, 2021

truecharts-admin commented Jul 9, 2023

Ornias1993 commented Mar 3, 2021 •

edited

Ornias1993 commented Apr 2, 2021 •

edited

Ornias1993 commented Apr 4, 2021 •

edited

Ornias1993 commented Apr 5, 2021 •

edited

Ornias1993 commented Apr 5, 2021 •

edited

Ornias1993 commented Apr 6, 2021 •

edited

Ornias1993 commented Apr 7, 2021 •

edited

Ornias1993 commented Apr 9, 2021 •

edited