config to disable auto-creating default policies

[lahabana]

Description

Maybe a user wants to have empty meshes and empty setups.

[johnharris85]

/assign

[johnharris85]

So looks like we create the default resources for every mesh created. When I was chatting with @lahabana about this we figured to make it a global flag (like skipDefaultMesh), but now wondering if it makes more sense on a 'per-mesh' basis? Thought potentially an annotation would be good except I guess that's not gonna work in Universal mode. Open to opinions on this :) Make a new field in the mesh config? Make it a global flag like skipDefaultMesh? Also thinking over which resources should be skipped if this is enabled. All the regular default policies make sense, but it looks like the EnsureDefaultX functions also create a signing key for the mesh. Should we skip that too? Seems like more fundamental functionality (than the default policies), but maybe if a user chooses this option we just assume they know what they're doing and they'll create their own?

[johnharris85]

Thoughts @lahabana ?

[lahabana]

Multiple questions here:

1. Should we make it per mesh?
   IMHO let's keep it simple and make it global (we talked about deprecating the skipDefaultMesh option too to avoid configuration flag blowup.
2. Is SigningKey a default resource?
   I think SigningKey should still be created as there's little value for a user to generate their own.
   I'd move this as a standalone method in mesh_helpers.go and move it up like EnsureCAs()

My time to ask questions:

1. why do we have a MeshReconciler and a DefaultMeshReconciler in k8s? Sounds like unnecessary complexity and could be folded into 1 like we do for mesh manager?

@jakubdyszkiewicz WDYT?

[github-actions]

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

[github-actions]

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

[github-actions]

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

[jakubdyszkiewicz]

If we want to introduce this, let's do this on the Mesh level. Something like

type: Mesh
defaults:
  createTrafficPermission: false
  createTrafficRoute: false
...

This would be also very convenient in E2E tests

[github-actions]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[jakubdyszkiewicz]

Once done, let's not create retry by default in E2E tests.

[github-actions]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[github-actions]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant, please comment on it or attend the next triage meeting.