fix: Perform fields filtering server side #4595

simster7 · 2020-11-24T19:09:05Z

Fixes: #4529

Not ready for review

Signed-off-by: Simon Behar simbeh7@gmail.com

Checklist:

My organization is added to USERS.md.

Signed-off-by: Simon Behar <simbeh7@gmail.com>

simster7 · 2020-12-01T17:24:12Z

util/json/json.go

-// https://github.com/ksonnet/ksonnet/blob/master/pkg/kubecfg/diff.go
-func removeFields(config, live interface{}) interface{} {


This code is not used anywhere

Signed-off-by: Simon Behar <simbeh7@gmail.com>

alexec · 2020-11-24T19:12:18Z

util/fields/fields.go

+	}
+
+	data := make(map[string]interface{})
+	err := json.Unmarshal(dataBytes, &data)


I think the fundamentals are really good in this PR.

Is there any way to do this without marshaling back and forth to JSON - this is an expensive operation and it'll add O(n) cost to ListWorkflows.

It's no biggie if:

Prior-art - this is already how we do this. I.e. I'm wrong.

It takes too long to find an alternative.

Prior-art - this is already how we do this. I.e. I'm wrong.

This is how we do this already. I did spend some modest time coming up with an alternative to no avail.

https://github.com/argoproj/pkg/blob/50e2680bec730dc985bb2e50d02c1d383caa0145/grpc/http/forwarders.go#L41-L108

cool - thank you for trying

alexec · 2020-12-01T17:52:54Z

go.mod

@@ -19,6 +19,7 @@ require (
 	github.com/evanphx/json-patch v4.5.0+incompatible
 	github.com/fatih/structs v1.1.0 // indirect
 	github.com/gavv/httpexpect/v2 v2.0.3
+	github.com/ghodss/yaml v1.0.0


alexec

just remove the ghodds yaml and you're golden.

Signed-off-by: Simon Behar <simbeh7@gmail.com>

Signed-off-by: github@finnesand.no <github@finnesand.no> feat(ui): Add Template/Cron workflow filter to workflow page. Closes argoproj#4532 (argoproj#4543) Signed-off-by: Tianchu Zhao <evantczhao@gmail.com> feat(executor): Auto create s3 bucket if not present. Signed-off-by: Alex Capras <alexcapras@gmail.com> Apply codegen Signed-off-by: Alex Capras <alexcapras@gmail.com> Add argo-e2e label to test wf Signed-off-by: Alex Capras <alexcapras@gmail.com> chore: Updated stress test YAML (argoproj#4569) Signed-off-by: Alex Collins <alex_collins@intuit.com> docs: Updated kubectl apply command in manifests README (argoproj#4577) Signed-off-by: Stefan Gloutnikov <stefan@gloutnikov.com> feat(controller): Make MAX_OPERATION_TIME configurable. Close argoproj#4239 (argoproj#4562) Signed-off-by: Alex Collins <alex_collins@intuit.com> docs: Fix a typo in example (argoproj#4590) Signed-off-by: Takayoshi Nishida <takayoshi.nishida@gmail.com> feat(controller): Retry transient offload errors. Resolves argoproj#4464 (argoproj#4482) Signed-off-by: Alex Collins <alex_collins@intuit.com> fix(server): use the correct name when downloading artifacts (argoproj#4579) Signed-off-by: Daniel Herman <dherman@factset.com> fix(server): serve artifacts directly from disk to support large artifacts (argoproj#4589) Signed-off-by: Daniel Herman <dherman@factset.com> fix(executor): Handle sidecar killing in a process-namespace-shared pod (argoproj#4575) Signed-off-by: Daisuke Taniwaki <daisuketaniwaki@gmail.com> docs: Add JSON schema for IDE validation (argoproj#4581) Signed-off-by: Paul Brabban <paul.brabban@gmail.com> refactor: Use polling model for workflow phase metric (argoproj#4557) Signed-off-by: Simon Behar <simbeh7@gmail.com> Addressing reviewers comments Signed-off-by: Alex Capras <alexcapras@gmail.com> Addressing reviewers comments docs: Minor typo fix (argoproj#4610) Signed-off-by: Paavo Pokkinen <paavo.pokkinen@vaimo.com> fix(controller): Prevent tasks with names starting with digit to use either 'depends' or 'dependencies' (argoproj#4598) Signed-off-by: terrytangyuan <terrytangyuan@gmail.com> fix(docs): Bring minio chart instructions up to date (argoproj#4586) Signed-off-by: Ranga Krishnan <ranga@bei.re> fix(executor): Fixed waitMainContainerStart returning prematurely. Closes argoproj#4599 (argoproj#4601) Signed-off-by: fsiegmund <siegmund@slb.com> feat(controller): Enhanced artifact repository ref. See argoproj#3184 (argoproj#4458) Signed-off-by: Alex Collins <alex_collins@intuit.com> fix: Null check pagination variable (argoproj#4617) Signed-off-by: Simon Behar <simbeh7@gmail.com> fix: Perform fields filtering server side (argoproj#4595) Signed-off-by: Simon Behar <simbeh7@gmail.com> fix(server): Correct webhook event payload marshalling. Fixes argoproj#4572 (argoproj#4594) Signed-off-by: Alex Collins <alex_collins@intuit.com> feat(ui): Add columns--narrower-height to AttributeRow (argoproj#4371) fix: Fix TestCleanFieldsExclude (argoproj#4625) Signed-off-by: Simon Behar <simbeh7@gmail.com> fix(argo-server): fix global variable validation error with reversed dag.tasks (argoproj#4369) Signed-off-by: chenyu.zheng <chenyu.zheng@hulu.com> fix: derive jsonschema and fix up issues, validate examples dir… (argoproj#4611) Signed-off-by: Paul Brabban <paul.brabban@gmail.com> fix(ui): Reference secrets in EnvVars. Fixes argoproj#3973 (argoproj#4419) Signed-off-by: Alejandro Tejera <aletepe@gmail.com> fix(ui): Fix Snyk issues (argoproj#4631) Signed-off-by: Alex Collins <alex_collins@intuit.com> feat(executor): More informative log when executors do not support output param from base image layer (argoproj#4620) Signed-off-by: terrytangyuan <terrytangyuan@gmail.com> Codegen patch. Signed off by alexcapras@gmail.com Codegen patch. Signed off by alexcapras@gmail.com Delete test.patch

Signed-off-by: github@finnesand.no <github@finnesand.no> feat(ui): Add Template/Cron workflow filter to workflow page. Closes argoproj#4532 (argoproj#4543) Signed-off-by: Tianchu Zhao <evantczhao@gmail.com> feat(executor): Auto create s3 bucket if not present. Signed-off-by: Alex Capras <alexcapras@gmail.com> Apply codegen Signed-off-by: Alex Capras <alexcapras@gmail.com> Add argo-e2e label to test wf Signed-off-by: Alex Capras <alexcapras@gmail.com> chore: Updated stress test YAML (argoproj#4569) Signed-off-by: Alex Collins <alex_collins@intuit.com> docs: Updated kubectl apply command in manifests README (argoproj#4577) Signed-off-by: Stefan Gloutnikov <stefan@gloutnikov.com> feat(controller): Make MAX_OPERATION_TIME configurable. Close argoproj#4239 (argoproj#4562) Signed-off-by: Alex Collins <alex_collins@intuit.com> docs: Fix a typo in example (argoproj#4590) Signed-off-by: Takayoshi Nishida <takayoshi.nishida@gmail.com> feat(controller): Retry transient offload errors. Resolves argoproj#4464 (argoproj#4482) Signed-off-by: Alex Collins <alex_collins@intuit.com> fix(server): use the correct name when downloading artifacts (argoproj#4579) Signed-off-by: Daniel Herman <dherman@factset.com> fix(server): serve artifacts directly from disk to support large artifacts (argoproj#4589) Signed-off-by: Daniel Herman <dherman@factset.com> fix(executor): Handle sidecar killing in a process-namespace-shared pod (argoproj#4575) Signed-off-by: Daisuke Taniwaki <daisuketaniwaki@gmail.com> docs: Add JSON schema for IDE validation (argoproj#4581) Signed-off-by: Paul Brabban <paul.brabban@gmail.com> refactor: Use polling model for workflow phase metric (argoproj#4557) Signed-off-by: Simon Behar <simbeh7@gmail.com> Addressing reviewers comments Signed-off-by: Alex Capras <alexcapras@gmail.com> Addressing reviewers comments docs: Minor typo fix (argoproj#4610) Signed-off-by: Paavo Pokkinen <paavo.pokkinen@vaimo.com> fix(controller): Prevent tasks with names starting with digit to use either 'depends' or 'dependencies' (argoproj#4598) Signed-off-by: terrytangyuan <terrytangyuan@gmail.com> fix(docs): Bring minio chart instructions up to date (argoproj#4586) Signed-off-by: Ranga Krishnan <ranga@bei.re> fix(executor): Fixed waitMainContainerStart returning prematurely. Closes argoproj#4599 (argoproj#4601) Signed-off-by: fsiegmund <siegmund@slb.com> feat(controller): Enhanced artifact repository ref. See argoproj#3184 (argoproj#4458) Signed-off-by: Alex Collins <alex_collins@intuit.com> fix: Null check pagination variable (argoproj#4617) Signed-off-by: Simon Behar <simbeh7@gmail.com> fix: Perform fields filtering server side (argoproj#4595) Signed-off-by: Simon Behar <simbeh7@gmail.com> fix(server): Correct webhook event payload marshalling. Fixes argoproj#4572 (argoproj#4594) Signed-off-by: Alex Collins <alex_collins@intuit.com> feat(ui): Add columns--narrower-height to AttributeRow (argoproj#4371) fix: Fix TestCleanFieldsExclude (argoproj#4625) Signed-off-by: Simon Behar <simbeh7@gmail.com> fix(argo-server): fix global variable validation error with reversed dag.tasks (argoproj#4369) Signed-off-by: chenyu.zheng <chenyu.zheng@hulu.com> fix: derive jsonschema and fix up issues, validate examples dir… (argoproj#4611) Signed-off-by: Paul Brabban <paul.brabban@gmail.com> fix(ui): Reference secrets in EnvVars. Fixes argoproj#3973 (argoproj#4419) Signed-off-by: Alejandro Tejera <aletepe@gmail.com> fix(ui): Fix Snyk issues (argoproj#4631) Signed-off-by: Alex Collins <alex_collins@intuit.com> feat(executor): More informative log when executors do not support output param from base image layer (argoproj#4620) Signed-off-by: terrytangyuan <terrytangyuan@gmail.com> Codegen patch. Signed off by alexcapras@gmail.com Codegen patch. Signed off by alexcapras@gmail.com Delete test.patch Signed-off-by: Alex Capras <alexcapras@gmail.com>

Signed-off-by: Simon Behar <simbeh7@gmail.com>

alexec · 2020-12-03T00:56:09Z

Merged into https://github.com/argoproj/argo/releases/tag/v2.12.0-rc4

simster7 added 2 commits November 24, 2020 13:08

fix: Perform fields filtering server side

e22d293

Signed-off-by: Simon Behar <simbeh7@gmail.com>

Merge branch 'master' into server-side-fields

288f7ff

simster7 commented Dec 1, 2020

View reviewed changes

simster7 added 2 commits December 1, 2020 11:42

Add tests

d00c881

Signed-off-by: Simon Behar <simbeh7@gmail.com>

lint

64523a1

Signed-off-by: Simon Behar <simbeh7@gmail.com>

simster7 marked this pull request as ready for review December 1, 2020 17:45

alexec reviewed Dec 1, 2020

View reviewed changes

alexec approved these changes Dec 1, 2020

View reviewed changes

clean

7c03428

Signed-off-by: Simon Behar <simbeh7@gmail.com>

simster7 merged commit 39c805f into argoproj:master Dec 1, 2020

alexec pushed a commit that referenced this pull request Dec 3, 2020

fix: Perform fields filtering server side (#4595)

b1d682e

Signed-off-by: Simon Behar <simbeh7@gmail.com>

alexec added this to the v2.12 milestone Dec 3, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Perform fields filtering server side #4595

fix: Perform fields filtering server side #4595

simster7 commented Nov 24, 2020

simster7 Dec 1, 2020

alexec Nov 24, 2020

simster7 Dec 1, 2020

alexec Dec 1, 2020

alexec Dec 1, 2020

simster7 Dec 1, 2020

alexec left a comment

alexec commented Dec 3, 2020

		// https://github.com/ksonnet/ksonnet/blob/master/pkg/kubecfg/diff.go
		func removeFields(config, live interface{}) interface{} {

fix: Perform fields filtering server side #4595

fix: Perform fields filtering server side #4595

Conversation

simster7 commented Nov 24, 2020

simster7 Dec 1, 2020

Choose a reason for hiding this comment

alexec Nov 24, 2020

Choose a reason for hiding this comment

simster7 Dec 1, 2020

Choose a reason for hiding this comment

alexec Dec 1, 2020

Choose a reason for hiding this comment

alexec Dec 1, 2020

Choose a reason for hiding this comment

simster7 Dec 1, 2020

Choose a reason for hiding this comment

alexec left a comment

Choose a reason for hiding this comment

alexec commented Dec 3, 2020