Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set the database column type to text for 'Workflow/Experiment/Schedule' #4151

Merged
merged 4 commits into from Aug 17, 2023

Conversation

moqimoqidea
Copy link
Contributor

@moqimoqidea moqimoqidea commented Aug 15, 2023

What problem does this PR solve?

The DB of chaos dashboard restricts the length of these three fields to normal varchar. This can easily lead to storage failure due to data being too long, logs show:

2023-08-09T07:59:54.584Z	ERROR	collector.NetworkChaos	collector/collector.go:114	failed to update experiment	{"archive": {"ID":16,"CreatedAt":"2023-08-05T11:15:45Z","UpdatedAt":"2023-08-09T07:59:54.583499664Z","DeletedAt":null,"uid":"d97a7ec5-8b9d-4acf-8ee4-a997b42648fb","kind":"NetworkChaos","name":"net-loss-100","namespace":"default","action":"loss","start_time":"2023-08-05T11:15:44Z","finish_time":null,"archived":false,"Experiment":"{\"kind\":\"NetworkChaos\",\"apiVersion\":\"chaos-mesh.org/v1alpha1\",\"metadata\":{\"name\":\"net-loss-100\",\"namespace\":\"default\",\"uid\":\"d97a7ec5-8b9d-4acf-8ee4-a997b42648fb\",\"resourceVersion\":\"4947395\",\"generation\":31,\"creationTimestamp\":\"2023-08-05T11:15:44Z\",\"annotations\":{\"experiment.chaos-mesh.org/pause\":\"true\"},\"finalizers\":[\"chaos-mesh/records\"],\"managedFields\":[{\"manager\":\"chaos-dashboard\",\"operation\":\"Update\",\"apiVersion\":\"chaos-mesh.org/v1alpha1\",\"time\":\"2023-08-05T11:17:51Z\",\"fieldsType\":\"FieldsV1\",\"fieldsV1\":{\"f:metadata\":{\"f:annotations\":{\".\":{},\"f:experiment.chaos-mesh.org/pause\":{}}},\"f:spec\":{\".\":{},\"f:action\":{},\"f:direction\":{},\"f:loss\":{\".\":{},\"f:loss\":{}},\"f:mode\":{},\"f:selector\":{\".\":{},\"f:labelSelectors\":{\".\":{},\"f:app\":{}},\"f:namespaces\":{}}},\"f:status\":{\".\":{},\"f:experiment\":{}}}},{\"manager\":\"chaos-controller-manager\",\"operation\":\"Update\",\"apiVersion\":\"chaos-mesh.org/v1alpha1\",\"time\":\"2023-08-05T11:22:47Z\",\"fieldsType\":\"FieldsV1\",\"fieldsV1\":{\"f:metadata\":{\"f:finalizers\":{\".\":{},\"v:\\\"chaos-mesh/records\\\"\":{}}},\"f:status\":{\"f:conditions\":{},\"f:experiment\":{\"f:containerRecords\":{},\"f:desiredPhase\":{}},\"f:instances\":{\".\":{},\"f:default/chaos-hello-web-block-5cff846575-7c7cf\":{}}}}}]},\"spec\":{\"selector\":{\"namespaces\":[\"default\"],\"labelSelectors\":{\"app\":\"chaos-hello-web-block\"}},\"mode\":\"all\",\"action\":\"loss\",\"loss\":{\"loss\":\"100\",\"correlation\":\"0\"},\"direction\":\"to\"},\"status\":{\"conditions\":[{\"type\":\"Selected\",\"status\":\"True\"},{\"type\":\"AllInjected\",\"status\":\"False\"},{\"type\":\"AllRecovered\",\"status\":\"True\"},{\"type\":\"Paused\",\"status\":\"True\"}],\"experiment\":{\"desiredPhase\":\"Stop\",\"containerRecords\":[{\"id\":\"default/chaos-hello-web-block-5cff846575-7c7cf\",\"selectorKey\":\".\",\"phase\":\"Not Injected\",\"injectedCount\":1,\"recoveredCount\":1,\"events\":[{\"type\":\"Succeeded\",\"operation\":\"Apply\",\"timestamp\":\"2023-08-05T11:15:44Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:17:51Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:17:52Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:17:52Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:17:52Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:17:52Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:17:52Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:17:52Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:17:52Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:17:53Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:17:53Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:17:54Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:17:54Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:17:59Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:17:59Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:18:20Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:18:20Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:19:42Z\"},{\"type\":\"Failed\",\"operation\":\"Recover\",\"message\":\"failed to apply for pod default/chaos-hello-web-block-5cff846575-7c7cf: unable to flush ip sets for pod chaos-hello-web-block-5cff846575-7c7cf\",\"timestamp\":\"2023-08-05T11:19:42Z\"},{\"type\":\"Succeeded\",\"operation\":\"Recover\",\"timestamp\":\"2023-08-05T11:22:47Z\"}]}]},\"instances\":{\"default/chaos-hello-web-block-5cff846575-7c7cf\":2}}}"}, "error": "Error 1406: Data too long for column 'experiment' at row 1"}
2023-08-09T08:04:21.815Z	ERROR	collector.event-collector.Event	collector/event_collector.go:105	failed to save event	{"event": {"id":0,"object_id":"4c5e13f1-e59b-48bc-afe0-e4061aefa16d","created_at":"2023-08-09T08:04:21Z","namespace":"default","name":"kill-tomcat-6","kind":"PodChaos","type":"Warning","reason":"Failed","message":"Failed to update records: Internal error occurred: failed calling webhook \"mpodchaos.kb.io\": failed to call webhook: Post \"https://chaos-mesh-controller-manager.chaos-mesh.svc:443/mutate-chaos-mesh-org-v1alpha1-podchaos?timeout=5s\": context deadline exceeded"}, "error": "Error 1406: Data too long for column 'message' at row 1"}

notes: no Schedule Data too long log, I haven't used Schedule heavily yet, but according to the storage structure, I predict we will encounter the same problem in the future.

What's changed and how it works?

Set the database column type to text for 'Workflow/Experiment/Schedule'

Checklist

CHANGELOG

Must include at least one of them.

  • I have updated the CHANGELOG.md
  • I have labeled this PR with "no-need-update-changelog"

Tests

Must include at least one of them.

  • Unit test
  • E2E test
  • Manual test

Side effects

  • Breaking backward compatibility

DCO

If you find the DCO check fails, please run commands like below (Depends on the actual situations. For example, if the failed commit isn't the most recent) to fix it:

git commit --amend --signoff
git push --force

Signed-off-by: moqimoqidea <moqimoqidea@gmail.com>
…t for 'Workflow/Experiment/Schedule'

Signed-off-by: moqimoqidea <moqimoqidea@gmail.com>
@codecov
Copy link

codecov bot commented Aug 15, 2023

Codecov Report

Merging #4151 (407f77c) into master (dc1704e) will not change coverage.
Report is 8 commits behind head on master.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #4151   +/-   ##
=======================================
  Coverage   38.55%   38.55%           
=======================================
  Files         167      167           
  Lines       13752    13752           
=======================================
  Hits         5302     5302           
+ Misses       8017     8015    -2     
- Partials      433      435    +2     

see 4 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 09e7c61...407f77c. Read the comment docs.

@g1eny0ung g1eny0ung requested review from g1eny0ung and removed request for YangKeao and Andrewmatilde August 15, 2023 10:17
@g1eny0ung
Copy link
Member

/cherrypick release-2.6

@chaos-mesh-cherrypick-robot

@g1eny0ung: once the present PR merges, I will cherry-pick it on top of release-2.6 in a new PR and assign it to you.

In response to this:

/cherrypick release-2.6

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Signed-off-by: Yue Yang <g1enyy0ung@gmail.com>
@g1eny0ung g1eny0ung requested review from STRRL and cwen0 August 16, 2023 07:48
@g1eny0ung g1eny0ung added type/bug-fix A bug needs to be fixed. and removed type/enhancement New feature or request labels Aug 16, 2023
@g1eny0ung
Copy link
Member

g1eny0ung commented Aug 16, 2023

@moqimoqidea These changes make sense to me. Thanks for your contribution!

@cwen0 @STRRL I'm planning to cherry-pick this fix into the v2.6 branch and then release the next patch version. What do you think?

@moqimoqidea
Copy link
Contributor Author

@moqimoqidea These changes make sense to me. Thanks for your contribution!

@cwen0 @STRRL I'm planning to cherry-pick this fix into the v2.6 branch and then release the next patch version. What do you think?

👍

@g1eny0ung
Copy link
Member

Now merging.

@g1eny0ung
Copy link
Member

/lgtm

@chaotic-prow
Copy link

chaotic-prow bot commented Aug 17, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: g1eny0ung

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@chaotic-prow chaotic-prow bot merged commit ebcf2ad into chaos-mesh:master Aug 17, 2023
62 checks passed
@chaos-mesh-cherrypick-robot

@g1eny0ung: #4151 failed to apply on top of branch "release-2.6":

Applying: Set the database column type to text for 'Workflow/Experiment/Schedule'
Applying: Update CHANGELOG.md with content: Set the database column type to text for 'Workflow/Experiment/Schedule'
Applying: Update CHANGELOG.md
Using index info to reconstruct a base tree...
M	CHANGELOG.md
Falling back to patching base and 3-way merge...
Auto-merging CHANGELOG.md
CONFLICT (content): Merge conflict in CHANGELOG.md
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0003 Update CHANGELOG.md
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherrypick release-2.6

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

g1eny0ung added a commit to g1eny0ung/chaos-mesh that referenced this pull request Aug 18, 2023
…e' (chaos-mesh#4151)

* Set the database column type to text for 'Workflow/Experiment/Schedule'

Signed-off-by: moqimoqidea <moqimoqidea@gmail.com>

* Update CHANGELOG.md with content: Set the database column type to text for 'Workflow/Experiment/Schedule'

Signed-off-by: moqimoqidea <moqimoqidea@gmail.com>

* Update CHANGELOG.md

Signed-off-by: Yue Yang <g1enyy0ung@gmail.com>

---------

Signed-off-by: moqimoqidea <moqimoqidea@gmail.com>
Signed-off-by: Yue Yang <g1enyy0ung@gmail.com>
Co-authored-by: Yue Yang <g1enyy0ung@gmail.com>
g1eny0ung added a commit that referenced this pull request Aug 18, 2023
…e' (#4151) (#4159)

Signed-off-by: moqimoqidea <moqimoqidea@gmail.com>
Signed-off-by: Yue Yang <g1enyy0ung@gmail.com>
Co-authored-by: moqimoqidea <39821951+moqimoqidea@users.noreply.github.com>
kenneth-buck-form3 pushed a commit to form3tech-oss/chaos-mesh that referenced this pull request Aug 29, 2023
…e' (chaos-mesh#4151)

* Set the database column type to text for 'Workflow/Experiment/Schedule'

Signed-off-by: moqimoqidea <moqimoqidea@gmail.com>

* Update CHANGELOG.md with content: Set the database column type to text for 'Workflow/Experiment/Schedule'

Signed-off-by: moqimoqidea <moqimoqidea@gmail.com>

* Update CHANGELOG.md

Signed-off-by: Yue Yang <g1enyy0ung@gmail.com>

---------

Signed-off-by: moqimoqidea <moqimoqidea@gmail.com>
Signed-off-by: Yue Yang <g1enyy0ung@gmail.com>
Co-authored-by: Yue Yang <g1enyy0ung@gmail.com>
Signed-off-by: Kenneth Buck <kenneth.buck@form3.tech>
kenneth-buck-form3 pushed a commit to form3tech-oss/chaos-mesh that referenced this pull request Aug 29, 2023
…e' (chaos-mesh#4151)

* Set the database column type to text for 'Workflow/Experiment/Schedule'

Signed-off-by: moqimoqidea <moqimoqidea@gmail.com>

* Update CHANGELOG.md with content: Set the database column type to text for 'Workflow/Experiment/Schedule'

Signed-off-by: moqimoqidea <moqimoqidea@gmail.com>

* Update CHANGELOG.md

Signed-off-by: Yue Yang <g1enyy0ung@gmail.com>

---------

Signed-off-by: moqimoqidea <moqimoqidea@gmail.com>
Signed-off-by: Yue Yang <g1enyy0ung@gmail.com>
Co-authored-by: Yue Yang <g1enyy0ung@gmail.com>
Signed-off-by: Kenneth Buck <kenneth.buck@form3.tech>
kenneth-buck-form3 pushed a commit to form3tech-oss/chaos-mesh that referenced this pull request Aug 29, 2023
…e' (chaos-mesh#4151)

* Set the database column type to text for 'Workflow/Experiment/Schedule'

Signed-off-by: moqimoqidea <moqimoqidea@gmail.com>

* Update CHANGELOG.md with content: Set the database column type to text for 'Workflow/Experiment/Schedule'

Signed-off-by: moqimoqidea <moqimoqidea@gmail.com>

* Update CHANGELOG.md

Signed-off-by: Yue Yang <g1enyy0ung@gmail.com>

---------

Signed-off-by: moqimoqidea <moqimoqidea@gmail.com>
Signed-off-by: Yue Yang <g1enyy0ung@gmail.com>
Co-authored-by: Yue Yang <g1enyy0ung@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants