Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qos/raft_service_level_distributed_data_accessor: print correct error message when trying to modify a service level in recovery mode #18841

Merged
merged 3 commits into from
May 27, 2024

Conversation

Jadw1
Copy link
Contributor

@Jadw1 Jadw1 commented May 23, 2024

Raft service levels are read-only in recovery mode. This patch adds check and proper error message when a user tries to modify service levels in recovery mode.

Fixes #18827

Jadw1 added 2 commits May 23, 2024 08:16
When setting/dropping a service level using raft data accessor, the same
validation steps are executed (this_shard_id = 0 and guard is present).
To not duplicate the calls in both functions, they can be extracted to a
helper function.
mode

When a cluster goes into recovery mode and service levels were migrated
to raft, service levels become temporarily read-only.

This commit adds a proper error message in case a user tries to do any
changes.
@Jadw1 Jadw1 requested a review from piodul May 23, 2024 06:28
@github-actions github-actions bot added symptom/ux Concerns regarding the user experience in working with Scylla. area/service levels labels May 23, 2024
Comment on lines 140 to 142
logging.info("Checking changes to service levels are forbidden during recovery")
with pytest.raises(InvalidRequest, match="The cluster is in recovery mode. Changes to service levels are not allowed."):
await cql.run_async(f"CREATE SERVICE LEVEL sl_{unique_name() }")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to check other possible operations on service levels as well - altering, dropping, attaching, detaching.

Copy link
Contributor Author

@Jadw1 Jadw1 May 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added tried to alter and drop a service levele.
However attaching/detaching a service level is part of auth, which also prints the same ugly message in recovery

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However attaching/detaching a service level is part of auth, which also prints the same ugly message in recovery

Ok, no need to increase the scope of the PR to cover them, then. I'll open an issue about the ugly error messages in auth, this can be handled there.

@piodul
Copy link
Contributor

piodul commented May 23, 2024

qos/raft_service_level_distributed_data_accessor: error message in recovery mode

"error message in recovery mode" doesn't really explain what your changes are supposed to do. Your PR improves error messages that are generated when trying to modify service levels in recovery mode.

Please improve the title.

@piodul
Copy link
Contributor

piodul commented May 23, 2024

In general, changes look good to me, but please make sure that you cover all kinds of operations on service levels and test them all.

@scylladb-promoter
Copy link
Contributor

🟢 CI State: SUCCESS

✅ - Build
✅ - Unit Tests Custom
The following new/updated tests ran 100 times for each mode:
🔹 auth_cluster/test_raft_service_levels::*
✅ - Container Test
✅ - dtest
✅ - dtest with topology changes
✅ - Unit Tests

Build Details:

  • Duration: 5 hr 50 min
  • Builder: spider2.cloudius-systems.com

@Jadw1 Jadw1 changed the title qos/raft_service_level_distributed_data_accessor: error message in recovery mode qos/raft_service_level_distributed_data_accessor: print correct error message when trying to modify a service level in recovery mode May 23, 2024
@Jadw1 Jadw1 requested a review from piodul May 23, 2024 16:09
@piodul
Copy link
Contributor

piodul commented May 24, 2024

Not sure what went wrong with the Jenkins job, it looks like Jenkins was being restarted around the time the job was scheduled and the job failed with a weird error. Retriggering.

@Jadw1
Copy link
Contributor Author

Jadw1 commented May 27, 2024

Jenkins crashed again

General pipeline failure

jenkins pipeline failure: Error when executing always post condition:

groovy.lang.MissingPropertyException: No such property: enableCoverage for class: WorkflowScript

retriggering

@scylladb-promoter
Copy link
Contributor

🟢 CI State: SUCCESS

✅ - Build
✅ - Unit Tests Custom
The following new/updated tests ran 100 times for each mode:
🔹 auth_cluster/test_raft_service_levels
✅ - Container Test
✅ - dtest with topology changes
✅ - dtest
✅ - Unit Tests

Build Details:

  • Duration: 5 hr 0 min
  • Builder: spider5.cloudius-systems.com

@piodul
Copy link
Contributor

piodul commented May 27, 2024

Queued

@scylladb-promoter scylladb-promoter merged commit fa142a9 into scylladb:master May 27, 2024
10 checks passed
piodul added a commit that referenced this pull request May 28, 2024
…r: print correct error message when trying to modify a service level in recovery mode' from ScyllaDB

Raft service levels are read-only in recovery mode. This patch adds check and proper error message when a user tries to modify service levels in recovery mode.

Fixes #18827

(cherry picked from commit 2b56158)

(cherry picked from commit ee08d7f)

(cherry picked from commit af0b6bc)

 Refs #18841

Closes #18913

* github.com:scylladb/scylladb:
  test/auth_cluster/test_raft_service_levels: try to create sl in recovery
  service/qos/raft_sl_dda: reject changes to service levels in recovery mode
  service/qos/raft_sl_dda: extract raft_sl_dda steps to common function
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/service levels backport/6.0 promoted-to-master symptom/ux Concerns regarding the user experience in working with Scylla.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

service levels on raft: ugly error while creating service level in recovery mode
3 participants