job-list: limit size of job constraint #5669
Assignees
Comments
chu11
added a commit
to chu11/flux-core
that referenced
this issue
Jan 13, 2024
Problem: Job-list constraints are not limited in their size. A nefarious user could cause a DoS of the job-list service by sending an extremely large constraint request. Solution: Set a reasonable max on constraint sizes. The maximum length a constraint array can be is 256 elements and the maximum recursive depth (via and/or/not operators) is set to 16. Add unit tests. Fixes flux-framework#5669
chu11
added a commit
to chu11/flux-core
that referenced
this issue
Jan 13, 2024
Problem: Job-list constraints are not limited in their size. A nefarious user could cause a DoS of the job-list service by sending an extremely large constraint request. Solution: Set a reasonable max on constraint sizes. The maximum length a constraint array can be is 256 elements and the maximum recursive depth (via and/or/not operators) is set to 16. Add unit tests. Fixes flux-framework#5669
chu11
added a commit
to chu11/flux-core
that referenced
this issue
Feb 1, 2024
Problem: Currently the max number of comparisons that can be done in a job-list constraint match is hard coded to 100 thousand. There may be circumstances where we'd like that number to be higher or lower. Support configuration of this maximum via a new job-list.max_comparisons configuration. Fixes flux-framework#5669
chu11
added a commit
to chu11/flux-core
that referenced
this issue
Feb 2, 2024
Problem: Job-list constraints are not limited in their size. A nefarious user could cause a DoS of the job-list service by sending an extremely large constraint request. Solution: To limit constraints, introduce the concept of constraint comparisons. All comparison "checks", such as seeing if a job was submitted by a specific user, are counted. When the maximum number of comparisons is reached, an error is returned to the user. The user can try again by limiting the number of jobs they are searching through, such as only searching active vs inactive jobs or if jobs were submitted since a certain time, For the time being, hard code the maximum number of comparisons to 100 thousand. Fixes flux-framework#5669
chu11
added a commit
to chu11/flux-core
that referenced
this issue
Mar 20, 2024
Problem: Job-list constraints are not limited in their size. A nefarious user could cause a DoS of the job-list service by sending an extremely large constraint request. Solution: To limit constraints, introduce the concept of constraint comparisons. All comparison "checks", such as seeing if a job was submitted by a specific user, are counted. When the maximum number of comparisons is reached, an error is returned to the user. The user can try again by limiting the number of jobs they are searching through, such as only searching active vs inactive jobs or if jobs were submitted since a certain time, For the time being, hard code the maximum number of comparisons to 100 thousand. Fixes flux-framework#5669
chu11
added a commit
to chu11/flux-core
that referenced
this issue
Mar 20, 2024
Problem: Job-list constraints are not limited in their size. A nefarious user could cause a DoS of the job-list service by sending an extremely large constraint request. Solution: To limit constraints, introduce the concept of constraint comparisons. All comparison "checks", such as seeing if a job was submitted by a specific user, are counted. When the maximum number of comparisons is reached, an error is returned to the user. The user can try again by limiting the number of jobs they are searching through, such as only searching active vs inactive jobs or if jobs were submitted since a certain time, For the time being, hard code the maximum number of comparisons to 100 thousand. Fixes flux-framework#5669
chu11
added a commit
to chu11/flux-core
that referenced
this issue
Apr 8, 2024
Problem: Job-list constraints are not limited in their size. A nefarious user could cause a DoS of the job-list service by sending an extremely large constraint request. Solution: To limit constraints, introduce the concept of constraint comparisons. All comparison "checks", such as seeing if a job was submitted by a specific user, are counted. When the maximum number of comparisons is reached, an error is returned to the user. The user can try again by limiting the number of jobs they are searching through, such as only searching active vs inactive jobs or if jobs were submitted since a certain time. For the time being, hard code the maximum number of comparisons to 100 thousand. Fixes flux-framework#5669
chu11
added a commit
to chu11/flux-core
that referenced
this issue
Apr 8, 2024
Problem: Job-list constraints are not limited in their size. A nefarious user could cause a DoS of the job-list service by sending an extremely large constraint request. Solution: To limit constraints, introduce the concept of constraint comparisons. All comparison "checks", such as seeing if a job was submitted by a specific user, are counted. When the maximum number of comparisons is reached, an error is returned to the user. The user can try again by limiting the number of jobs they are searching through, such as only searching active vs inactive jobs or if jobs were submitted since a certain time. For the time being, hard code the maximum number of comparisons to 100 thousand. Fixes flux-framework#5669
chu11
added a commit
to chu11/flux-core
that referenced
this issue
Apr 17, 2024
Problem: Job-list constraints are not limited in their size. A nefarious user could cause a DoS of the job-list service by sending an extremely large constraint request. Solution: To limit constraints, introduce the concept of constraint comparisons. All comparison "checks", such as seeing if a job was submitted by a specific user, are counted. When the maximum number of comparisons is reached, an error is returned to the user. The user can try again by limiting the number of jobs they are searching through, such as only searching active vs inactive jobs or if jobs were submitted since a certain time. For the time being, hard code the maximum number of comparisons to 100 thousand. Fixes flux-framework#5669
chu11
added a commit
to chu11/flux-core
that referenced
this issue
Apr 18, 2024
Problem: Job-list constraints are not limited in their size. A nefarious user could cause a DoS of the job-list service by sending an extremely large constraint request. Solution: To limit constraints, introduce the concept of constraint comparisons. All comparison "checks", such as seeing if a job was submitted by a specific user, are counted. When the maximum number of comparisons is reached, an error is returned to the user. The user can try again by limiting the number of jobs they are searching through, such as only searching active vs inactive jobs or if jobs were submitted since a certain time. For the time being, hard code the maximum number of comparisons to 100 thousand. Fixes flux-framework#5669
chu11
added a commit
to chu11/flux-core
that referenced
this issue
Apr 18, 2024
Problem: Job-list constraints are not limited in their size. A nefarious user could cause a DoS of the job-list service by sending an extremely large constraint request. Solution: To limit constraints, introduce the concept of constraint comparisons. All comparison "checks", such as seeing if a job was submitted by a specific user, are counted. When the maximum number of comparisons is reached, an error is returned to the user. The user can try again by limiting the number of jobs they are searching through, such as only searching active vs inactive jobs or if jobs were submitted since a certain time. For the time being, hard code the maximum number of comparisons to 100 thousand. Fixes flux-framework#5669
chu11
added a commit
to chu11/flux-core
that referenced
this issue
Apr 19, 2024
Problem: Job-list constraints are not limited in their size. A nefarious user could cause a DoS of the job-list service by sending an extremely large constraint request. Solution: To limit constraints, introduce the concept of constraint comparisons. All comparison "checks", such as seeing if a job was submitted by a specific user, are counted. When the maximum number of comparisons is reached, an error is returned to the user. The user can try again by limiting the number of jobs they are searching through, such as only searching active vs inactive jobs or if jobs were submitted since a certain time. For the time being, hard code the maximum number of comparisons to 100 thousand. Fixes flux-framework#5669
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
As @grondo notes in #5656, "very large" constraints might have the ability to hang (or minimally slow down) the job-list service.
We may wish to put a limit on the size of job constraints to limit this.
Without thinking about this that much, perhaps a cap on the length of an operator array might be worthwhile. 32 or 64? i.e.
{ "and": [ { "userid": [ ... ]}, { "name":[ ...] }, ... ] }`If we say there's a cap of 32, the max length of the values array for "and" would be 32, and the max of "userid" and "name" would also be 32.
Also we'd probably need a "recursive" limit too, without thinking about it much ... 8 or 16?
The text was updated successfully, but these errors were encountered: