Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only allow one fixup task at a time #5413

Closed
mreynolds389 opened this issue Aug 10, 2022 · 6 comments
Closed

Only allow one fixup task at a time #5413

mreynolds389 opened this issue Aug 10, 2022 · 6 comments
Labels
needs triage The issue will be triaged during scrum

Comments

@mreynolds389
Copy link
Contributor

Issue Description

We keep seeing issues where customers incorrectly create 100's or 1000's of memberOf fixup tasks which leads to overall terrible server performance. We should only allow one fixup task per plugin.

@mreynolds389 mreynolds389 added the needs triage The issue will be triaged during scrum label Aug 10, 2022
mreynolds389 added a commit to mreynolds389/389-ds-base that referenced this issue Sep 12, 2022
Description:  only allow one fixup task to run at a time, and improve
the task logging

relates: 389ds#5413

Reviewed by: progier & tbordaz(Thanks!)
mreynolds389 added a commit that referenced this issue Sep 12, 2022
Description:  only allow one fixup task to run at a time, and improve
the task logging

relates: #5413

Reviewed by: progier & tbordaz(Thanks!)
mreynolds389 added a commit that referenced this issue Sep 12, 2022
Description:  only allow one fixup task to run at a time, and improve
the task logging

relates: #5413

Reviewed by: progier & tbordaz(Thanks!)
mreynolds389 added a commit that referenced this issue Sep 12, 2022
Description:  only allow one fixup task to run at a time, and improve
the task logging

relates: #5413

Reviewed by: progier & tbordaz(Thanks!)
mreynolds389 added a commit that referenced this issue Sep 12, 2022
Description:  only allow one fixup task to run at a time, and improve
the task logging

relates: #5413

Reviewed by: progier & tbordaz(Thanks!)
mreynolds389 added a commit that referenced this issue Sep 12, 2022
Description:  only allow one fixup task to run at a time, and improve
the task logging

relates: #5413

Reviewed by: progier & tbordaz(Thanks!)
@mreynolds389
Copy link
Contributor Author

198d921..fc205ec 389-ds-base-2.2 -> 389-ds-base-2.2
814f8b5..d1b1678 389-ds-base-2.1 -> 389-ds-base-2.1
6083fd1..4401b66 389-ds-base-2.0 -> 389-ds-base-2.0
e665ae0..5dd7043 389-ds-base-1.4.3 -> 389-ds-base-1.4.3

@tbordaz tbordaz reopened this Dec 1, 2022
@tbordaz
Copy link
Contributor

tbordaz commented Dec 1, 2022

The detection of concurrent memberof task should be done consider base/filter.
IPA is doing valid parallel fixup tasks

dn: cn=Update PBAC memberOf $TIME, cn=memberof task, cn=tasks, cn=config
objectClass: top
objectClass: extensibleObject
cn: IPA PBAC memberOf $TIME
basedn: cn=privileges,cn=pbac,$SUFFIX
filter: (objectclass=*)
ttl: 10

dn: cn=Update Role memberOf $TIME, cn=memberof task, cn=tasks, cn=config
objectClass: top
objectClass: extensibleObject
cn: Update Role memberOf $TIME
basedn: cn=roles,cn=accounts,$SUFFIX
filter: (objectclass=*)
ttl: 10

@Firstyear
Copy link
Contributor

This may not be true @tbordaz because IPA has nested groups. so you could have cn=some_priv,cn=privileges,cn=pbac,$SUFFIX which includes a member cn=myrole,cn=roles,cn=accounts,$SUFFIX. Since these are nested, this would trigger the members of cn=myrole to be adjusted too. Since there is also the second fixup in update role, it could conflict on the changes and cause loss of member of records.

I think that nested groups pretty much means you have to consider only one memberof fixup at a time.

@tbordaz
Copy link
Contributor

tbordaz commented Dec 2, 2022

I absolutely agree with your point. With nested groups it is unpredictable if parallel fixup will or not steps on each other. In addition I would say that parallel fixup will increase DB pressure (even if accessing different entries) with a higher risk of retries so from performance pov, there is a possibility that parallel fixups will take more time that serialized one.

Now IMHO it should be in the hand of the administrators/applications to decide/experiment if parallel fixups bring some benefits.

This ticket was initially created because of an administrator launched multiple times by mistake the exact same fixup. So I would rather see this ticket to prevent same base/filter tasks.

@mreynolds389
Copy link
Contributor Author

mreynolds389 commented Dec 13, 2022

Well the real issue I was trying to prevent was allowing "identical" tasks to be run at the same time. But there are legitimate cases where you might want to run fixup tasks on two different branches. Refining fix...

mreynolds389 added a commit to mreynolds389/389-ds-base that referenced this issue Dec 13, 2022
…/filters

Description:

A change was made to only allow a single fixup task at a time, but there are
cases where you would want to run mutliple tasks but on different branches/filters.

Now we maintain a linked list of bases/filters of the current running tasks to
monitor this.

relates: 389ds#5413

Reviewed by: tbordaz(Thanks!)
mreynolds389 added a commit that referenced this issue Dec 13, 2022
…/filters

Description:

A change was made to only allow a single fixup task at a time, but there are
cases where you would want to run mutliple tasks but on different branches/filters.

Now we maintain a linked list of bases/filters of the current running tasks to
monitor this.

relates: #5413

Reviewed by: tbordaz(Thanks!)
mreynolds389 added a commit that referenced this issue Dec 13, 2022
…/filters

Description:

A change was made to only allow a single fixup task at a time, but there are
cases where you would want to run mutliple tasks but on different branches/filters.

Now we maintain a linked list of bases/filters of the current running tasks to
monitor this.

relates: #5413

Reviewed by: tbordaz(Thanks!)
mreynolds389 added a commit that referenced this issue Dec 13, 2022
…/filters

Description:

A change was made to only allow a single fixup task at a time, but there are
cases where you would want to run mutliple tasks but on different branches/filters.

Now we maintain a linked list of bases/filters of the current running tasks to
monitor this.

relates: #5413

Reviewed by: tbordaz(Thanks!)
mreynolds389 added a commit that referenced this issue Dec 13, 2022
…/filters

Description:

A change was made to only allow a single fixup task at a time, but there are
cases where you would want to run mutliple tasks but on different branches/filters.

Now we maintain a linked list of bases/filters of the current running tasks to
monitor this.

relates: #5413

Reviewed by: tbordaz(Thanks!)
mreynolds389 added a commit that referenced this issue Dec 13, 2022
…/filters

Description:

A change was made to only allow a single fixup task at a time, but there are
cases where you would want to run mutliple tasks but on different branches/filters.

Now we maintain a linked list of bases/filters of the current running tasks to
monitor this.

relates: #5413

Reviewed by: tbordaz(Thanks!)
@mreynolds389
Copy link
Contributor Author

216c14d..1091ca9 389-ds-base-2.2 -> 389-ds-base-2.2
36e32f4..9d086f4 389-ds-base-2.1 -> 389-ds-base-2.1
959d36e..70e0175 389-ds-base-2.0 -> 389-ds-base-2.0
08ce159..90a8319 389-ds-base-1.4.3 -> 389-ds-base-1.4.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs triage The issue will be triaged during scrum
Projects
None yet
Development

No branches or pull requests

3 participants