Concurrent auth failure due to duplicate DB write conflict

[m4dcoder]

With RBAC enabled, when there are multiple concurrent auth operations, the auth fails because RBACRemoteGroupToRoleSyncer sync operation error on duplicate DB write conflict.

ERROR base [-] Conflict while trying to save in DB.Traceback (most recent call last):File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/persistence/base.py", line 173, in add_or_updatemodel_object = cls._get_impl().add_or_update(model_object)
File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/models/db/__init__.py", line 314, in add_or_updateinstance.save()
File "/opt/stackstorm/st2/lib/python2.7/site-packages/mongoengine/document.py", line 408,
NotUniqueError(message % six.text_type(err))NotUniqueError: Tried to save duplicate unique keys (E11000 duplicate key error collection: st2.user_role_assignment_d_b index: role_1_user_1 dup key: { : "base_permissions", : "foobar" })
ERROR handlers [-] Failed to synchronize remote groups for user "foobar"
Traceback (most recent call last):File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2auth/handlers.py", line 202, in handle_auth syncer.sync(user_db=user_db, groups=user_groups) File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/rbac/syncer.py", line 423, in sync is_remote=True)
File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/services/rbac.py", line 207, in assign_role_to_userrole_assignment_db = UserRoleAssignment.add_or_update(role_assignment_db)
File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/persistence/base.py", line 185, in add_or_updatemodel_object=model_object)
StackStormDBObjectConflictError: Tried to save duplicate unique keys (E11000 duplicate key error collection: st2.user_role_assignment_d_b index: role_1_user_1 dup key

[m4dcoder]

This DB conflict occurs at https://github.com/StackStorm/st2/blob/v2.3/st2common/st2common/rbac/syncer.py#L421.

[Kami]

With multiple concurrent auth operations I assume you mean multiple invocations of st2-apply-rbac-definitions run in parallel.

Edit: Never mind, it's related to LDAP mapping functionality (but same could potentially also happen if multiple st2-apply-rbac-definitions invocations run in parallel, although that's not a supported use case / not a good idea).

In any case, thanks for reporting and digging in, I will have a look.