sendRoleSearchRequest returns role doesn't exist for some roles

[sdgun]

Steps
Go to https://platform.topcoder-dev.com/taas/myteams/createnewteam
Click Select on the SELECT A ROLE box
Select Role 'support Engineer' and search
Check the sendRoleSearchRequest API response

Current behavior
Shows a blank page

sendRoleSearchRequest API response returns message: "id: 8c76a436-3e39-4c09-8c88-6983ebd5bcc1 "Role" doesn't exists."

Console error observed:

2021-06-21_13-31-28.mp4

[eisbilir]

@nikolay83
Role table in the UI gets the roles from ES and when we search roles via taas-teams/sendRoleSearchRequest it uses DB
Right now, Support Engineer role is only exists in ES. So it causes that blank page after role search request is sent.

Reindexing is needed to fix this issue.

[urwithat]

The mentioned role exists in ES and not in DB.
I’ve synched the roles now on the dev environment, why did the data get out of sync on ES vs DB!
Note: While investigating I found 9 active roles in DB vs 6 in ES, was this due to Bus Api being down? Needs to be investigated.