[BUG] Super event throwing error in production

[sandeep-ps]

Describe the bug
Publishing a super event throwing error in events manager PROD instance.

Initially reported by Vanessa.
The super event title is "IlliniFest: Campus-Wide Open House for Prospective Students". Approx created time :
Created for group " Enrollment Management Event Admins". This super event was linked to around 80 subevents.

Screenshot of some subevents linked to that superevent :

Describing what I understand the user might have done to create these events :
The user created all the subevents first, just "added " them and not "publish". Then the user created the superevent, "added" it and not publish. Linked all the subevents and then published. This would ideally result in publishing both the superevent and all subevents.
The events have a physical location and some subevents have end time after the super event end time. Some events have "&" in the title. All events have category "Campus Visits".

The user tried to publish the super event at approx 8:20 AM on Sept 14th.

Looking to get the PROD logs for more debugging.

To Reproduce
I tried creating and publishing a super event linked with subevents with the event details mentioned above and I was able to publish the events without error. Hence this issue might be specific to that event.

Expected behavior
User is able to publish the superevent with all (approx 80+) subevents.

[minump]

@pascoem @pmarkhennessy Was wondering if you can share Events Manager and EventsBB prod instance logs for the time period Sept 14th 8:00 AM to 8:30 AM?

[minump]

Mike has shared the eventsBB and events manager PROD logs in Slack message. Thankyou.

[bingzhang]

@minump shall we link the logs to this issue, please?

[minump]

Events_091423.csv
Events_Manager_091423.csv

Sharing the logs Mike shared in Slack. However, I see that the "Message" column is missing in both. I have requested Mike for the same. Thank you.

[minump]

1694715202_360306_898CD4DA-CBBD-45B1-807D-90D5F98FE851.csv
1694714511_360025_898CD4DA-CBBD-45B1-807D-90D5F98FE851.csv

Log files from events manager and events BB prod instance. the "_raw" field has the desired logs
@bingzhang

[minump]

On checking out the events manager logs, I see a bunch of "event submission fails" messages. This could be because the publishing of the super event failed (https://github.com/rokwire/events-manager/blob/develop/utilities/user_utilities.py#L396) . There is also a bunch of corresponding GET requests in eventsBB with query params "targetAudience", "startDate" "endDate" etc from here . When the POST request failed, the event status is set to "Pending".

On checking the eventsBB aws logs, I do not see POST requests. Maybe these were missed / outside of the time period.

[minump]

This issue is observed on Sept 14th and not on Aug 9th.

[minump]

@vburgett Can me and @bingzhang be added as "admin" roles for the group " Enrollment Management Event Admins"? Asking this because we both are able to publish super events and link subevents in PROD instance. Looking at the logs we see some "event submission failed" responses. For more details we would like to look at the events itself. We wil be able to access the events only if we are part of that group.

If a user has "member" role, that user will not be able to see "pending" events.

[vburgett]

@minump @bingzhang sure. Please request membership and I will make you admins.

[vburgett]

@minump you are now an admin of the Group.

To restate the issues that need solving—1) the actual Group admins need to be able to publish (Laura) and 2) we need to remove the multiple instances of this super event from the app.

[bingzhang]

I checked on the events manager in prod. this reported super event has been published correctly. Please check and consider to close it if it has been fixed.

[bingzhang]

remove @vburgett because this issue is targeted to the developer.

[minump]

This issue has been investigated more. We deployed a hot fix for events manager which captures more logs when a failure happens. One particualr user is facing an authentication issue when publishing an event in PROD instance. Need to check more in eventsBB auth-middleware.

[bingzhang]

@minump I created a issue in auth middleware at rokwire/rokwire-building-blocks-api#1197

[minump]

The actual cause of the user permission issue is not yet clear. This bug is specific to a particular user, as multiple other users (within and outside of Rokwire) are able to create and publish both super events and normal events.

Closing this issue as no further resolutions can be discovered. Closing this issue as "wont fix".