Takes a long time to create incident. #766

sfc-gh-pkommini · 2021-01-12T00:15:42Z

Describe the bug
Create incident is stuck almost for 5-10 minutes without any response.

To Reproduce
This is the setup I have so far:

Empty database without sample data
Slack app and credentials setup
JIRA credentials setup
UI Works but has no data
Entered incident priority fields from UI
Entered Incident type from UI

Expected behavior
The expected behavior is for the incident to be created in a few seconds at most.

Screenshots

Additional context
Setup using ECS fargate and RDS

sfc-gh-pkommini · 2021-01-12T00:41:10Z

May have found the issue in the logs:


ERROR:JiraError HTTP 400 url: https://<redacted>.atlassian.net/rest/api/2/issue
--
text: project is required
 
response headers = {'Server': 'AtlassianProxy/1.15.8.1', 'cache-control': 'no-cache, no-store, no-transform', 'Content-Type': 'application/json;charset=UTF-8', 'Strict-Transport-Security': 'max-age=315360000; includeSubDomains; preload', 'Date': 'Tue, 12 Jan 2021 00:04:48 GMT', 'ATL-TraceId': 'redacted', 'x-arequestid': 'redacted', 'x-aaccountid': 'redacted', 'X-XSS-Protection': '1; mode=block', 'Transfer-Encoding': 'chunked', 'timing-allow-origin': '*', 'x-envoy-upstream-service-time': '48', 'X-Content-Type-Options': 'nosniff', 'Connection': 'close', 'Expect-CT': 'report-uri="https://web-security-reports.services.atlassian.com/expect-ct-report/global-proxy", enforce, max-age=86400'}
response text = {     "errorMessages": [],     "errors": {         "project": "project is required"     } }:/usr/local/lib/python3.8/site-packages/dispatch/decorators.py:wrapper:44
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/decorators.py", line 37, in wrapper
result = func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/dispatch/incident/flows.py", line 426, in incident_create_flow
ticket = create_incident_ticket(incident, db_session)
File "/usr/local/lib/python3.8/site-packages/dispatch/incident/flows.py", line 110, in create_incident_ticket
ticket = plugin.instance.create(
File "/usr/local/lib/python3.8/site-packages/dispatch/decorators.py", line 74, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/dispatch/decorators.py", line 58, in wrapper
result = func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/dispatch/plugins/dispatch_jira/plugin.py", line 147, in create
return create(client, issue_fields, type=JIRA_PROJECT_KEY)
File "/usr/local/lib/python3.8/site-packages/dispatch/plugins/dispatch_jira/plugin.py", line 82, in create
issue = client.create_issue(fields=issue_fields)
File "/usr/local/lib/python3.8/site-packages/jira/client.py", line 1107, in create_issue
r = self._session.post(url, data=json.dumps(data))
File "/usr/local/lib/python3.8/site-packages/jira/resilientsession.py", line 154, in post
return self.__verb('POST', url, **kwargs)
File "/usr/local/lib/python3.8/site-packages/jira/resilientsession.py", line 147, in __verb
raise_on_error(response, verb=verb, **kwargs)
File "/usr/local/lib/python3.8/site-packages/jira/resilientsession.py", line 56, in raise_on_error
raise JIRAError(
jira.exceptions.JIRAError: JiraError HTTP 400 url: https://redacted-test.atlassian.net/rest/api/2/issue
text: project is required

sfc-gh-pkommini · 2021-01-12T01:06:09Z

Would you know what could possibly cause this error?
This is my environment set for JIRA config

JIRA_API_URL=https://company-test.atlassian.net/
JIRA_BROWSER_URL=https://company-test.atlassian.net/
JIRA_ISSUE_TYPE_ID=Incident
JIRA_PROJECT_KEY=dispatch-test

kevgliss · 2021-01-12T01:23:05Z

I think it could be a api version issue:

We are passing {"key": <project-id>} while your API (v2) wants {"id": <project-id>}
Our code:
https://github.com/Netflix/dispatch/blob/master/src/dispatch/plugins/dispatch_jira/plugin.py#L140

Jira Docs: https://developer.atlassian.com/cloud/jira/platform/rest/v2/api-group-issues/#api-rest-api-2-issue-post

Unfortunately, we host our own Jira with a different API so its hard for me to troubleshoot. That plugin was contributed by the community but I'm not sure for which Jira API version.

mvilanova · 2021-01-12T01:45:05Z

Note that JIRA_ISSUE_TYPE_ID is a numerical value. Info on how to find it here: https://confluence.atlassian.com/jirakb/finding-the-id-for-issue-types-646186508.html

sfc-gh-pkommini · 2021-01-12T04:58:03Z

Hi @kevgliss Is it possible to request a change to use id as opposed to using key, It seems that the usage as described here in the jira python library docs uses id instead of key

https://jira.readthedocs.io/en/latest/examples.html#issues

sfc-gh-pkommini · 2021-01-12T04:59:24Z

@mvilanova Also, the issueType here in the recent version seems to require a string as a value to name.

issue_dict = {
    'project': {'id': 123},
    'summary': 'New issue from jira-python',
    'description': 'Look into this one',
    'issuetype': {'name': 'Bug'},
}
new_issue = jira.create_issue(fields=issue_dict)

sfc-gh-pkommini · 2021-01-12T05:21:13Z

I can push the PR for the change. Let me know if you will be able to merge it.

sfc-gh-pkommini · 2021-01-12T06:22:53Z

@kevgliss @mvilanova Another question, could you clarify the difference between the following two config variables?

JIRA_API_URL=https://company-test.atlassian.net/
JIRA_BROWSER_URL=https://company-test.atlassian.net/

Is it okay to pass the same URL value for both the variables?

mvilanova · 2021-01-12T15:56:26Z

The JIRA_API_URL is used by the Jira python client and the JIRA_BROWSER_URL for UI-related purposes (e.g. presenting links to Jira issues to users). If both the Jira Web and API live in the same server, then these variables need to have the same value.

Also, the issueType here in the recent version seems to require a string as a value to name.

That's the issue type not issue type id.

The examples under "You can even bulk create multiple issues:" suggest that you can use id, key, and even name. Before we make any code changes, can you set JIRA_ISSUE_TYPE_ID to the correct id and verify that a Jira project with name dispatch-test exists in the Jira server?

mvilanova · 2021-01-12T16:15:11Z

Looking at the Jira create_issue code, the client resolves the project id from key using the project function and uses issue type id if you provide the id or resolves it to an id if you don't.

        p = data["fields"]["issuetype"]
        if isinstance(p, int):
            data["fields"]["issuetype"] = {"id": p}
        if isinstance(p, str) or isinstance(p, int):
            data["fields"]["issuetype"] = {"id": self.issue_type_by_name(p).id}

sfc-gh-pkommini · 2021-01-12T21:49:35Z

The examples under "You can even bulk create multiple issues:" suggest that you can use id, key, and even name. Before we make any code changes, can you set JIRA_ISSUE_TYPE_ID to the correct id and verify that a Jira project with name dispatch-test exists in the Jira server?

I can confirm that the project exists and I pass that value. From that code it does seem like id, key and name are accepted for project. But it doesn't demonstrate the same for issueType. Should we make the assumption that it accepts, id, key and name for issueType as well?

sfc-gh-pkommini · 2021-01-12T21:56:46Z

I'll correct the issueType to numeric value of the ID and report back. Give me 5 minutes.

sfc-gh-pkommini · 2021-01-12T22:10:03Z

@mvilanova

Also, I got the following warning messge, would you know a possible fix?

WARNING:Attempted to fetch active plugin, but none were found. PluginType: oncall:/usr/local/lib/python3.8/site-packages/dispatch/plugin/service.py:get_active:30

And I tried a fix and got this error message:

ERROR:Unable to find slug: jira-ticket in self.all version 1: <generator object PluginManager.all at 0x7f8254353970> or version 2: <generator object PluginManager.all at 0x7f8254353970>:/usr/local/lib/python3.8/site-packages/dispatch/plugins/base/manager.py:get:38

I think I maybe missing something related to JIRA ticket.

sfc-gh-pkommini · 2021-01-12T22:13:49Z

Full error message:

ERROR:'jira-ticket':/usr/local/lib/python3.8/site-packages/dispatch/decorators.py:wrapper:44
--
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/decorators.py", line 37, in wrapper
result = func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/dispatch/incident/flows.py", line 426, in incident_create_flow
ticket = create_incident_ticket(incident, db_session)
File "/usr/local/lib/python3.8/site-packages/dispatch/incident/flows.py", line 110, in create_incident_ticket
ticket = plugin.instance.create(
File "/usr/local/lib/python3.8/site-packages/dispatch/plugin/models.py", line 30, in instance
return plugins.get(self.slug)
File "/usr/local/lib/python3.8/site-packages/dispatch/plugins/base/manager.py", line 41, in get
raise KeyError(slug)
KeyError: 'jira-ticket'

mvilanova · 2021-01-13T01:18:23Z

@sfc-gh-pkommini did you run dispatch plugins install? Can you paste the output of dispatch plugins list here?

sfc-gh-pkommini · 2021-01-13T01:37:14Z

@mvilanova @kevgliss

I was able to create issue from API using the jira python library version 2.0.0. (As version is non pinned this is the version that gets installed)

I had the following observations:

For project: key, name doesn't work as a parameter to the library, only id works when not using bulk create.
excerpt from CLI, using key

>>> client
<jira.client.JIRA object at 0x106650af0>
>>> 
>>> issue_fields = {
...     "project": {"key": 'dispatch-test'},
...     "issuetype": {"name": 'Incident'},
...     "summary": "This is very bad",
...     "assignee": {"name": "user.name"},
...     "reporter": {"name": "user.name"}
... }
>>> client.create_issue(fields=issue_fields)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/redacted/workspace/dispatch/venv/lib/python3.8/site-packages/jira/client.py", line 1107, in create_issue
    r = self._session.post(url, data=json.dumps(data))
  File "/Users/redacted/workspace/dispatch/venv/lib/python3.8/site-packages/jira/resilientsession.py", line 154, in post
    return self.__verb('POST', url, **kwargs)
  File "/Users/redacted/workspace/dispatch/venv/lib/python3.8/site-packages/jira/resilientsession.py", line 147, in __verb
    raise_on_error(response, verb=verb, **kwargs)
  File "/Users/redacted/workspace/dispatch/venv/lib/python3.8/site-packages/jira/resilientsession.py", line 56, in raise_on_error
    raise JIRAError(
jira.exceptions.JIRAError: JiraError HTTP 400 url: https://jira.atlassian.net/rest/api/2/issue
        text: project is required

        response headers = {'Server': 'AtlassianProxy/1.15.8.1', 'cache-control': 'no-cache, no-store, no-transform', 'Content-Type': 'application/json;charset=UTF-8', 'Strict-Transport-Security': 'max-age=315360000; includeSubDomains; preload', 'Date': 'Wed, 13 Jan 2021 00:30:02 GMT', 'ATL-TraceId': '<redacted>', 'x-arequestid': '<redacted>', 'x-aaccountid': '<redacted>', 'X-XSS-Protection': '1; mode=block', 'Transfer-Encoding': 'chunked', 'timing-allow-origin': '*', 'x-envoy-upstream-service-time': '40', 'X-Content-Type-Options': 'nosniff', 'Connection': 'close', 'Expect-CT': 'report-uri="https://web-security-reports.services.atlassian.com/expect-ct-report/global-proxy", enforce, max-age=86400'}
        response text = {"errorMessages":[],"errors":{"project":"project is required"}}

excerpt from CLI, using name for project

>>> issue_fields = {
...     "project": {"name": 'dispatch-test'},
...     "issuetype": {"id": '10010'},
...     "summary": "This is very bad",
...     "assignee": {"name": "user.name"},
...     "reporter": {"name": "user.name"}
... }
>>> client.create_issue(fields=issue_fields)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/redacted/workspace/dispatch/venv/lib/python3.8/site-packages/jira/client.py", line 1107, in create_issue
    r = self._session.post(url, data=json.dumps(data))
  File "/Users/redacted/workspace/dispatch/venv/lib/python3.8/site-packages/jira/resilientsession.py", line 154, in post
    return self.__verb('POST', url, **kwargs)
  File "/Users/redacted/workspace/dispatch/venv/lib/python3.8/site-packages/jira/resilientsession.py", line 147, in __verb
    raise_on_error(response, verb=verb, **kwargs)
  File "/Users/redacted/workspace/dispatch/venv/lib/python3.8/site-packages/jira/resilientsession.py", line 56, in raise_on_error
    raise JIRAError(
jira.exceptions.JIRAError: JiraError HTTP 400 url: https://jira.atlassian.net/rest/api/2/issue
        text: project is required

        response headers = {'Server': 'AtlassianProxy/1.15.8.1', 'cache-control': 'no-cache, no-store, no-transform', 'Content-Type': 'application/json;charset=UTF-8', 'Strict-Transport-Security': 'max-age=315360000; includeSubDomains; preload', 'Date': 'Wed, 13 Jan 2021 00:33:43 GMT', 'ATL-TraceId': 'redacted', 'x-arequestid': 'redacted', 'x-aaccountid': 'redacted', 'X-XSS-Protection': '1; mode=block', 'Transfer-Encoding': 'chunked', 'timing-allow-origin': '*', 'x-envoy-upstream-service-time': '57', 'X-Content-Type-Options': 'nosniff', 'Connection': 'close', 'Expect-CT': 'report-uri="https://web-security-reports.services.atlassian.com/expect-ct-report/global-proxy", enforce, max-age=86400'}
        response text = {"errorMessages":[],"errors":{"project":"project is required"}}

For issueType, id and name both work (I think name is better than id as admins/non-admins have access and is human readable), key doesn't work.
There have been a lot of changes to the JIRA API over the last year. Specifically the GDPR mode which doesn't doesn't support username fields.

Let me know if these tests are good enough to warrant the changes for project to use id and issueType to use name? Also, I'm also adding a helper method to help with getting accountId for reporter and assignee and then using that to set issue_fields with accountId instead of name for reporter and assignee as documented in the changes due to GDPR shown below provided in the link above.

mvilanova · 2021-01-13T01:46:53Z

Cool, let's do it. Thanks for testing and debugging.

sfc-gh-pkommini · 2021-01-13T18:05:14Z

@kevgliss @mvilanova The PR is created.

I'm still getting the below error. Would you know what could cause such an error. I think I'm missing a config variable but I don't know what it could be.


ERROR:Unable to find slug: jira-ticket in self.all version 1: <generator object PluginManager.all at 0x7f91c4129f90> or version 2: <generator object PluginManager.all at 0x7f91c4129f90>:/usr/local/lib/python3.8/site-packages/dispatch/plugins/base/manager.py:get:38
--
ERROR:'jira-ticket':/usr/local/lib/python3.8/site-packages/dispatch/decorators.py:wrapper:44
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/decorators.py", line 37, in wrapper
result = func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/dispatch/incident/flows.py", line 426, in incident_create_flow
ticket = create_incident_ticket(incident, db_session)
File "/usr/local/lib/python3.8/site-packages/dispatch/incident/flows.py", line 110, in create_incident_ticket
ticket = plugin.instance.create(
File "/usr/local/lib/python3.8/site-packages/dispatch/plugin/models.py", line 30, in instance
return plugins.get(self.slug)
File "/usr/local/lib/python3.8/site-packages/dispatch/plugins/base/manager.py", line 41, in get
raise KeyError(slug)
KeyError: 'jira-ticket'

mvilanova · 2021-01-13T18:17:14Z

Not sure if you saw my questions here. I'd like to see if plugins are installed and the jira one enabled.

sfc-gh-pkommini · 2021-01-13T18:21:05Z

@mvilanova Oh Sorry. I missed that question. I believe I did run the plugins install. Let me double check. I configured it as part of the web container. Unfortunately, I can't run commands on our containers as they run on ECS Fargate.

Could I query a database table to check?

mvilanova · 2021-01-13T18:23:41Z

No worries. You can also see the list of plugins and whether they're enabled or not in /plugins in the Web UI. I should have time this afternoon to take a look at the PR.

sfc-gh-pkommini · 2021-01-13T20:24:56Z

@kevgliss @mvilanova I see the following plugins enabled from the UI. I did a dispatch plugins install while starting the container and then later I manually enabled these from the UI.

sfc-gh-pkommini · 2021-01-13T20:27:48Z

I disabled everything except the first two plugins (Slack Plugin - Conversation..., Slack Plugin - Contact Info..) and then tried creating an incident and it caused the following error:


ERROR:'NoneType' object has no attribute 'lower':/usr/local/lib/python3.8/site-packages/dispatch/incident/flows.py:incident_create_flow:627
--
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dispatch/incident/flows.py", line 599, in incident_create_flow
conversation = create_conversation(incident, participant_emails, db_session)
File "/usr/local/lib/python3.8/site-packages/dispatch/incident/flows.py", line 344, in create_conversation
conversation = plugin.instance.create(incident.name, participants)
File "/usr/local/lib/python3.8/site-packages/dispatch/decorators.py", line 74, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/dispatch/decorators.py", line 58, in wrapper
result = func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/dispatch/plugins/dispatch_slack/plugin.py", line 92, in create
return create_conversation(self.client, name, participants, is_private)
File "/usr/local/lib/python3.8/site-packages/dispatch/plugins/dispatch_slack/service.py", line 272, in create_conversation
name=name.lower(),  # slack disallows upperCase
AttributeError: 'NoneType' object has no attribute 'lower'
WARNING:Attempted to fetch active plugin, but none were found. PluginType: ticket:/usr/local/lib/python3.8/site-packages/dispatch/plugin/service.py:get_active:30

Do we need the contact plugin for create incident to work?

kevgliss · 2021-01-13T21:10:57Z

Weird, it looks like it can't find your jira plugin. This is the query we use:
https://github.com/Netflix/dispatch/blob/master/src/dispatch/plugin/service.py#L22

Which according to your screen shot looks like it should resolve?

One thing we could also try is to disable the jira ticket plugin and enable the dispatch jira plugin.

If you look at the server startup logs, it should point to any issues loading the jira plugins.

sfc-gh-pkommini · 2021-01-13T21:11:56Z

I'll check on that and report my findings. Thank you. :)

sfc-gh-pkommini · 2021-01-13T21:30:03Z

Okay. I think I found the issue. It seems like the jira plugin failed to load due to issue in the Program ID. Forgot to makes the PR changes on my container.

Also, I get an error message regarding the channel not found.

ERROR:SlackError. Response: {'ok': False, 'error': 'channel_not_found'} Endpoint: chat.postMessage kwargs: {'channel': 'incidents', 'text': 'Incident Notification', 'blocks': [{'type': 'divider'}, {'type': 'section', 'text': {'type': 'mrkdwn', 'text': '*Incident Title*\nbad bad bad'}}, {'type': 'section', 'text': {'type': 'mrkdwn', 'text': '*Incident Status - Active*\nThis incident is under active investigation.'}}, {'type': 'section', 'text': {'type': 'mrkdwn', 'text': '*Incident Type - Vulnerability*\nThis is an incident involving a misconfiguration or vulnerability.\t'}}, {'type': 'section', 'text': {'type': 'mrkdwn', 'text': '*Incident Priority - Medium*\nPriority for medium level incidents.'}}, {'type': 'section', 'text': {'type': 'mrkdwn', 'text': '*<Unknown\|Incident Commander - <Redacted>*\nThe Incident Commander (IC) is responsible for knowing the full context of the incident. Contact them about any questions or concerns.'}}]}:/usr/local/lib/python3.8/site-packages/dispatch/plugins/dispatch_slack/service.py:make_call:110
--
ERROR:RetryError[<Future at 0x7f91bed611c0 state=finished raised TryAgain>]:/usr/local/lib/python3.8/site-packages/dispatch/decorators.py:wrapper:44

Are we expected to create a channel named #incidents? I didn't see this anywhere in the config. Or is this message for dispatch-help channel in the global config?

sfc-gh-pkommini · 2021-01-13T21:35:57Z

Again, I think I found the issue, I believe it's this config variable that was causing the problem. I had `incidents' there which I didn't create the channel for.

mvilanova · 2021-01-13T21:59:17Z

Yeah, that would do it. Re: the following error, seems like the plugin code failed, because the incident didn't have a name.

File "/usr/local/lib/python3.8/site-packages/dispatch/plugins/dispatch_slack/service.py", line 272, in create_conversation
name=name.lower(), # slack disallows upperCase
AttributeError: 'NoneType' object has no attribute 'lower'

sfc-gh-pkommini · 2021-01-13T22:28:11Z

Another error message which I was able to fix but thought I'll mention here for future users. If we have INCIDENT_NOTIFICATION_CONVERSATIONS set, then the bot user has to be added to that channel. Else we'll get the below error message.

ERROR:SlackError. Response: {'ok': False, 'error': 'not_in_channel'} Endpoint: chat.postMessage kwargs: {'channel': 'incidents', 'text': 'Incident Notification', 'blocks': [{'type': 'divider'}, {'type': 'section', 'text': {'type': 'mrkdwn', 'text': '*<https://redacted-test.atlassian.net//browse/DT-14\|DT-14 Incident Notification>*\nThis message is for notification purposes only.'}, 'block_id': <ConversationButtonActions.invite_user: 'invite-user'>, 'accessory': {'type': 'button', 'text': {'type': 'plain_text', 'text': 'Join Incident'}, 'value': '6'}}, {'type': 'section', 'text': {'type': 'mrkdwn', 'text': '*Incident Title*\nbad bad bad'}}, {'type': 'section', 'text': {'type': 'mrkdwn', 'text': '*Incident Status - Active*\nThis incident is under active investigation.'}}, {'type': 'section', 'text': {'type': 'mrkdwn', 'text': '*Incident Type - Vulnerability*\nThis is an incident involving a misconfiguration or vulnerability.\t'}}, {'type': 'section', 'text': {'type': 'mrkdwn', 'text': '*Incident Priority - Medium*\nPriority for medium level incidents.'}}, {'type': 'section', 'text': {'type': 'mrkdwn', 'text': '*<Unknown\|Incident Commander - redacted>*\nThe Incident Commander (IC) is responsible for knowing the full context of the incident. Contact them about any questions or concerns.'}}]}:/usr/local/lib/python3.8/site-packages/dispatch/plugins/dispatch_slack/service.py:make_call:110
--

sfc-gh-pkommini · 2021-01-13T22:33:00Z

Thank you folks for help me out with this. I was able to fix the issue and create incidents successfully.

rilutham · 2021-01-24T07:35:35Z

@mvilanova I have the same issue where the incident does not have a name after creation and also cannot create slack conversation

mvilanova · 2021-01-24T14:30:29Z

@rilutham can you open a separate issue and provide server logs? We can take a look at it on Monday.

rilutham · 2021-01-25T10:37:12Z

@mvilanova already open a new issue #793 . Thank you for your help.

sfc-gh-pkommini closed this as completed Jan 13, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Takes a long time to create incident. #766

Takes a long time to create incident. #766

sfc-gh-pkommini commented Jan 12, 2021

sfc-gh-pkommini commented Jan 12, 2021 •

edited

sfc-gh-pkommini commented Jan 12, 2021

kevgliss commented Jan 12, 2021 •

edited

mvilanova commented Jan 12, 2021

sfc-gh-pkommini commented Jan 12, 2021 •

edited

sfc-gh-pkommini commented Jan 12, 2021 •

edited

sfc-gh-pkommini commented Jan 12, 2021

sfc-gh-pkommini commented Jan 12, 2021

mvilanova commented Jan 12, 2021

mvilanova commented Jan 12, 2021 •

edited

sfc-gh-pkommini commented Jan 12, 2021 •

edited

sfc-gh-pkommini commented Jan 12, 2021

sfc-gh-pkommini commented Jan 12, 2021 •

edited

sfc-gh-pkommini commented Jan 12, 2021 •

edited

mvilanova commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021 •

edited

mvilanova commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021

mvilanova commented Jan 13, 2021 •

edited

sfc-gh-pkommini commented Jan 13, 2021

mvilanova commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021 •

edited

kevgliss commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021 •

edited

mvilanova commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021 •

edited

rilutham commented Jan 24, 2021

mvilanova commented Jan 24, 2021

rilutham commented Jan 25, 2021

Takes a long time to create incident. #766

Takes a long time to create incident. #766

Comments

sfc-gh-pkommini commented Jan 12, 2021

sfc-gh-pkommini commented Jan 12, 2021 • edited

sfc-gh-pkommini commented Jan 12, 2021

kevgliss commented Jan 12, 2021 • edited

mvilanova commented Jan 12, 2021

sfc-gh-pkommini commented Jan 12, 2021 • edited

sfc-gh-pkommini commented Jan 12, 2021 • edited

sfc-gh-pkommini commented Jan 12, 2021

sfc-gh-pkommini commented Jan 12, 2021

mvilanova commented Jan 12, 2021

mvilanova commented Jan 12, 2021 • edited

sfc-gh-pkommini commented Jan 12, 2021 • edited

sfc-gh-pkommini commented Jan 12, 2021

sfc-gh-pkommini commented Jan 12, 2021 • edited

sfc-gh-pkommini commented Jan 12, 2021 • edited

mvilanova commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021 • edited

mvilanova commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021

mvilanova commented Jan 13, 2021 • edited

sfc-gh-pkommini commented Jan 13, 2021

mvilanova commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021 • edited

kevgliss commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021 • edited

mvilanova commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021

sfc-gh-pkommini commented Jan 13, 2021 • edited

rilutham commented Jan 24, 2021

mvilanova commented Jan 24, 2021

rilutham commented Jan 25, 2021

sfc-gh-pkommini commented Jan 12, 2021 •

edited

kevgliss commented Jan 12, 2021 •

edited

sfc-gh-pkommini commented Jan 12, 2021 •

edited

sfc-gh-pkommini commented Jan 12, 2021 •

edited

mvilanova commented Jan 12, 2021 •

edited

sfc-gh-pkommini commented Jan 12, 2021 •

edited

sfc-gh-pkommini commented Jan 12, 2021 •

edited

sfc-gh-pkommini commented Jan 12, 2021 •

edited

sfc-gh-pkommini commented Jan 13, 2021 •

edited

mvilanova commented Jan 13, 2021 •

edited

sfc-gh-pkommini commented Jan 13, 2021 •

edited

sfc-gh-pkommini commented Jan 13, 2021 •

edited

sfc-gh-pkommini commented Jan 13, 2021 •

edited