Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add API Metrics Reporting Support to CommonServer #16724

Merged
merged 61 commits into from Jun 8, 2022

Conversation

amshamah419
Copy link
Contributor

@amshamah419 amshamah419 commented Dec 30, 2021

Contributing to Cortex XSOAR Content

Make sure to register your contribution by filling the contribution registration form

The Pull Request will be reviewed only after the contribution registration form is filled.

Status

  • In Progress
  • Ready
  • In Hold - (Reason for hold)

Related Issues

https://github.com/demisto/etc/issues/32115

Description

This PR adds the rate limit handling to CommonServerPython, UrlScan, and to VirusTotal

Minimum version of Cortex XSOAR

  • 7.0.0

Does it break backward compatibility?

  • Yes
    • Further details:
  • No

Must have

  • Tests
  • Documentation

@amshamah419 amshamah419 marked this pull request as draft December 30, 2021 11:54
@amshamah419 amshamah419 self-assigned this Dec 30, 2021
@amshamah419
Copy link
Contributor Author

@yuvalbenshalom @anara123 This was more or less my idea as to how to handle the rate limited requests. We set a new property called ErrorType as RateLimited. Any other error type is not indexed currently (we already log command failures)

…-GSB-CommonServer

� Conflicts:
�	Packs/Base/ReleaseNotes/1_18_0.md
�	Packs/Base/pack_metadata.json
�	Packs/GoogleSafeBrowsing/pack_metadata.json
@amshamah419 amshamah419 changed the title Add Rate Limit Support to GSB and CommonServer Add Rate Limit Support to GSB, CommonServer, and Urlscan Mar 1, 2022
…-GSB-CommonServer

� Conflicts:
�	Packs/UrlScan/Integrations/UrlScan/UrlScan.py
�	Packs/UrlScan/pack_metadata.json
Copy link
Contributor

@yuvalbenshalom yuvalbenshalom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see my comments

@amshamah419
Copy link
Contributor Author

demisto.results([{
            'Type': entryTypes['note'],
            'ContentsFormat': formats['markdown'],
            'Contents': "This is an Error",
            'HumanReadable': f"This command returned 60 ratelimit errors",
            'ErrorType': "RateLimited",
            'APICalls': 60
        }])

Screen Shot 2022-03-08 at 23 49 55

@anara123 @yuvalbenshalom I have the API Call count working, but we really should think of a way to pretty this up.

@lgtm-com
Copy link

lgtm-com bot commented Mar 17, 2022

This pull request introduces 1 alert when merging 6a5d311 into c0aa406 - view on LGTM.com

new alerts:

  • 1 for Unused import

…-GSB-CommonServer

� Conflicts:
�	Packs/Base/ReleaseNotes/1_19_0.md
�	Packs/Base/pack_metadata.json
�	Packs/VirusTotal/ReleaseNotes/2_2_1.md
@amshamah419
Copy link
Contributor Author

@yuvalbenshalom @anara123 Validations fail due to moving UrlScan to Py3, otherwise I believe we are good to go.

@amshamah419 amshamah419 changed the title Add Rate Limit Support to CommonServer, and Urlscan Add API Metrics Reporting Support to CommonServer Jun 7, 2022
@amshamah419 amshamah419 requested a review from tkatzir as a code owner June 8, 2022 13:43
@content-bot
Copy link
Collaborator

@yuvalbenshalom yuvalbenshalom merged commit 6a824c7 into master Jun 8, 2022
@yuvalbenshalom yuvalbenshalom deleted the ratelimiting-GSB-CommonServer branch June 8, 2022 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants