Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

slack event collector: fixed an issue where the oldest timestamp was newer from the cursor i… #31528

Merged
merged 5,113 commits into from Dec 27, 2023

Conversation

moishce
Copy link
Contributor

@moishce moishce commented Dec 18, 2023

Status

  • In Progress
  • Ready
  • In Hold - (Reason for hold)

Related Issues

fixes: link to the issue

Description

fixed an issue where the oldest timestamp was newer than the cursor in the request, because we saved only the cursor in the last run.

Must have

  • Tests
  • Documentation

content-bot and others added 30 commits November 29, 2023 21:08
* Updated Metadata Of Pack Armorblox

* Added release notes to pack Armorblox

* Packs/Armorblox/Integrations/Armorblox/Armorblox.yml Docker image update
* Updated Metadata Of Pack Intezer

* Added release notes to pack Intezer

* Packs/Intezer/Integrations/IntezerV2/IntezerV2.yml Docker image update

* Updated Metadata Of Pack Zabbix

* Added release notes to pack Zabbix

* Packs/Zabbix/Integrations/Zabbix/Zabbix.yml Docker image update

* Updated Metadata Of Pack FeedMalwareBazaar

* Added release notes to pack FeedMalwareBazaar

* Packs/FeedMalwareBazaar/Integrations/MalwareBazaarFeed/MalwareBazaarFeed.yml Docker image update

* Updated Metadata Of Pack FeedGCPWhitelist

* Added release notes to pack FeedGCPWhitelist

* Packs/FeedGCPWhitelist/Integrations/FeedGoogleIPRanges/FeedGoogleIPRanges.yml Docker image update

* Updated Metadata Of Pack AccentureCTI_Feed

* Added release notes to pack AccentureCTI_Feed

* Packs/AccentureCTI_Feed/Integrations/ACTIIndicatorFeed/ACTIIndicatorFeed.yml Docker image update

* Updated Metadata Of Pack SEKOIAIntelligenceCenter

* Added release notes to pack SEKOIAIntelligenceCenter

* Packs/SEKOIAIntelligenceCenter/Integrations/SEKOIAIntelligenceCenter/SEKOIAIntelligenceCenter.yml Docker image update

* Updated Metadata Of Pack JARM

* Added release notes to pack JARM

* Packs/JARM/Integrations/JARM/JARM.yml Docker image update

* Updated Metadata Of Pack CommonWidgets

* Added release notes to pack CommonWidgets

* Packs/CommonWidgets/Scripts/RSSWidget/RSSWidget.yml Docker image update

* Updated Metadata Of Pack FiltersAndTransformers

* Added release notes to pack FiltersAndTransformers

* Packs/FiltersAndTransformers/Scripts/Jmespath/Jmespath.yml Docker image update
* Updated Metadata Of Pack OracleCloudInfrastructure

* Added release notes to pack OracleCloudInfrastructure

* Packs/OracleCloudInfrastructure/Integrations/OracleCloudInfrastructureEventCollector/OracleCloudInfrastructureEventCollector.yml Docker image update
* Updated Metadata Of Pack Exterro

* Added release notes to pack Exterro

* Packs/Exterro/Integrations/Exterro/Exterro.yml Docker image update

* Fix DS108

---------

Co-authored-by: israelpolishook <ipolishuk@paloaltonetworks.com>
* Updated Metadata Of Pack CarbonBlackDefense

* Added release notes to pack CarbonBlackDefense

* Packs/CarbonBlackDefense/Integrations/CarbonBlackLiveResponseCloud/CarbonBlackLiveResponseCloud.yml Docker image update
* Updated Metadata Of Pack FeedUnit42v2

* Added release notes to pack FeedUnit42v2

* Packs/FeedUnit42v2/Integrations/FeedUnit42v2/FeedUnit42v2.yml Docker image update
* Updated Metadata Of Pack AzureKeyVault

* Added release notes to pack AzureKeyVault

* Packs/AzureKeyVault/Integrations/AzureKeyVault/AzureKeyVault.yml Docker image update

* Updated Metadata Of Pack AzureSentinel

* Added release notes to pack AzureSentinel

* Packs/AzureSentinel/Integrations/AzureSentinel/AzureSentinel.yml Docker image update

* Updated Metadata Of Pack AzureDevOps

* Added release notes to pack AzureDevOps

* Packs/AzureDevOps/Integrations/AzureDevOps/AzureDevOps.yml Docker image update

* Updated Metadata Of Pack MicrosoftCloudAppSecurity

* Added release notes to pack MicrosoftCloudAppSecurity

* Packs/MicrosoftCloudAppSecurity/Integrations/MicrosoftCloudAppSecurity/MicrosoftCloudAppSecurity.yml Docker image update

* Updated Metadata Of Pack AzureRiskyUsers

* Added release notes to pack AzureRiskyUsers

* Packs/AzureRiskyUsers/Integrations/AzureRiskyUsers/AzureRiskyUsers.yml Docker image update

* Updated Metadata Of Pack MicrosoftGraphGroups

* Added release notes to pack MicrosoftGraphGroups

* Packs/MicrosoftGraphGroups/Integrations/MicrosoftGraphGroups/MicrosoftGraphGroups.yml Docker image update

* Updated Metadata Of Pack AzureSQLManagement

* Added release notes to pack AzureSQLManagement

* Packs/AzureSQLManagement/Integrations/AzureSQLManagement/AzureSQLManagement.yml Docker image update

* Updated Metadata Of Pack MicrosoftGraphAPI

* Added release notes to pack MicrosoftGraphAPI

* Packs/MicrosoftGraphAPI/Integrations/MicrosoftGraphAPI/MicrosoftGraphAPI.yml Docker image update

* Updated Metadata Of Pack MicrosoftTeams

* Added release notes to pack MicrosoftTeams

* Packs/MicrosoftTeams/Integrations/MicrosoftTeamsManagement/MicrosoftTeamsManagement.yml Docker image update

* Updated Metadata Of Pack MicrosoftGraphApplications

* Added release notes to pack MicrosoftGraphApplications

* Packs/MicrosoftGraphApplications/Integrations/MicrosoftGraphApplications/MicrosoftGraphApplications.yml Docker image update
* Updated Metadata Of Pack OPNSense

* Added release notes to pack OPNSense

* Packs/OPNSense/Integrations/OPNSense/OPNSense.yml Docker image update
* Updated Metadata Of Pack Cylance_Protect

* Added release notes to pack Cylance_Protect

* Packs/Cylance_Protect/Integrations/Cylance_Protect_v2/Cylance_Protect_v2.yml Docker image update

* Updated Metadata Of Pack Zoom

* Added release notes to pack Zoom

* Packs/Zoom/Integrations/ZoomEventCollector/ZoomEventCollector.yml Docker image update

* Updated Metadata Of Pack Silverfort

* Added release notes to pack Silverfort

* Packs/Silverfort/Integrations/Silverfort/Silverfort.yml Docker image update

* Updated Metadata Of Pack AzureDataExplorer

* Added release notes to pack AzureDataExplorer

* Packs/AzureDataExplorer/Integrations/AzureDataExplorer/AzureDataExplorer.yml Docker image update

* Updated Metadata Of Pack MicrosoftManagementActivity

* Added release notes to pack MicrosoftManagementActivity

* Packs/MicrosoftManagementActivity/Integrations/MicrosoftManagementActivity/MicrosoftManagementActivity.yml Docker image update

* Updated Metadata Of Pack Box

* Added release notes to pack Box

* Packs/Box/Integrations/BoxEventsCollector/BoxEventsCollector.yml Docker image update

* Packs/Box/Integrations/BoxV2/BoxV2.yml Docker image update

* Updated Metadata Of Pack Troubleshoot

* Added release notes to pack Troubleshoot

* Packs/Troubleshoot/Scripts/CertificatesTroubleshoot/CertificatesTroubleshoot.yml Docker image update

* commit

---------

Co-authored-by: israelpolishook <ipolishuk@paloaltonetworks.com>
* Updated Metadata Of Pack Whois

* Added release notes to pack Whois

* Packs/Whois/Integrations/Whois/Whois.yml Docker image update
* Updated Metadata Of Pack QualysFIM

* Added release notes to pack QualysFIM

* Packs/QualysFIM/Integrations/QualysFIM/QualysFIM.yml Docker image update

* Updated Metadata Of Pack FortiSIEM

* Added release notes to pack FortiSIEM

* Packs/FortiSIEM/Integrations/FortiSIEMV2/FortiSIEMV2.yml Docker image update

* Updated Metadata Of Pack FreshworksFreshservice

* Added release notes to pack FreshworksFreshservice

* Packs/FreshworksFreshservice/Integrations/FreshworksFreshservice/FreshworksFreshservice.yml Docker image update

* Updated Metadata Of Pack KnowBe4_KMSAT

* Added release notes to pack KnowBe4_KMSAT

* Packs/KnowBe4_KMSAT/Integrations/KnowBe4KMSATEventCollector/KnowBe4KMSATEventCollector.yml Docker image update

* Packs/KnowBe4_KMSAT/Integrations/KnowBe4KMSAT/KnowBe4KMSAT.yml Docker image update

* Updated Metadata Of Pack SafeNet_Trusted_Access

* Added release notes to pack SafeNet_Trusted_Access

* Packs/SafeNet_Trusted_Access/Integrations/SafeNetTrustedAccessEventCollector/SafeNetTrustedAccessEventCollector.yml Docker image update

* Updated Metadata Of Pack DelineaSS

* Added release notes to pack DelineaSS

* Packs/DelineaSS/Integrations/DelineaSS/DelineaSS.yml Docker image update

* Updated Metadata Of Pack Cryptocurrency

* Added release notes to pack Cryptocurrency

* Packs/Cryptocurrency/Integrations/Cryptocurrency/Cryptocurrency.yml Docker image update

* Updated Metadata Of Pack PANOSPolicyOptimizer

* Added release notes to pack PANOSPolicyOptimizer

* Packs/PANOSPolicyOptimizer/Integrations/PANOSPolicyOptimizer/PANOSPolicyOptimizer.yml Docker image update

* Updated Metadata Of Pack DeveloperTools

* Added release notes to pack DeveloperTools

* Packs/DeveloperTools/Integrations/CreateIncidents/CreateIncidents.yml Docker image update
* Updated Metadata Of Pack SecurityIntelligenceServicesFeed

* Added release notes to pack SecurityIntelligenceServicesFeed

* Packs/SecurityIntelligenceServicesFeed/Integrations/SecurityIntelligenceServicesFeed/SecurityIntelligenceServicesFeed.yml Docker image update

* Updated Metadata Of Pack AWS-IAM

* Added release notes to pack AWS-IAM

* Packs/AWS-IAM/Integrations/AWS-IAM/AWS-IAM.yml Docker image update

* Updated Metadata Of Pack AWS-Route53

* Added release notes to pack AWS-Route53

* Packs/AWS-Route53/Integrations/AWSRoute53/AWSRoute53.yml Docker image update

* Updated Metadata Of Pack AWS-AccessAnalyzer

* Added release notes to pack AWS-AccessAnalyzer

* Packs/AWS-AccessAnalyzer/Integrations/AWS-AccessAnalyzer/AWS-AccessAnalyzer.yml Docker image update

* Updated Metadata Of Pack AWS-GuardDuty

* Added release notes to pack AWS-GuardDuty

* Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/AWSGuardDutyEventCollector.yml Docker image update

* Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/AWSGuardDuty.yml Docker image update

* Updated Metadata Of Pack AWS-SecurityHub

* Added release notes to pack AWS-SecurityHub

* Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/AWSSecurityHubEventCollector.yml Docker image update

* Updated Metadata Of Pack Aws-SecretsManager

* Added release notes to pack Aws-SecretsManager

* Packs/Aws-SecretsManager/Integrations/AwsSecretsManager/AwsSecretsManager.yml Docker image update
* Updated Metadata Of Pack Exterro

* Added release notes to pack Exterro

* Packs/Exterro/Integrations/Exterro/Exterro.yml Docker image update
* Updated Metadata Of Pack OracleCloudInfrastructure

* Added release notes to pack OracleCloudInfrastructure

* Packs/OracleCloudInfrastructure/Integrations/OracleCloudInfrastructureEventCollector/OracleCloudInfrastructureEventCollector.yml Docker image update
* Updated Metadata Of Pack Intezer

* Added release notes to pack Intezer

* Packs/Intezer/Integrations/IntezerV2/IntezerV2.yml Docker image update

* Updated Metadata Of Pack Zabbix

* Added release notes to pack Zabbix

* Packs/Zabbix/Integrations/Zabbix/Zabbix.yml Docker image update

* Updated Metadata Of Pack FeedMalwareBazaar

* Added release notes to pack FeedMalwareBazaar

* Packs/FeedMalwareBazaar/Integrations/MalwareBazaarFeed/MalwareBazaarFeed.yml Docker image update

* Updated Metadata Of Pack FeedGCPWhitelist

* Added release notes to pack FeedGCPWhitelist

* Packs/FeedGCPWhitelist/Integrations/FeedGoogleIPRanges/FeedGoogleIPRanges.yml Docker image update

* Updated Metadata Of Pack AccentureCTI_Feed

* Added release notes to pack AccentureCTI_Feed

* Packs/AccentureCTI_Feed/Integrations/ACTIIndicatorFeed/ACTIIndicatorFeed.yml Docker image update

* Updated Metadata Of Pack SEKOIAIntelligenceCenter

* Added release notes to pack SEKOIAIntelligenceCenter

* Packs/SEKOIAIntelligenceCenter/Integrations/SEKOIAIntelligenceCenter/SEKOIAIntelligenceCenter.yml Docker image update

* Updated Metadata Of Pack JARM

* Added release notes to pack JARM

* Packs/JARM/Integrations/JARM/JARM.yml Docker image update

* Updated Metadata Of Pack Anomali_ThreatStream

* Added release notes to pack Anomali_ThreatStream

* Packs/Anomali_ThreatStream/Integrations/AnomaliThreatStreamv3/AnomaliThreatStreamv3.yml Docker image update

* Updated Metadata Of Pack CommonWidgets

* Added release notes to pack CommonWidgets

* Packs/CommonWidgets/Scripts/RSSWidget/RSSWidget.yml Docker image update

* Updated Metadata Of Pack FiltersAndTransformers

* Added release notes to pack FiltersAndTransformers

* Packs/FiltersAndTransformers/Scripts/Jmespath/Jmespath.yml Docker image update
* - Added new playbook for quarantine_file
- Old playbook deprecated
- New image added

* release notes added

* - New playbook for _Retrieve_File_Playbook_v2 created
- Old playbook _Retrieve_File_Playbook deprecated
- Image was added

* Release notes were added

* Changed the name of the playbook

* Readme file added

* Added image

* fixed image location in readme file

* Update Packs/CortexXDR/Playbooks/Cortex_XDR_-_Retrieve_File_v2.yml

Co-authored-by: Sasha Sokolovich <88268646+ssokolovich@users.noreply.github.com>

* Bump pack from version CortexXDR to 6.0.4.

* Removed unnecessary tests

* Readme files were updated

* Fixes for the playbooks

* fixed Tests/conf.json file

* image issue fixed

* Added new images

* Update Packs/CortexXDR/Playbooks/Cortex_XDR_-_Retrieve_File_Playbook_README.md

Co-authored-by: ShirleyDenkberg <62508050+ShirleyDenkberg@users.noreply.github.com>

* Update Packs/CortexXDR/Playbooks/Cortex_XDR_-_Retrieve_File_Playbook_README.md

Co-authored-by: ShirleyDenkberg <62508050+ShirleyDenkberg@users.noreply.github.com>

* Update Packs/CortexXDR/ReleaseNotes/6_0_4.md

Co-authored-by: ShirleyDenkberg <62508050+ShirleyDenkberg@users.noreply.github.com>

* Update Packs/CortexXDR/ReleaseNotes/6_0_4.md

Co-authored-by: ShirleyDenkberg <62508050+ShirleyDenkberg@users.noreply.github.com>

* Update Packs/CortexXDR/ReleaseNotes/6_0_4.md

Co-authored-by: ShirleyDenkberg <62508050+ShirleyDenkberg@users.noreply.github.com>

* Update Packs/CortexXDR/ReleaseNotes/6_0_4.md

Co-authored-by: ShirleyDenkberg <62508050+ShirleyDenkberg@users.noreply.github.com>

* Update Packs/CortexXDR/Playbooks/Cortex_XDR_-_Retrieve_File_Playbook_README.md

Co-authored-by: ShirleyDenkberg <62508050+ShirleyDenkberg@users.noreply.github.com>

* Update Packs/CortexXDR/Playbooks/Cortex_XDR_-_quarantine_file_v2_README.md

Co-authored-by: ShirleyDenkberg <62508050+ShirleyDenkberg@users.noreply.github.com>

* PR fixes - new condition to check if the task finished successfully

* release notes updated

* image path fixed

* Added new outputs for playbook

* release notes updated

* fix

* readme files fixed

* image issue

* image issue

* fix

* fix

* fix

* fix

* uploaded new playbook because of the image issue

* fix for image issue

* delete photo

* fixes

* test playbooks fixed

* test playbooks removed

---------

Co-authored-by: Sasha Sokolovich <88268646+ssokolovich@users.noreply.github.com>
Co-authored-by: Content Bot <bot@demisto.com>
Co-authored-by: ShirleyDenkberg <62508050+ShirleyDenkberg@users.noreply.github.com>
* Updated Metadata Of Pack AzureKeyVault

* Added release notes to pack AzureKeyVault

* Packs/AzureKeyVault/Integrations/AzureKeyVault/AzureKeyVault.yml Docker image update

* Updated Metadata Of Pack AzureSentinel

* Added release notes to pack AzureSentinel

* Packs/AzureSentinel/Integrations/AzureSentinel/AzureSentinel.yml Docker image update

* Updated Metadata Of Pack AzureDevOps

* Added release notes to pack AzureDevOps

* Packs/AzureDevOps/Integrations/AzureDevOps/AzureDevOps.yml Docker image update

* Updated Metadata Of Pack MicrosoftCloudAppSecurity

* Added release notes to pack MicrosoftCloudAppSecurity

* Packs/MicrosoftCloudAppSecurity/Integrations/MicrosoftCloudAppSecurity/MicrosoftCloudAppSecurity.yml Docker image update

* Updated Metadata Of Pack AzureRiskyUsers

* Added release notes to pack AzureRiskyUsers

* Packs/AzureRiskyUsers/Integrations/AzureRiskyUsers/AzureRiskyUsers.yml Docker image update

* Updated Metadata Of Pack MicrosoftGraphGroups

* Added release notes to pack MicrosoftGraphGroups

* Packs/MicrosoftGraphGroups/Integrations/MicrosoftGraphGroups/MicrosoftGraphGroups.yml Docker image update

* Updated Metadata Of Pack AzureSQLManagement

* Added release notes to pack AzureSQLManagement

* Packs/AzureSQLManagement/Integrations/AzureSQLManagement/AzureSQLManagement.yml Docker image update

* Updated Metadata Of Pack MicrosoftGraphAPI

* Added release notes to pack MicrosoftGraphAPI

* Packs/MicrosoftGraphAPI/Integrations/MicrosoftGraphAPI/MicrosoftGraphAPI.yml Docker image update

* Updated Metadata Of Pack MicrosoftTeams

* Added release notes to pack MicrosoftTeams

* Packs/MicrosoftTeams/Integrations/MicrosoftTeamsManagement/MicrosoftTeamsManagement.yml Docker image update

* Updated Metadata Of Pack MicrosoftGraphApplications

* Added release notes to pack MicrosoftGraphApplications

* Packs/MicrosoftGraphApplications/Integrations/MicrosoftGraphApplications/MicrosoftGraphApplications.yml Docker image update
#31135)

* fixed an issue where we get a Bad Request error.

* pre-commit

* added test

* fixed Flake8 error

* fixed cr comments

* fixed cr comments

* update Docker image
…31161)

* fixes

* code and test

* remove Json feed from this pr

* test

* note

* pre commit

* RN

* CR and Flake8

* format

* pre commit
* Fixes For 'URL Enrichment - Generic v2' Playbook

* RN

* Bump pack from version CommonPlaybooks to 2.4.36.

---------

Co-authored-by: Content Bot <bot@demisto.com>
* First commit

* Added RN

* Update Packs/CommonScripts/ReleaseNotes/1_12_54.md

Co-authored-by: Dean Arbel <darbel@paloaltonetworks.com>

---------

Co-authored-by: Dean Arbel <darbel@paloaltonetworks.com>
* Deprecate 'Get endpoint details - Generic' Playbook

* RN

* Bump pack from version CommonPlaybooks to 2.4.36.

* Bump pack from version CommonPlaybooks to 2.4.37.

---------

Co-authored-by: Content Bot <bot@demisto.com>
#31197)

* Replacing the deprecated sub-playbook within the 'NGFW Internal Scan' XSIAM playbook

* RN
* "pack contribution initial commit"

* Update pack_metadata.json

* Update and rename dashboard-98f353a2-312b-49f2-8e58-d71f60daf3a7-CISO_Metrics.json to dashboard-98f353a2-312b-49f2-8e58-d71f60daf3a7-CommunityCommonDashboards.json

Rename to CommunityCommonDashboards

* Update pack_metadata.json

Renamed  "name": "CommunityCommonDashboards"

* Update README.md

Added description

* Update README.md

* Update and rename README.md to README.md

* Rename dashboard-98f353a2-312b-49f2-8e58-d71f60daf3a7-CommunityCommonDashboards.json to dashboard-98f353a2-312b-49f2-8e58-d71f60daf3a7-CommunityCommonDashboards.json

* Rename .pack-ignore to .pack-ignore

* Rename .secrets-ignore to .secrets-ignore

* Rename pack_metadata.json to pack_metadata.json

* Update .pack-ignore

* Update pack_metadata.json

* Update .pack-ignore

* Update and rename dashboard-98f353a2-312b-49f2-8e58-d71f60daf3a7-CommunityCommonDashboards.json to CISOMetrics.json

Renamed to CISOMetrics

* Update pack_metadata.json

* Update pack_metadata.json

* Update README.md

---------

Co-authored-by: xsoar-bot <67315154+xsoar-bot@users.noreply.github.com>
Co-authored-by: Sapir Shuker <49246861+sapirshuker@users.noreply.github.com>
Co-authored-by: David Uhrlaub <90627446+rurhrlaub@users.noreply.github.com>
* added v2.1.14 codebase

* fix pr comments

* replace dummy md5 placeholder

* Update Packs/Cybereason/Integrations/Cybereason/Cybereason.py



* updated docker image python version

* updated release notes docker version

* added pagination params

* updated docker image

* fix lint errors

* fix demisto validate errors

* updated release notes

* updated release notes

* updated release notes

* updated command name as per PR comment

* removed manual filtering for response

* updated function name to match the command name format

* updated unit test as per new command name

* added machinename filter to api query

* moved empty output message to the top

* updated docker image tag to latest

* undo changes from unisolate endpoint playbook

---------

Co-authored-by: suraj-metron <87964764+suraj-metron@users.noreply.github.com>
Co-authored-by: Sapir Shuker <49246861+sapirshuker@users.noreply.github.com>
* fixed polling support

* fixed rn

* added rn

* added rn
* Added failing UT

* Fixed the issue

* Updated docker image

* Updated RN

* Update Packs/PAN-OS/ReleaseNotes/2_1_15.md

Co-authored-by: Dan Tavori <38749041+dantavori@users.noreply.github.com>

* Updated the bug fix and the UT

* updated docker image

---------

Co-authored-by: Dan Tavori <38749041+dantavori@users.noreply.github.com>
Copy link
Contributor

@ShahafBenYakir ShahafBenYakir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great testing.

if not cursor and aggregated_logs:
# we need to know where to stop in the next runs
last_run['last_id'] = aggregated_logs[0].get('id')
if aggregated_logs:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a comment to this if to explain the logic here

…sto/content into slack-evant-collector-remove-oldest

# Conflicts:
#	Packs/Slack/Integrations/SlackEventCollector/SlackEventCollector.py
#	Packs/Slack/Integrations/SlackEventCollector/SlackEventCollector_test.py
Comment on lines 144 to 149
2. We did a new query this time and did not finish fetching all the events,
so we save the newest event as 'newest_event_fetched' and the 'cursor' for the next run.
3. We continued to fetch events by 'cursor' and finished fetching them all,
saving the 'newest_event_fetched' as 'last_fetched_event'.
3. We continued to fetch events by 'cursor', and we still haven't finished fetching them all,
so we only need to save the 'cursor'.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In 2 and 4 we also save the last_search_stop_point_event_id no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I added it

@moishce moishce merged commit 4a07d53 into master Dec 27, 2023
18 checks passed
@moishce moishce deleted the slack-evant-collector-remove-oldest branch December 27, 2023 18:45
maimorag pushed a commit that referenced this pull request Dec 28, 2023
maimorag pushed a commit that referenced this pull request Dec 31, 2023
maimorag pushed a commit that referenced this pull request Dec 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet