chore: update octolens mentions sink frequency#3896
Conversation
|
Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability. Example:
Projects:
Please add a Jira issue key to your PR title. |
2 similar comments
|
Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability. Example:
Projects:
Please add a Jira issue key to your PR title. |
|
Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability. Example:
Projects:
Please add a Jira issue key to your PR title. |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Requested by PCC:
This pull request updates the scheduling and filtering logic for the Octolens mentions sink pipe to enable more frequent, hourly exports of data instead of daily exports. The changes ensure that only mentions created in the previous hour are exported, improving the timeliness of data ingestion.
Scheduling and data export frequency updates:
Data filtering and selection adjustments:
toStartOfHour(now())andINTERVAL 1 HOUR.datefield in the exported data from the start of the day to the current timestamp for more precise time tracking.Note
Medium Risk
Changes the export window and cron schedule for a Kafka sink, which could cause missed or duplicated rows if time boundaries/timezones are misaligned. Increased run frequency may also raise load on Tinybird/Kafka ingestion.
Overview
Updates
octolens_mentions_sink.pipeto export Octolens mentions hourly (at:40) instead of daily.Adjusts the SQL incremental filter from a previous-day
createdAtwindow to the previous-hour window, and changes the exporteddatefield fromtoStartOfDay(now())tonow()for finer-grained timestamps.Written by Cursor Bugbot for commit 75a13d7. This will update automatically on new commits. Configure here.