Skip to content

chore: update octolens mentions sink frequency#3896

Merged
joanagmaia merged 1 commit intomainfrom
chore/update-octolens-mentions-sink
Mar 6, 2026
Merged

chore: update octolens mentions sink frequency#3896
joanagmaia merged 1 commit intomainfrom
chore/update-octolens-mentions-sink

Conversation

@joanagmaia
Copy link
Copy Markdown
Contributor

@joanagmaia joanagmaia commented Mar 6, 2026

Requested by PCC:
This pull request updates the scheduling and filtering logic for the Octolens mentions sink pipe to enable more frequent, hourly exports of data instead of daily exports. The changes ensure that only mentions created in the previous hour are exported, improving the timeliness of data ingestion.

Scheduling and data export frequency updates:

  • Changed the export schedule from daily at 00:30 UTC to hourly at :40 UTC, enabling more frequent data exports.
  • Updated the description to reflect the new hourly schedule and incremental sync logic based on the previous hour.

Data filtering and selection adjustments:

  • Modified the SQL query to filter mentions created in the previous hour instead of the previous day, using toStartOfHour(now()) and INTERVAL 1 HOUR.
  • Changed the date field in the exported data from the start of the day to the current timestamp for more precise time tracking.

Note

Medium Risk
Changes the export window and cron schedule for a Kafka sink, which could cause missed or duplicated rows if time boundaries/timezones are misaligned. Increased run frequency may also raise load on Tinybird/Kafka ingestion.

Overview
Updates octolens_mentions_sink.pipe to export Octolens mentions hourly (at :40) instead of daily.

Adjusts the SQL incremental filter from a previous-day createdAt window to the previous-hour window, and changes the exported date field from toStartOfDay(now()) to now() for finer-grained timestamps.

Written by Cursor Bugbot for commit 75a13d7. This will update automatically on new commits. Configure here.

@joanagmaia joanagmaia requested a review from gaspergrom March 6, 2026 13:29
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 6, 2026

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

2 similar comments
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 6, 2026

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 6, 2026

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Comment thread services/libs/tinybird/pipes/octolens_mentions_sink.pipe
@joanagmaia joanagmaia merged commit 50eb125 into main Mar 6, 2026
18 checks passed
@joanagmaia joanagmaia deleted the chore/update-octolens-mentions-sink branch March 6, 2026 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants