feat: add the TInyBird resources for the search volume data #3178
Conversation
Signed-off-by: Raúl Santos <4837+borfast@users.noreply.github.com>
Signed-off-by: Raúl Santos <4837+borfast@users.noreply.github.com>
| @@ -0,0 +1,12 @@ | |||
| SCHEMA > | |||
| `id` UInt64 `json:$.record.id`, | |||
| `projectId` UUID `json:$.record.project_id`, | |||
There was a problem hiding this comment.
NIT: can we call this insightsProjectId to be consistent with the rest?
There was a problem hiding this comment.
Sure, no problem.
| NODE searchVolume_pipe | ||
| SQL > | ||
| % | ||
| SELECT projectId, keyword, dataTimestamp, volume, updatedAt |
There was a problem hiding this comment.
NIT: same naming nitpick as above
|
|
||
| ENGINE ReplacingMergeTree | ||
| ENGINE_PARTITION_KEY toYear(dataTimestamp) | ||
| ENGINE_SORTING_KEY projectId |
There was a problem hiding this comment.
We can add dataTimestamp to the sorting key - since queries will filter out first by insightsProjectId and timestamp, I'm guessing it's gonna help with filtering performance
There was a problem hiding this comment.
Good point. In fact, the project ID will probably not help much here at all, since we're going to query by keyword (the project slug), not project ID, so I think removing the project ID from here is even better.
| SQL > | ||
| % | ||
| SELECT projectId, keyword, dataTimestamp, volume, updatedAt | ||
| FROM searchVolume |
There was a problem hiding this comment.
You probably want FINAL here since the engine is ReplacingMergeTree in the source
| ) | ||
| }} | ||
| {% end %} | ||
| {% if defined(projectId) %} AND searchVolume.projectId = {{ projectId }} {% end %} |
There was a problem hiding this comment.
NIT: It's gonna help with consistency and usability of the pipe if we provide a project slug filter instead of an opaque id filter here. (since everywhere else FE expects this filtering to be done by slug)
There's a reusable example here
Another benefit of using segments_filtered pipe for this is that it provides insightsProjects.enabled check as well
There was a problem hiding this comment.
The keyword filter below this one is actually the slug. I'll change the name, it doesn't make sense to use the syntax from Keywords Everywhere here.
borfast
force-pushed
the
feature/ins-658-search-volume-tinybird-resources
branch
from
d7cf633 to
8726d7d
Compare
Signed-off-by: Raúl Santos <4837+borfast@users.noreply.github.com>
borfast
force-pushed
the
feature/ins-658-search-volume-tinybird-resources
branch
from
8726d7d to
97d233e
Compare
Signed-off-by: Raúl Santos <4837+borfast@users.noreply.github.com>
Signed-off-by: Raúl Santos <4837+borfast@users.noreply.github.com>
|
@epipav , can you review this again, please? |
Ticket in Linear
This adds the TinyBird data source and pipe with the respective endpoint for the search volume data, to be consumed by the Insights application.