Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ DESCRIPTION >
- `previousPeriodValue` is the comparison metric from the previous period (0 for non-temporal leaderboards).
- `collectionsSlugs` list of collection slugs the project belongs to.
- `isLF` indicates whether the project is a LF project (1 = true, 0 = false).
- `githubHandle` is the GitHub username for contributor leaderboards (empty string for non-contributor leaderboards).

TAGS "Analytics, Leaderboards, Project rankings"

Expand All @@ -28,7 +29,8 @@ SCHEMA >
`value` Float64,
`previousPeriodValue` Float64,
`collectionsSlugs` Array(String),
`isLF` UInt8
`isLF` UInt8,
`githubHandle` String

ENGINE MergeTree
ENGINE_SORTING_KEY leaderboardType, rank, slug, id
4 changes: 3 additions & 1 deletion services/libs/tinybird/datasources/members_sorted.datasource
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ DESCRIPTION >
- `displayName` is the member's preferred display name across platforms.
- `score` is the computed activity/engagement score for the member.
- `publicName` is the denormalized public display name for efficient queries.
- `githubHandle` is the member's GitHub username from memberIdentities (empty string if not available).

SCHEMA >
`id` String,
Expand All @@ -29,7 +30,8 @@ SCHEMA >
`updatedAt` DateTime64(3),
`displayName` String,
`score` Int32,
`publicName` String
`publicName` String,
`githubHandle` String

ENGINE MergeTree
ENGINE_PARTITION_KEY toYear(joinedAt)
Expand Down
3 changes: 2 additions & 1 deletion services/libs/tinybird/pipes/contributor_dependency.pipe
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ DESCRIPTION >
- `activity_type`: Optional string filter for single activity type (e.g., 'authored-commit')
- `activity_types`: Optional array of activity types (e.g., ['authored-commit', 'co-authored-commit'])
- `onlyContributions`: Optional boolean, defaults to 1 (contributions only), set to 0 for all activities
- Response: `id`, `displayName`, `contributionCount`, `contributionPercentage`, `roles`, `contributionPercentageRunningTotal`, `totalContributorCount`
- Response: `id`, `displayName`, `githubHandle`, `contributionCount`, `contributionPercentage`, `roles`, `contributionPercentageRunningTotal`, `totalContributorCount`

NODE contributions_percentage_running_total
SQL >
Expand All @@ -24,6 +24,7 @@ SQL >
SELECT
id,
displayName,
githubHandle,
contributionCount,
contributionPercentage,
roles,
Expand Down
3 changes: 2 additions & 1 deletion services/libs/tinybird/pipes/contributors_leaderboard.pipe
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ DESCRIPTION >
- `offset`: Optional integer for result pagination, defaults to 0
- Response:
- Count mode (`count=true`): `count` (total number of contributors)
- Data mode (default): `id`, `avatar`, `displayName`, `contributionCount`, `contributionPercentage`, `roles`
- Data mode (default): `id`, `avatar`, `displayName`, `githubHandle`, `contributionCount`, `contributionPercentage`, `roles`

NODE memberId_aggregates
SQL >
Expand Down Expand Up @@ -55,6 +55,7 @@ SQL >
m.id,
m.avatar,
case when is_non_lf.result then m.publicName else m.displayName end as displayName,
m.githubHandle,
ma.contributionCount,
ma.contributionPercentage,
mr.roles
Expand Down
32 changes: 18 additions & 14 deletions services/libs/tinybird/pipes/leaderboards_copy.pipe
Original file line number Diff line number Diff line change
@@ -1,53 +1,57 @@
DESCRIPTION >
Aggregates all individual leaderboard pipes into a single unified datasource for querying.
Consolidates 13 different leaderboard types with their respective rankings and values,
Consolidates 10 different leaderboard types with their respective rankings and values,
and copies the results to a datasource on a daily schedule at 1 AM.

NODE leaderboards_copy_union
DESCRIPTION >
Unions all leaderboard results with their respective type identifiers for consolidated querying

SQL >
SELECT *, 'active-contributors' as leaderboardType
SELECT *, '' as githubHandle, 'active-contributors' as leaderboardType
FROM leaderboards_project_active_contributors
UNION ALL
SELECT *, 'active-organizations' as leaderboardType
SELECT *, '' as githubHandle, 'active-organizations' as leaderboardType
FROM leaderboards_project_active_organizations
UNION ALL
SELECT *, 'commit-activity' as leaderboardType
SELECT *, '' as githubHandle, 'commit-activity' as leaderboardType
FROM leaderboards_commits
UNION ALL
SELECT *, 'stars' as leaderboardType
SELECT *, '' as githubHandle, 'stars' as leaderboardType
FROM leaderboards_stars
UNION ALL
SELECT *, 'forks' as leaderboardType
SELECT *, '' as githubHandle, 'forks' as leaderboardType
FROM leaderboards_forks
UNION ALL
SELECT *, 'package-downloads' as leaderboardType
SELECT *, '' as githubHandle, 'package-downloads' as leaderboardType
FROM leaderboards_package_downloads
UNION ALL
SELECT *, 0.0 as previousPeriodValue, 'focused-teams' as leaderboardType
SELECT *, 0.0 as previousPeriodValue, '' as githubHandle, 'focused-teams' as leaderboardType
FROM leaderboards_avg_commits_per_author
UNION ALL
SELECT *, 0.0 as previousPeriodValue, 'small-teams-massive-output' as leaderboardType
SELECT
*,
0.0 as previousPeriodValue,
'' as githubHandle,
'small-teams-massive-output' as leaderboardType
FROM leaderboards_small_project_commit
UNION ALL
SELECT *, 0.0 as previousPeriodValue, 'codebase-size' as leaderboardType
SELECT *, 0.0 as previousPeriodValue, '' as githubHandle, 'codebase-size' as leaderboardType
FROM leaderboards_codebase_size
UNION ALL
SELECT *, 'fastest-mergers' as leaderboardType
SELECT *, '' as githubHandle, 'fastest-mergers' as leaderboardType
FROM leaderboards_merge_time
UNION ALL
SELECT *, 'fastest-responders' as leaderboardType
SELECT *, '' as githubHandle, 'fastest-responders' as leaderboardType
FROM leaderboards_issue_response
UNION ALL
SELECT *, 0.0 as previousPeriodValue, 'resolution-rate' as leaderboardType
SELECT *, 0.0 as previousPeriodValue, '' as githubHandle, 'resolution-rate' as leaderboardType
FROM leaderboards_resolution_rate
UNION ALL
SELECT *, 'contributors' as leaderboardType
FROM leaderboards_members
UNION ALL
SELECT *, 'organizations' as leaderboardType
SELECT *, '' as githubHandle, 'organizations' as leaderboardType
FROM leaderboards_organizations

TYPE COPY
Expand Down
8 changes: 5 additions & 3 deletions services/libs/tinybird/pipes/leaderboards_members.pipe
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ DESCRIPTION >
Retrieves all members from the populated datasource

SQL >
SELECT id, displayName, avatar FROM members_sorted
SELECT id, displayName, avatar, githubHandle FROM members_sorted

NODE leaderboards_member_activity_types
DESCRIPTION >
Expand Down Expand Up @@ -66,13 +66,15 @@ SQL >
arrayDistinct(arrayFlatten(groupArray(proj.collectionsSlugs))) as collectionSlugs,
0 as isLF,
cast(coalesce(c.memberActivityCount, 0) as Float64) as value,
cast(coalesce(pp.memberActivityCount, 0) as Float64) as previousPeriodValue
cast(coalesce(pp.memberActivityCount, 0) as Float64) as previousPeriodValue,
p.githubHandle as githubHandle
FROM leaderboards_members_data p
INNER JOIN leaderboards_members_current_period c ON p.id = c.memberId
LEFT JOIN leaderboards_members_previous_period pp ON p.id = pp.memberId ARRAY
JOIN c.segmentIds as sid
LEFT JOIN leaderboards_members_projects proj ON proj.segmentId = sid
WHERE c.memberActivityCount > 0
GROUP BY p.id, p.displayName, p.avatar, c.memberActivityCount, pp.memberActivityCount
GROUP BY
p.id, p.displayName, p.avatar, p.githubHandle, c.memberActivityCount, pp.memberActivityCount
ORDER BY value DESC
LIMIT 100
24 changes: 16 additions & 8 deletions services/libs/tinybird/pipes/members_sorted_copy_pipe.pipe
Original file line number Diff line number Diff line change
@@ -1,16 +1,24 @@
DESCRIPTION >
Filters out `isBot = true` and `isTeamMember=true` members. Also possible to use more granular sorting keys in the destination datasource `members_sorted` Copy pipe runs every hour

NODE github_usernames
SQL >
SELECT memberId, argMax(value, createdAt) AS githubHandle
FROM memberIdentities FINAL
WHERE platform = 'github' AND type = 'username'
GROUP BY memberId

NODE members_pre_filter
SQL >
SELECT members.*, pub.publicName
FROM members final
left join members_public_names_ds pub on pub.memberId = members.id
where
not isBot
and not isTeamMember
and not isOrganization
and members.id in (select distinct memberId from activityRelations)
SELECT members.*, pub.publicName, coalesce(gh.githubHandle, '') AS githubHandle
FROM members FINAL
LEFT JOIN members_public_names_ds pub ON pub.memberId = members.id
LEFT JOIN github_usernames gh ON gh.memberId = members.id
WHERE
NOT isBot
AND NOT isTeamMember
AND NOT isOrganization
AND members.id IN (SELECT DISTINCT memberId FROM activityRelations)

TYPE COPY
TARGET_DATASOURCE members_sorted
Expand Down