diff --git a/services/libs/tinybird/datasources/leaderboards_copy_ds.datasource b/services/libs/tinybird/datasources/leaderboards_copy_ds.datasource index 99c561c5ff..2ac558d9cd 100644 --- a/services/libs/tinybird/datasources/leaderboards_copy_ds.datasource +++ b/services/libs/tinybird/datasources/leaderboards_copy_ds.datasource @@ -14,6 +14,7 @@ DESCRIPTION > - `previousPeriodValue` is the comparison metric from the previous period (0 for non-temporal leaderboards). - `collectionsSlugs` list of collection slugs the project belongs to. - `isLF` indicates whether the project is a LF project (1 = true, 0 = false). + - `githubHandle` is the GitHub username for contributor leaderboards (empty string for non-contributor leaderboards). TAGS "Analytics, Leaderboards, Project rankings" @@ -28,7 +29,8 @@ SCHEMA > `value` Float64, `previousPeriodValue` Float64, `collectionsSlugs` Array(String), - `isLF` UInt8 + `isLF` UInt8, + `githubHandle` String ENGINE MergeTree ENGINE_SORTING_KEY leaderboardType, rank, slug, id diff --git a/services/libs/tinybird/datasources/members_sorted.datasource b/services/libs/tinybird/datasources/members_sorted.datasource index 78a9dcfec0..8d0d46bece 100644 --- a/services/libs/tinybird/datasources/members_sorted.datasource +++ b/services/libs/tinybird/datasources/members_sorted.datasource @@ -13,6 +13,7 @@ DESCRIPTION > - `displayName` is the member's preferred display name across platforms. - `score` is the computed activity/engagement score for the member. - `publicName` is the denormalized public display name for efficient queries. + - `githubHandle` is the member's GitHub username from memberIdentities (empty string if not available). SCHEMA > `id` String, @@ -29,7 +30,8 @@ SCHEMA > `updatedAt` DateTime64(3), `displayName` String, `score` Int32, - `publicName` String + `publicName` String, + `githubHandle` String ENGINE MergeTree ENGINE_PARTITION_KEY toYear(joinedAt) diff --git a/services/libs/tinybird/pipes/contributor_dependency.pipe b/services/libs/tinybird/pipes/contributor_dependency.pipe index be50e0554c..830cdc4cdb 100644 --- a/services/libs/tinybird/pipes/contributor_dependency.pipe +++ b/services/libs/tinybird/pipes/contributor_dependency.pipe @@ -14,7 +14,7 @@ DESCRIPTION > - `activity_type`: Optional string filter for single activity type (e.g., 'authored-commit') - `activity_types`: Optional array of activity types (e.g., ['authored-commit', 'co-authored-commit']) - `onlyContributions`: Optional boolean, defaults to 1 (contributions only), set to 0 for all activities - - Response: `id`, `displayName`, `contributionCount`, `contributionPercentage`, `roles`, `contributionPercentageRunningTotal`, `totalContributorCount` + - Response: `id`, `displayName`, `githubHandle`, `contributionCount`, `contributionPercentage`, `roles`, `contributionPercentageRunningTotal`, `totalContributorCount` NODE contributions_percentage_running_total SQL > @@ -24,6 +24,7 @@ SQL > SELECT id, displayName, + githubHandle, contributionCount, contributionPercentage, roles, diff --git a/services/libs/tinybird/pipes/contributors_leaderboard.pipe b/services/libs/tinybird/pipes/contributors_leaderboard.pipe index dbc2ba3b0b..c4242aa8aa 100644 --- a/services/libs/tinybird/pipes/contributors_leaderboard.pipe +++ b/services/libs/tinybird/pipes/contributors_leaderboard.pipe @@ -18,7 +18,7 @@ DESCRIPTION > - `offset`: Optional integer for result pagination, defaults to 0 - Response: - Count mode (`count=true`): `count` (total number of contributors) - - Data mode (default): `id`, `avatar`, `displayName`, `contributionCount`, `contributionPercentage`, `roles` + - Data mode (default): `id`, `avatar`, `displayName`, `githubHandle`, `contributionCount`, `contributionPercentage`, `roles` NODE memberId_aggregates SQL > @@ -55,6 +55,7 @@ SQL > m.id, m.avatar, case when is_non_lf.result then m.publicName else m.displayName end as displayName, + m.githubHandle, ma.contributionCount, ma.contributionPercentage, mr.roles diff --git a/services/libs/tinybird/pipes/leaderboards_copy.pipe b/services/libs/tinybird/pipes/leaderboards_copy.pipe index 296c4b7a58..8d6bc06e7e 100644 --- a/services/libs/tinybird/pipes/leaderboards_copy.pipe +++ b/services/libs/tinybird/pipes/leaderboards_copy.pipe @@ -1,6 +1,6 @@ DESCRIPTION > Aggregates all individual leaderboard pipes into a single unified datasource for querying. - Consolidates 13 different leaderboard types with their respective rankings and values, + Consolidates 10 different leaderboard types with their respective rankings and values, and copies the results to a datasource on a daily schedule at 1 AM. NODE leaderboards_copy_union @@ -8,46 +8,50 @@ DESCRIPTION > Unions all leaderboard results with their respective type identifiers for consolidated querying SQL > - SELECT *, 'active-contributors' as leaderboardType + SELECT *, '' as githubHandle, 'active-contributors' as leaderboardType FROM leaderboards_project_active_contributors UNION ALL - SELECT *, 'active-organizations' as leaderboardType + SELECT *, '' as githubHandle, 'active-organizations' as leaderboardType FROM leaderboards_project_active_organizations UNION ALL - SELECT *, 'commit-activity' as leaderboardType + SELECT *, '' as githubHandle, 'commit-activity' as leaderboardType FROM leaderboards_commits UNION ALL - SELECT *, 'stars' as leaderboardType + SELECT *, '' as githubHandle, 'stars' as leaderboardType FROM leaderboards_stars UNION ALL - SELECT *, 'forks' as leaderboardType + SELECT *, '' as githubHandle, 'forks' as leaderboardType FROM leaderboards_forks UNION ALL - SELECT *, 'package-downloads' as leaderboardType + SELECT *, '' as githubHandle, 'package-downloads' as leaderboardType FROM leaderboards_package_downloads UNION ALL - SELECT *, 0.0 as previousPeriodValue, 'focused-teams' as leaderboardType + SELECT *, 0.0 as previousPeriodValue, '' as githubHandle, 'focused-teams' as leaderboardType FROM leaderboards_avg_commits_per_author UNION ALL - SELECT *, 0.0 as previousPeriodValue, 'small-teams-massive-output' as leaderboardType + SELECT + *, + 0.0 as previousPeriodValue, + '' as githubHandle, + 'small-teams-massive-output' as leaderboardType FROM leaderboards_small_project_commit UNION ALL - SELECT *, 0.0 as previousPeriodValue, 'codebase-size' as leaderboardType + SELECT *, 0.0 as previousPeriodValue, '' as githubHandle, 'codebase-size' as leaderboardType FROM leaderboards_codebase_size UNION ALL - SELECT *, 'fastest-mergers' as leaderboardType + SELECT *, '' as githubHandle, 'fastest-mergers' as leaderboardType FROM leaderboards_merge_time UNION ALL - SELECT *, 'fastest-responders' as leaderboardType + SELECT *, '' as githubHandle, 'fastest-responders' as leaderboardType FROM leaderboards_issue_response UNION ALL - SELECT *, 0.0 as previousPeriodValue, 'resolution-rate' as leaderboardType + SELECT *, 0.0 as previousPeriodValue, '' as githubHandle, 'resolution-rate' as leaderboardType FROM leaderboards_resolution_rate UNION ALL SELECT *, 'contributors' as leaderboardType FROM leaderboards_members UNION ALL - SELECT *, 'organizations' as leaderboardType + SELECT *, '' as githubHandle, 'organizations' as leaderboardType FROM leaderboards_organizations TYPE COPY diff --git a/services/libs/tinybird/pipes/leaderboards_members.pipe b/services/libs/tinybird/pipes/leaderboards_members.pipe index 82730ed376..0ded47ad0e 100644 --- a/services/libs/tinybird/pipes/leaderboards_members.pipe +++ b/services/libs/tinybird/pipes/leaderboards_members.pipe @@ -8,7 +8,7 @@ DESCRIPTION > Retrieves all members from the populated datasource SQL > - SELECT id, displayName, avatar FROM members_sorted + SELECT id, displayName, avatar, githubHandle FROM members_sorted NODE leaderboards_member_activity_types DESCRIPTION > @@ -66,13 +66,15 @@ SQL > arrayDistinct(arrayFlatten(groupArray(proj.collectionsSlugs))) as collectionSlugs, 0 as isLF, cast(coalesce(c.memberActivityCount, 0) as Float64) as value, - cast(coalesce(pp.memberActivityCount, 0) as Float64) as previousPeriodValue + cast(coalesce(pp.memberActivityCount, 0) as Float64) as previousPeriodValue, + p.githubHandle as githubHandle FROM leaderboards_members_data p INNER JOIN leaderboards_members_current_period c ON p.id = c.memberId LEFT JOIN leaderboards_members_previous_period pp ON p.id = pp.memberId ARRAY JOIN c.segmentIds as sid LEFT JOIN leaderboards_members_projects proj ON proj.segmentId = sid WHERE c.memberActivityCount > 0 - GROUP BY p.id, p.displayName, p.avatar, c.memberActivityCount, pp.memberActivityCount + GROUP BY + p.id, p.displayName, p.avatar, p.githubHandle, c.memberActivityCount, pp.memberActivityCount ORDER BY value DESC LIMIT 100 diff --git a/services/libs/tinybird/pipes/members_sorted_copy_pipe.pipe b/services/libs/tinybird/pipes/members_sorted_copy_pipe.pipe index b4fd5dd3b8..c3b229f2ce 100644 --- a/services/libs/tinybird/pipes/members_sorted_copy_pipe.pipe +++ b/services/libs/tinybird/pipes/members_sorted_copy_pipe.pipe @@ -1,16 +1,24 @@ DESCRIPTION > Filters out `isBot = true` and `isTeamMember=true` members. Also possible to use more granular sorting keys in the destination datasource `members_sorted` Copy pipe runs every hour +NODE github_usernames +SQL > + SELECT memberId, argMax(value, createdAt) AS githubHandle + FROM memberIdentities FINAL + WHERE platform = 'github' AND type = 'username' + GROUP BY memberId + NODE members_pre_filter SQL > - SELECT members.*, pub.publicName - FROM members final - left join members_public_names_ds pub on pub.memberId = members.id - where - not isBot - and not isTeamMember - and not isOrganization - and members.id in (select distinct memberId from activityRelations) + SELECT members.*, pub.publicName, coalesce(gh.githubHandle, '') AS githubHandle + FROM members FINAL + LEFT JOIN members_public_names_ds pub ON pub.memberId = members.id + LEFT JOIN github_usernames gh ON gh.memberId = members.id + WHERE + NOT isBot + AND NOT isTeamMember + AND NOT isOrganization + AND members.id IN (SELECT DISTINCT memberId FROM activityRelations) TYPE COPY TARGET_DATASOURCE members_sorted