Import GitLab projects #5120

AlexTugarev · 2021-08-09T15:10:04Z

This PR enables importing of GitLab repositories as projects into Gitpod.

Your GitLab account needs to have at the access level of a maintainer in order to install webhooks for prebuilds. That's also a filter criteria for repositories to be choose from.

Closes #5278

AlexTugarev · 2021-08-09T15:12:01Z

components/server/ee/src/gitlab/gitlab-app-support.ts

+import { Gitlab } from "@gitbeaker/node";
+
+@injectable()
+export class GitLabAppSupport {


The separation in EE and non-EE and the scope of the GitLabService binding makes it impossible to use that for the additions, unfortunately.

gtsiolis

Thank for pushing this forward @AlexTugarev! 🍊

Not sure if this was ready to review but left some early comments below. 🍋

components/dashboard/src/projects/NewProject.tsx

gtsiolis · 2021-08-09T18:11:25Z

components/server/ee/src/gitlab/gitlab-app-support.ts

+    @inject(UserDB) protected readonly userDB: UserDB;
+    @inject(TokenProvider) protected readonly tokenProvider: TokenProvider;
+
+    async getProviderRepositoriesForUser(params: { user: User, provider: string, hints?: object }): Promise<ProviderRepository[]> {


issue: This is probably minor for this iteration, as this is an edge case, but it takes 5-6 seconds for me to load the next step. FWIW, I'm a member of ~30 groups. In the future a loading indicator could help.

Thank for the hint. Just addmin perPage: 100 to override the default value of 20. That should be significant improvement in your case.

Thanks @AlexTugarev! Seems 2x faster! FWIW, Still taking 2-3 seconds.

Awesome. How many projects do you see there? According to the logs, each API response takes ~600ms, which is really slow.

Most groups contain less than 10 projects. For the personal account all projects (38) are listed. 💯

thought: I think the delay happens while fetching all groups or the ones with proper permissions and maybe because the account is a direct member of the gitlab-org group which contains 300+ second-level groups including public and private groups. Tried this with a test account and groups and projects loaded in under 1 second. 🎉

components/server/ee/src/container-module.ts

jankeromnes · 2021-08-13T13:16:39Z

Removing the automatic review request because this is still Draft. 😊

components/dashboard/src/projects/Project.tsx

gtsiolis · 2021-08-16T16:09:39Z

components/server/ee/src/gitlab/gitlab-app-support.ts

+    @inject(UserDB) protected readonly userDB: UserDB;
+    @inject(TokenProvider) protected readonly tokenProvider: TokenProvider;
+
+    async getProviderRepositoriesForUser(params: { user: User, provider: string, hints?: object }): Promise<ProviderRepository[]> {


Thanks @AlexTugarev! Seems 2x faster! FWIW, Still taking 2-3 seconds.

components/server/ee/src/gitlab/gitlab-app-support.ts

AlexTugarev · 2021-08-17T10:59:27Z

/hold

gtsiolis · 2021-08-17T13:50:52Z

Looking at this now! 👀

gtsiolis

Thanks @AlexTugarev! 🦊

Did another pass and left some comments below. 🏀

components/dashboard/src/projects/NewProject.tsx

gtsiolis · 2021-08-17T15:03:09Z

components/dashboard/src/projects/NewProject.tsx

@@ -322,7 +371,7 @@ export default function NewProject() {

    return (<div className="flex flex-col w-96 mt-24 mx-auto items-center">
        <h1>New Project</h1>
-        <p className="text-gray-500 text-center text-base">Select a git repository.</p>
+        <p className="text-gray-500 text-center text-base">Select a git repository on <strong>{provider}</strong>.</p>


question: Worth linking to the provider itself? Feel free to ignore if it does not seem helpful.

fyi: Opened follow up issue in #5333

gtsiolis · 2021-08-17T15:11:25Z

components/dashboard/src/projects/NewProject.tsx

+                            {icon && (
+                                <img src={icon} className="rounded-full w-6 h-6 absolute top-1/4 left-4" />
+                            )}
+                            <input className="w-full px-12 cursor-pointer font-semibold" readOnly type="text" value={selectedAccount || ""}></input>


issue: Is it expected that group avatars fail to load for groups? It seems that shared projects with write access on personal accounts successfully fetch the avatars.

Missing avatars on groups (a) Missing avatars on personal accounts (b)

fyi: Opened follow up issue in #5330.

gtsiolis · 2021-08-17T15:13:09Z

components/dashboard/src/projects/NewProject.tsx

-                        <img src={CaretDown} title="Select Account" className="filter-grayscale absolute top-1/2 right-3" />
+        const renderRepos = () => (<>
+            <div className={`mt-10 border ${isGitHub() ? "rounded-t-xl border-b-0" : "rounded-xl"} border-gray-100 flex-col`}>
+                <div className="px-8 pt-8 flex flex-col space-y-2">


question: For some reason, the pre-selected account is a specific but maybe random group. Does it make sense to default to pre-selecting your own account?

defaulting to own account sounds good.

DEAL. Let's do this here or open a follow up issue to fine tune this later! Your call. 📞

fyi: Opened follow up issue in #5331.

components/dashboard/src/projects/NewProject.tsx

AlexTugarev · 2021-08-19T15:51:37Z

question: Did we remove the loading placeholder?

@gtsiolis, thanks for the reminder! first I thought it got so fast, that it's no longer needed ;-)
It should be back in with the last update.

AlexTugarev · 2021-08-19T15:57:36Z

Known issue: wrong width for the bottom section.

@gtsiolis, wdyt of getting rid of the grayish background and render as simple as in #5120 (comment)?

jankeromnes · 2021-08-19T16:21:10Z

Many thanks for adding GitLab support! 💯 🎆

I briefly skimmed the code, but I'm unfamiliar with the GitLab integration code and didn't have time to read more deeply into it.

I also briefly tested this PR:

Connected both GitHub and GitLab accounts
Tried to create a new project for my user
There is a new loading state before anything shows up (empty list felt better)
It shows me this -- it feels I don't get the choice between GitHub or GitLab?

Adding one of the proposed GitLab project seems to work well 🎉 (the old Configurator is even able to fetch the config)

Trying again a few times, I randomly stumbled on this hidden option to add GitHub projects and not GitLab -- feels like this should be placed somewhere much more obvious (e.g. a simple "change" link right after "Select a git repository on gitlab.com")

Authorizing GitHub didn't work (but I guess that's expected since it's not the focus of this PR -- still would have been cool to test GitLab and GitHub projects together)

It doesn't even take that long to add the GitLab repo itself 💪 (took barely a few seconds to import the project and load the config)

AlexTugarev · 2021-08-20T09:53:15Z

/werft run

👍 started the job as gitpod-build-at-gitlab-projects.31

svenefftinge · 2021-08-20T12:41:24Z

components/server/src/projects/projects-service.ts

+    protected async onDidCreateProject(project: Project) {
+        let { userId, teamId, cloneUrl } = project;
+        const parsedUrl = parseRepoUrl(project.cloneUrl);
+        if ("gitlab.com" === parsedUrl?.host) {


It should work with self-managed gitlab's as well.

fyi: There's a follow up issue for self hosted support in #5115. Cc @svenefftinge

svenefftinge

Looks good and works well with gitlab.com. From the code besides one place it looks like it should work with self-managed GitLab's too.

roboquat · 2021-08-20T12:42:17Z

LGTM label has been added.

Git tree hash: 22ca396613977b4cf0d10117e4d89c57daa1e41a

roboquat · 2021-08-20T12:42:21Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: svenefftinge

Associated issue: #5278

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [svenefftinge]
~~components/dashboard/OWNERS~~ [svenefftinge]
~~components/server/OWNERS~~ [svenefftinge]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

AlexTugarev · 2021-08-20T12:45:44Z

From the code besides one place it looks like it should work with self-managed GitLab's too.

@svenefftinge, have you tested prebuilds with self-managed GitLabs?

AlexTugarev · 2021-08-20T12:53:29Z

/hold cancel

roboquat added the do-not-merge/work-in-progress label Aug 9, 2021

roboquat requested review from jankeromnes and laushinka August 9, 2021 15:10

roboquat added the size/M label Aug 9, 2021

AlexTugarev requested review from gtsiolis and removed request for laushinka August 9, 2021 15:10

AlexTugarev commented Aug 9, 2021

View reviewed changes

gtsiolis reviewed Aug 9, 2021

View reviewed changes

gtsiolis added component: dashboard 🧑‍🚀 crew: teams and projects labels Aug 9, 2021

gtsiolis linked an issue Aug 9, 2021 that may be closed by this pull request

Add support for importing GitLab projects #5078

Closed

jankeromnes removed their request for review August 13, 2021 13:16

AlexTugarev force-pushed the at/gitlab-projects branch 2 times, most recently from 5b87dab to f6990e7 Compare August 16, 2021 09:19

roboquat added size/L and removed size/M labels Aug 16, 2021

AlexTugarev commented Aug 16, 2021

View reviewed changes

components/dashboard/src/projects/Project.tsx Show resolved Hide resolved

gtsiolis reviewed Aug 16, 2021

View reviewed changes

roboquat added size/XL and removed size/L labels Aug 17, 2021

AlexTugarev changed the title ~~WIP Import GitLab projects~~ Import GitLab projects Aug 17, 2021

roboquat added the do-not-merge/hold label Aug 17, 2021

AlexTugarev force-pushed the at/gitlab-projects branch 2 times, most recently from 79d9a8f to 55b86f3 Compare August 17, 2021 12:21

gtsiolis reviewed Aug 17, 2021

View reviewed changes

AlexTugarev force-pushed the at/gitlab-projects branch 2 times, most recently from 38e3465 to 8608b77 Compare August 18, 2021 07:26

roboquat added the do-not-merge/hold label Aug 19, 2021

AlexTugarev mentioned this pull request Aug 19, 2021

Highlight the default branch for GitLab projects in branches #5278

Closed

AlexTugarev force-pushed the at/gitlab-projects branch 2 times, most recently from 1ad647a to 25b6037 Compare August 20, 2021 06:54

[projects] enable GitLab projects

485509e

AlexTugarev force-pushed the at/gitlab-projects branch from 25b6037 to 485509e Compare August 20, 2021 08:03

svenefftinge reviewed Aug 20, 2021

View reviewed changes

svenefftinge approved these changes Aug 20, 2021

View reviewed changes

roboquat assigned svenefftinge Aug 20, 2021

roboquat added the lgtm label Aug 20, 2021

roboquat added the approved label Aug 20, 2021

roboquat removed the do-not-merge/hold label Aug 20, 2021

roboquat merged commit 7e0f738 into main Aug 20, 2021

roboquat deleted the at/gitlab-projects branch August 20, 2021 12:54

This was referenced Sep 1, 2021

Fix dark theme colors and overflowing rounded corners for the project import pages #4717

Closed

[dashboard] Adjust Prebuild and Project Configurator pages to spec #5474

Merged

jankeromnes mentioned this pull request Sep 3, 2021

Improve repository-provider selection UX (when adding a new Project) #5524

Closed

jldec mentioned this pull request Nov 20, 2021

Add support for adding projects from Bitbucket.org git provider #5980

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Import GitLab projects #5120

Import GitLab projects #5120

AlexTugarev commented Aug 9, 2021 •

edited

Loading

AlexTugarev Aug 9, 2021

gtsiolis left a comment

gtsiolis Aug 9, 2021

AlexTugarev Aug 16, 2021

gtsiolis Aug 16, 2021

AlexTugarev Aug 16, 2021

gtsiolis Aug 17, 2021 •

edited

Loading

jankeromnes commented Aug 13, 2021 •

edited by werft-gitpod-dev-com bot

Loading

gtsiolis Aug 16, 2021

AlexTugarev commented Aug 17, 2021

gtsiolis commented Aug 17, 2021 •

edited by werft-gitpod-dev-com bot

Loading

gtsiolis left a comment

gtsiolis Aug 17, 2021

gtsiolis Aug 23, 2021

gtsiolis Aug 17, 2021

AlexTugarev Aug 17, 2021

gtsiolis Aug 23, 2021

gtsiolis Aug 17, 2021

AlexTugarev Aug 17, 2021

gtsiolis Aug 17, 2021

gtsiolis Aug 23, 2021

AlexTugarev commented Aug 19, 2021

AlexTugarev commented Aug 19, 2021

jankeromnes commented Aug 19, 2021 •

edited

Loading

AlexTugarev commented Aug 20, 2021 •

edited by werft-gitpod-dev-com bot

Loading

svenefftinge Aug 20, 2021

gtsiolis Aug 20, 2021

svenefftinge left a comment

roboquat commented Aug 20, 2021

roboquat commented Aug 20, 2021

AlexTugarev commented Aug 20, 2021

AlexTugarev commented Aug 20, 2021

Import GitLab projects #5120

Import GitLab projects #5120

Conversation

AlexTugarev commented Aug 9, 2021 • edited Loading

Choose a reason for hiding this comment

gtsiolis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gtsiolis Aug 17, 2021 • edited Loading

Choose a reason for hiding this comment

jankeromnes commented Aug 13, 2021 • edited by werft-gitpod-dev-com bot Loading

Choose a reason for hiding this comment

AlexTugarev commented Aug 17, 2021

gtsiolis commented Aug 17, 2021 • edited by werft-gitpod-dev-com bot Loading

gtsiolis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlexTugarev commented Aug 19, 2021

AlexTugarev commented Aug 19, 2021

jankeromnes commented Aug 19, 2021 • edited Loading

AlexTugarev commented Aug 20, 2021 • edited by werft-gitpod-dev-com bot Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

svenefftinge left a comment

Choose a reason for hiding this comment

roboquat commented Aug 20, 2021

roboquat commented Aug 20, 2021

AlexTugarev commented Aug 20, 2021

AlexTugarev commented Aug 20, 2021

AlexTugarev commented Aug 9, 2021 •

edited

Loading

gtsiolis Aug 17, 2021 •

edited

Loading

jankeromnes commented Aug 13, 2021 •

edited by werft-gitpod-dev-com bot

Loading

gtsiolis commented Aug 17, 2021 •

edited by werft-gitpod-dev-com bot

Loading

jankeromnes commented Aug 19, 2021 •

edited

Loading

AlexTugarev commented Aug 20, 2021 •

edited by werft-gitpod-dev-com bot

Loading