GitLab integration with Wekan Rules Sync

[xet7]

Moved to here from #109 as requested by GitLab integration Sponsor @gotjoshua . Currently @xet7 working on this issue, as first step here is Roadmap combined by @xet7 .

Wekan Sync to GitLab and back

• Linking issues to the external service for #number syntax, like Pretty ID and Board Key
• Is repo push mirroring already in GitLab CE? See 1 - 2 - 3. Related CI/CD to GitHub and Kubernetes and HN. CI/CD to external repos not open yet maybe ?
• Sync issues between Wekan and GitLab with Wekan Rules and SyncedCron timing, like with Gogs
• Sync comments
• Sync activities
• Sync attachments
• Post-actions to sync with Wekan Rules like group/private Push notifications, and notification settings. What is LINE messenger ?
• Embed Wekan to GitLab, GitLab to Wekan, with iframe
• GitLab as OAuth2 provider to Wekan, maybe already works
• Some ideas from related GitLab issue boards

About GitLab

• There is GitLab interview at FLOSS Weekly, and from that xet7 got info that not all features are in Open Source version. For example, when xet7 tried to enable syncing latest Wekan changes from GitHub to GitLab.com, there was text that it is paid only / Enterprise feature. There seemed to be some translation support in GitLab, but no Finnish translation yet.
• External CI/CD configuration in Starter and Bronze "Note that as part of our commitment to open source, public projects on Free GitLab.com have features equivalent to a Gold level subscription. So those public projects will continue to have this feature." so if I setup CI/CD at GitLab.com for Wekan Open Source project, I presume it stays working, and I can look at all features at Gold level that should work now at GitLab.com
• Navigate to external issue tracker, could this be used to replace GitLab issues/board with Wekan board
• SAST security report on pipelines view This would be nice, previously xet7 has already got security reports from quay.io

Related: What else to sync with Wekan Rules

• Jira
• Trello
• GitHub Projects and issues
• Sending to REST API
• Receiving Webhook

Related: Other Wekan platforms

• Friend: automate other Friend apps
• Univention: automate NextCloud etc
• Sandstorm: automate other Sandstorm apps

[gotjoshua]

Nice start @xet7 !
Here is my personal priority list:

1. linking to an external issue and displaying specific parts of the issue within the Wekan Card
   • I don't think any CI/CD would be needed for that as Wekan could simply poll the api.
2. Allowing the issue to be edited from within the Wekan card
   LeanLabs did many things VERY nicely in terms of UI eg:
   • Quick adding of tasks
   • Labels
   • Milestones
   • Swimlanes

I started a wekan board on the demo site:
https://wekan.indie.host/b/q94r7uguevibEKX29/wekan-gitlab-integration

[xet7]

@gotjoshua

Please move your demo site to https://boards.wekan.team and invite me there instead and add BoardAdmin access to me, because there is newest Wekan.

That https://wekan.indie.host is running old version of Wekan, and sometimes is not online.

[xet7]

@gotjoshua

About your personal priority list, can you add to this issue screenshots or animated gifs to this issue (or link to them elsewhere) about LeanLabs UI, so I can see how that UI is different from Wekan. Wekan also has Cards (Tasks), Labels, Swimlanes (click top bar switch button Lists/Swimlanes/Calendar).

[gotjoshua]

Yes! I will do all that later today or tomorrow... but maybe you should update the Readme Doc so that the DEMO link points to the newest wekan - i just followed the link and added the board there...

[xet7]

@gotjoshua

I updated readme, it had some old links.

[gotjoshua]

Unfortunately, leanlabs demo site is down, so i will try to get it running again connected to our gitlab instance...

In the meantime i recreated the wekan board for this issue:
https://boards.wekan.team/b/7mPjujLn25K5hTDG4/wekan-gitlab-integration

[gotjoshua]

Here you see the switch between swimlanes by user and milestone:

Possible to Choose a Milestone on the card (and will update via the api), but it is not possible to create or edit the milestones (this must be done in the gitlab ui):

[gotjoshua]

I think one main thing that is different about the swimlanes in leanlabs is that they are data-driven based on the project and issues (not just horizontally manually named lanes like in Wekan)

[gotjoshua]

Both Gitlab Boards and lean labs are using labels to indicate which "list" the card belongs in.
This seems great for visibility when viewing the issue in gitlab, but it requires that only one of the list labels are applied to a given issue (or at least that the board knows which to choose to place the issue - eg default to leftmost or rightmost)

[xet7]

Yes, leanlabs is data-driven and Wekan is not yet, because data is not yet synced to Wekan.

Wekan has roles like CommentOnly, where only comments are editable. So similar role can be added if something does not need to be editable.

Wekan also has board Outgoing Webhooks, so board changes could be sent to GitLab with it, if there is some way at GitLab to receive it, or that way could be added to Wekan.

[xet7]

@gotjoshua

Please add GitLab API docs URLs to these cards Description where I added you to card:
https://boards.wekan.team/b/7mPjujLn25K5hTDG4/wekan-gitlab-integration

I'm just a newbie on both LeanLabs and GitLab. I know only Wekan.

Is there some docker-compose.yml that would start both LeanLabs and GitLab on localhost? I failed setting up them.

[xet7]

Well, after failing to build and update LeanLabs kanban, I wrote this blogpost.

[gotjoshua]

Well, yeah staying agnostic is the key to enlightenment ; -)
... and productivity for that matter.

I think it will be best to use a library such as:
https://www.npmjs.com/package/gitlab
has links to both the official docs and mentions the libraries "goodies"

[gotjoshua]

Hello again after a long silence...
@xet7, Have you looked into gitlab integration any more?
Did you see my links to the gitlab api docs on those cards in the wekan team board?

• is that what you meant?

[xet7]

@gotjoshua

1. Please add screenshots as attachments to this card of data at GitLab, the same data that should be also visible at Wekan.

2. Then please add to GitLab Integration board more lists etc to mockup how GitLab data should be visible in Wekan.

This way I will get better idea about what I should do.

[miff2000]

Well, after failing to build and update LeanLabs kanban, I wrote this blogpost.

I'm keen to see this integration too. I also have a working docker-compose.yml file for LeanLabs KanBan if that will help?

Are there any other bits I can help with to push this forward?

Here are the GitLab API docs by the way. I've used them myself, they're pretty comprehensive:

• https://docs.gitlab.com/ee/api/api_resources.html

[xet7]

@miff2000

Please add your working docker-compose.yml as comment to this issue. Thanks!

Problem with LeanLabs KanBan was, that I could not get newest repo version to compile. Maybe some previous release tag would work. I did make modifications to source code paths to point to my forked repo at my newly created GitHub org:
https://github.com/kanlab/kanlab

From what I tested the installed version that @gotjoshua provided login to, LeanLabs kanban is very basic, minimal amount of features, and many bugs that do not exist in Wekan at all.

LeanLabs kanban has the advantage that it is written in Go, so it uses much less RAM memory and disk space, and I presume it's one executeable binary, that is easy to crosscompile to multiple platforms. Compiling Go is very fast. I have programmed with Go, with the experience that Go binaries do crash very easily, I have not learned how to do error handling in Go properly.

Wekan currently has about 30 000 Javascript files. When I try to unzip Wekan bundle on ReactOS, filesystem crashes, ReactOS can not handle that much files. Building Wekan is slow, so I use fast server that has NVME disk.

I will continue doing GitLab integration. Problem is I made a mistake, I accepted more paid work to my todo list. And it's just me coding all those features, on my todo list. I do know how to code all these features, but problem is that it takes some time to code all those features. I can not promise that all those features will be ready immediately.

[magick93]

@xet7 - thanks, your efforts are really appreciated.

I think there would be a lot of interest in this bidirectional issue syncing with gitlab.

Im a bit rusty with javascript but am happy to help.

[magick93]

And regarding https://boards.wekan.team/b/7mPjujLn25K5hTDG4/wekan-gitlab-integration/nayBBMDqZ7sK39bFk - May I suggest that, rather than board-to-project mapping, that you have a board that can import selected issues from any gitlab project that the user has access to.

The reasons for this are:

• A gitlab project already comes with a board. But to have a board that allows a kanban view of many projects requires the EE license. If Wekan can have a board that can show issues from many projects would be a point of difference from Gitlab CE.
• Ultimately it would be good if Wekan can have a board that pull in, and sync, with many other issue trackers.

[gotjoshua]

... that can import selected issues from any gitlab project that the user has access to ...

I agree, and still I would also love to have flexible swimlane options.

the great thing about project to board mapping is that you can rearrange by assignee, author, priority, milestone, labels etc into swim lanes

maybe it could be cool to have special dynamic swimlanes that would move cards vertically based on those parameters (but of course maintain their horizontal "list" position).

would be a point of difference from Gitlab CE.

Wekan is already worlds apart from the primitive GitLab board...

Another thought about the project-to-board vs individual issue-to-card syncing:

• Ideally, it would be great to have both options
• If we only had project-to-board, one could make a meta board and use Wekan card links to add cards from the "project board" to the "meta board"

[xet7]

The reasons for this are:

* A gitlab project already comes with a board. But to have a board that allows a kanban view of many projects requires the EE license. If Wekan can have a board that can show issues from many projects would be a point of difference from Gitlab CE.

But is that evil ? Am I destroying business models of GitLab, Trello and Jira ?

[xet7]

Hmm, I think I am not anywhere close of destroying GitLab/Trello/Jira, they still have a lot of customers.

[miff2000]

Please add your working docker-compose.yml as comment to this issue. Thanks!

Here you go. Taken from https://gitlab.com/whirm/kanban/blob/master/docker-compose.yml. All I needed to do was clone the repo and run docker-compose up.

To set the OAuth side working, you need to use a GitLab account (sign-up is free), then create an Application, and you'll get the given the KANBAN_GITLAB_CLIENT and KANBAN_GITLAB_SECRET below.

proxy:
  image: leanlabs/nginx:1.0.1
  volumes:
    - "./build/conf.d:/etc/nginx/conf.d"
    - "./build/certs:/etc/nginx/certs"
    - "./build/sites-enabled:/etc/nginx/sites-enabled"
  links:
    - kanban:kanban
  ports:
    - "443:443"
    - "80:80"

kanban:
  image: leanlabs/kanban:1.7.2
  environment:
    # URL on which Leanlabs Kanban will be reachable
    - KANBAN_SERVER_HOSTNAME=http://kanban.192.168.10.5.xip.io
    # This string is used to generate user auth tokens
    - KANBAN_SECURITY_SECRET=example2019
    # Your GitLab host URL
    - KANBAN_GITLAB_URL=https://gitlab.example.com
    # Your GitLab OAuth client ID
    - KANBAN_GITLAB_CLIENT=f9ad3039c4e3d9f614704988b7d6ad28c0af978642a28ec68a7313b7293892d9
    # Your GitLab OAuth client secret key
    - KANBAN_GITLAB_SECRET=19498ae2298002d1b78a898246f761463ac874f91b37195d757620dff5eeef38
    # Wheter to enable sign up with user API token
    - KANBAN_ENABLE_SIGNUP=true
    # Redis server address - IP:PORT
    - KANBAN_REDIS_ADDR=redis:6379
  links:
    - redis:redis
  command: ./kanban server

redis:
  image: leanlabs/redis:1.0.0
  volumes:
    - "/data:/data"
```

[gotjoshua]

not sure if its a great idea to post your client ID and secret publicly...

I usually also don't include them in my docker-compose, but rather as VARs in a .env file