Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitLab integration with Wekan Rules Sync #2698

Open
xet7 opened this issue Sep 11, 2019 · 24 comments

Comments

@xet7
Copy link
Member

@xet7 xet7 commented Sep 11, 2019

Moved to here from #109 as requested by GitLab integration Sponsor @gotjoshua . Currently @xet7 working on this issue, as first step here is Roadmap combined by @xet7 .

Wekan Sync to GitLab and back

About GitLab

  • There is GitLab interview at FLOSS Weekly, and from that xet7 got info that not all features are in Open Source version. For example, when xet7 tried to enable syncing latest Wekan changes from GitHub to GitLab.com, there was text that it is paid only / Enterprise feature. There seemed to be some translation support in GitLab, but no Finnish translation yet.
  • External CI/CD configuration in Starter and Bronze "Note that as part of our commitment to open source, public projects on Free GitLab.com have features equivalent to a Gold level subscription. So those public projects will continue to have this feature." so if I setup CI/CD at GitLab.com for Wekan Open Source project, I presume it stays working, and I can look at all features at Gold level that should work now at GitLab.com
  • Navigate to external issue tracker, could this be used to replace GitLab issues/board with Wekan board
  • SAST security report on pipelines view This would be nice, previously xet7 has already got security reports from quay.io

Related: What else to sync with Wekan Rules

  • Jira
  • Trello
  • GitHub Projects and issues
  • Sending to REST API
  • Receiving Webhook

Related: Other Wekan platforms

  • Friend: automate other Friend apps
  • Univention: automate NextCloud etc
  • Sandstorm: automate other Sandstorm apps
@gotjoshua

This comment has been minimized.

Copy link

@gotjoshua gotjoshua commented Sep 11, 2019

Nice start @xet7 !
Here is my personal priority list:

  1. linking to an external issue and displaying specific parts of the issue within the Wekan Card
    • I don't think any CI/CD would be needed for that as Wekan could simply poll the api.
  2. Allowing the issue to be edited from within the Wekan card
    LeanLabs did many things VERY nicely in terms of UI eg:
    • Quick adding of tasks
    • Labels
    • Milestones
    • Swimlanes

I started a wekan board on the demo site:
https://wekan.indie.host/b/q94r7uguevibEKX29/wekan-gitlab-integration

@xet7

This comment has been minimized.

Copy link
Member Author

@xet7 xet7 commented Sep 11, 2019

@gotjoshua

Please move your demo site to https://boards.wekan.team and invite me there instead and add BoardAdmin access to me, because there is newest Wekan.

That https://wekan.indie.host is running old version of Wekan, and sometimes is not online.

@xet7

This comment has been minimized.

Copy link
Member Author

@xet7 xet7 commented Sep 12, 2019

@gotjoshua

About your personal priority list, can you add to this issue screenshots or animated gifs to this issue (or link to them elsewhere) about LeanLabs UI, so I can see how that UI is different from Wekan. Wekan also has Cards (Tasks), Labels, Swimlanes (click top bar switch button Lists/Swimlanes/Calendar).

@gotjoshua

This comment has been minimized.

Copy link

@gotjoshua gotjoshua commented Sep 12, 2019

Yes! I will do all that later today or tomorrow... but maybe you should update the Readme Doc so that the DEMO link points to the newest wekan - i just followed the link and added the board there...

@xet7

This comment has been minimized.

Copy link
Member Author

@xet7 xet7 commented Sep 12, 2019

@gotjoshua

I updated readme, it had some old links.

@gotjoshua

This comment has been minimized.

Copy link

@gotjoshua gotjoshua commented Sep 12, 2019

Unfortunately, leanlabs demo site is down, so i will try to get it running again connected to our gitlab instance...

In the meantime i recreated the wekan board for this issue:
https://boards.wekan.team/b/7mPjujLn25K5hTDG4/wekan-gitlab-integration

@gotjoshua

This comment has been minimized.

Copy link

@gotjoshua gotjoshua commented Sep 12, 2019

Here you see the switch between swimlanes by user and milestone:
LeanLabsChooseSwimlane
Possible to Choose a Milestone on the card (and will update via the api), but it is not possible to create or edit the milestones (this must be done in the gitlab ui):
LeanLabsMilestonesChosenOnCard

@gotjoshua

This comment has been minimized.

Copy link

@gotjoshua gotjoshua commented Sep 12, 2019

I think one main thing that is different about the swimlanes in leanlabs is that they are data-driven based on the project and issues (not just horizontally manually named lanes like in Wekan)

@gotjoshua

This comment has been minimized.

Copy link

@gotjoshua gotjoshua commented Sep 13, 2019

Both Gitlab Boards and lean labs are using labels to indicate which "list" the card belongs in.
This seems great for visibility when viewing the issue in gitlab, but it requires that only one of the list labels are applied to a given issue (or at least that the board knows which to choose to place the issue - eg default to leftmost or rightmost)

@xet7

This comment has been minimized.

Copy link
Member Author

@xet7 xet7 commented Sep 13, 2019

Yes, leanlabs is data-driven and Wekan is not yet, because data is not yet synced to Wekan.

Wekan has roles like CommentOnly, where only comments are editable. So similar role can be added if something does not need to be editable.

Wekan also has board Outgoing Webhooks, so board changes could be sent to GitLab with it, if there is some way at GitLab to receive it, or that way could be added to Wekan.

@xet7

This comment has been minimized.

Copy link
Member Author

@xet7 xet7 commented Sep 18, 2019

@gotjoshua

Please add GitLab API docs URLs to these cards Description where I added you to card:
https://boards.wekan.team/b/7mPjujLn25K5hTDG4/wekan-gitlab-integration

I'm just a newbie on both LeanLabs and GitLab. I know only Wekan.

Is there some docker-compose.yml that would start both LeanLabs and GitLab on localhost? I failed setting up them.

@xet7

This comment has been minimized.

Copy link
Member Author

@xet7 xet7 commented Sep 18, 2019

Well, after failing to build and update LeanLabs kanban, I wrote this blogpost.

@gotjoshua

This comment has been minimized.

Copy link

@gotjoshua gotjoshua commented Sep 19, 2019

Well, yeah staying agnostic is the key to enlightenment ; -)
... and productivity for that matter.

I think it will be best to use a library such as:
https://www.npmjs.com/package/gitlab
has links to both the official docs and mentions the libraries "goodies"

@gotjoshua

This comment has been minimized.

Copy link

@gotjoshua gotjoshua commented Oct 25, 2019

Hello again after a long silence...
@xet7, Have you looked into gitlab integration any more?
Did you see my links to the gitlab api docs on those cards in the wekan team board?

  • is that what you meant?
@xet7

This comment has been minimized.

Copy link
Member Author

@xet7 xet7 commented Oct 25, 2019

@gotjoshua

  1. Please add screenshots as attachments to this card of data at GitLab, the same data that should be also visible at Wekan.

  2. Then please add to GitLab Integration board more lists etc to mockup how GitLab data should be visible in Wekan.

This way I will get better idea about what I should do.

@miff2000

This comment has been minimized.

Copy link

@miff2000 miff2000 commented Nov 28, 2019

Well, after failing to build and update LeanLabs kanban, I wrote this blogpost.

I'm keen to see this integration too. I also have a working docker-compose.yml file for LeanLabs KanBan if that will help?

Are there any other bits I can help with to push this forward?

Here are the GitLab API docs by the way. I've used them myself, they're pretty comprehensive:

@xet7

This comment has been minimized.

Copy link
Member Author

@xet7 xet7 commented Nov 28, 2019

@miff2000

Please add your working docker-compose.yml as comment to this issue. Thanks!

Problem with LeanLabs KanBan was, that I could not get newest repo version to compile. Maybe some previous release tag would work. I did make modifications to source code paths to point to my forked repo at my newly created GitHub org:
https://github.com/kanlab/kanlab

From what I tested the installed version that @gotjoshua provided login to, LeanLabs kanban is very basic, minimal amount of features, and many bugs that do not exist in Wekan at all.

LeanLabs kanban has the advantage that it is written in Go, so it uses much less RAM memory and disk space, and I presume it's one executeable binary, that is easy to crosscompile to multiple platforms. Compiling Go is very fast. I have programmed with Go, with the experience that Go binaries do crash very easily, I have not learned how to do error handling in Go properly.

Wekan currently has about 30 000 Javascript files. When I try to unzip Wekan bundle on ReactOS, filesystem crashes, ReactOS can not handle that much files. Building Wekan is slow, so I use fast server that has NVME disk.

I will continue doing GitLab integration. Problem is I made a mistake, I accepted more paid work to my todo list. And it's just me coding all those features, on my todo list. I do know how to code all these features, but problem is that it takes some time to code all those features. I can not promise that all those features will be ready immediately.

@magick93

This comment has been minimized.

Copy link

@magick93 magick93 commented Nov 29, 2019

@xet7 - thanks, your efforts are really appreciated.

I think there would be a lot of interest in this bidirectional issue syncing with gitlab.

Im a bit rusty with javascript but am happy to help.

@magick93

This comment has been minimized.

Copy link

@magick93 magick93 commented Nov 29, 2019

And regarding https://boards.wekan.team/b/7mPjujLn25K5hTDG4/wekan-gitlab-integration/nayBBMDqZ7sK39bFk - May I suggest that, rather than board-to-project mapping, that you have a board that can import selected issues from any gitlab project that the user has access to.

The reasons for this are:

  • A gitlab project already comes with a board. But to have a board that allows a kanban view of many projects requires the EE license. If Wekan can have a board that can show issues from many projects would be a point of difference from Gitlab CE.
  • Ultimately it would be good if Wekan can have a board that pull in, and sync, with many other issue trackers.
@gotjoshua

This comment has been minimized.

Copy link

@gotjoshua gotjoshua commented Nov 29, 2019

... that can import selected issues from any gitlab project that the user has access to ...

I agree, and still I would also love to have flexible swimlane options.

the great thing about project to board mapping is that you can rearrange by assignee, author, priority, milestone, labels etc into swim lanes

maybe it could be cool to have special dynamic swimlanes that would move cards vertically based on those parameters (but of course maintain their horizontal "list" position).

would be a point of difference from Gitlab CE.

Wekan is already worlds apart from the primitive GitLab board...

Another thought about the project-to-board vs individual issue-to-card syncing:

  • Ideally, it would be great to have both options
  • If we only had project-to-board, one could make a meta board and use Wekan card links to add cards from the "project board" to the "meta board"
@xet7

This comment has been minimized.

Copy link
Member Author

@xet7 xet7 commented Nov 29, 2019

The reasons for this are:

* A gitlab project already comes with a board. But to have a board that allows a kanban view of many projects requires the EE license. If Wekan can have a board that can show issues from many projects would be a point of difference from Gitlab CE.

But is that evil ? Am I destroying business models of GitLab, Trello and Jira ?

@xet7

This comment has been minimized.

Copy link
Member Author

@xet7 xet7 commented Nov 29, 2019

Hmm, I think I am not anywhere close of destroying GitLab/Trello/Jira, they still have a lot of customers.

@miff2000

This comment has been minimized.

Copy link

@miff2000 miff2000 commented Nov 29, 2019

Please add your working docker-compose.yml as comment to this issue. Thanks!

Here you go. Taken from https://gitlab.com/whirm/kanban/blob/master/docker-compose.yml. All I needed to do was clone the repo and run docker-compose up.

To set the OAuth side working, you need to use a GitLab account (sign-up is free), then create an Application, and you'll get the given the KANBAN_GITLAB_CLIENT and KANBAN_GITLAB_SECRET below.

proxy:
  image: leanlabs/nginx:1.0.1
  volumes:
    - "./build/conf.d:/etc/nginx/conf.d"
    - "./build/certs:/etc/nginx/certs"
    - "./build/sites-enabled:/etc/nginx/sites-enabled"
  links:
    - kanban:kanban
  ports:
    - "443:443"
    - "80:80"

kanban:
  image: leanlabs/kanban:1.7.2
  environment:
    # URL on which Leanlabs Kanban will be reachable
    - KANBAN_SERVER_HOSTNAME=http://kanban.192.168.10.5.xip.io
    # This string is used to generate user auth tokens
    - KANBAN_SECURITY_SECRET=example2019
    # Your GitLab host URL
    - KANBAN_GITLAB_URL=https://gitlab.example.com
    # Your GitLab OAuth client ID
    - KANBAN_GITLAB_CLIENT=f9ad3039c4e3d9f614704988b7d6ad28c0af978642a28ec68a7313b7293892d9
    # Your GitLab OAuth client secret key
    - KANBAN_GITLAB_SECRET=19498ae2298002d1b78a898246f761463ac874f91b37195d757620dff5eeef38
    # Wheter to enable sign up with user API token
    - KANBAN_ENABLE_SIGNUP=true
    # Redis server address - IP:PORT
    - KANBAN_REDIS_ADDR=redis:6379
  links:
    - redis:redis
  command: ./kanban server

redis:
  image: leanlabs/redis:1.0.0
  volumes:
    - "/data:/data"
```
@gotjoshua

This comment has been minimized.

Copy link

@gotjoshua gotjoshua commented Nov 29, 2019

not sure if its a great idea to post your client ID and secret publicly...

I usually also don't include them in my docker-compose, but rather as VARs in a .env file

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.