Add private gitlab storage #493

agaget · 2019-07-30T12:07:04Z

Hi,
Thanks for the gitlab storage feature,
Is there a way to use a private gitlab ? When I click on "Authorize" it goes to "gitlab.com" but I would like it to store on my gitlab company (https://gitlab.example.fr/) ?
If not, can it be integrated ?

Thanks

davidjgraph · 2019-07-30T12:14:57Z

Happy to look at PR to add it.

fhemberger · 2019-08-04T06:59:05Z

References to 'gitlab.com' are in the following files:

src/main/webapp/js/app.min.js
src/main/webapp/js/diagramly/App.js
src/main/webapp/js/diagramly/Dialogs.js
src/main/webapp/js/diagramly/GitLabClient.js

You could for example modify the current Dockerfile like this:

FROM frekele/ant:1.10.3-jdk8 as BUILD

ARG GITLAB_DOMAIN=https://gitlab.com/
ENV DEBIAN_FRONTEND=noninteractive

# Download the latest draw.io release from GitHub
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN apt-get update \
  && apt-get install jq -y --no-install-recommends \
  && mkdir /usr/build \
  && curl -fsSL -o drawio.tar.gz --compressed "$(curl -fsSL --compressed "https://api.github.com/repos/jgraph/drawio/releases/latest" | jq -r ".tarball_url")" \
  && tar xzf drawio.tar.gz -C /usr/build --strip-components=1

# Replace all references to gitlab.com
WORKDIR /usr/build
RUN grep -rl 'https://gitlab.com/' | xargs sed -i "s|https://gitlab.com/|$GITLAB_DOMAIN|g"

# Continue to build like the official release 
WORKDIR /usr/build/etc/build/
RUN ant war

FROM tomcat:9.0 as TARGET
COPY --from=BUILD /usr/build/build/draw.war /usr/local/tomcat/webapps/
EXPOSE 8080
CMD ["catalina.sh", "run"]

Then build the Dockerfile with your custom URL:

docker build \
  --build-arg GITLAB_DOMAIN=https://gitlab.example.fr/ \
  -t drawio-custom-gitlab \
  .

Maybe not the nicest solution, but it should work. 😉

m4r10k · 2019-08-04T13:58:09Z

@fhemberger we did the first tries to do the same on Friday but weren't able to finish our testing. Therefore many thanks for your great summary - we will test it next week for our on-premises installation too. 🤗

fhemberger · 2019-08-04T14:07:51Z

@m4r10k Haven't tested it yet with our GitLab instance (maybe next week), just a quick idea I came up with. It should do the trick … looking forward for your feedback!

m4r10k · 2019-08-04T14:12:28Z

@fhemberger ... sure! Such ideas are great to exchange - you are some steps ahead. I just hacked this stuff quick and dirty into the draw.io Docker image directly for testing 🤣

The problem that we have is, that I know were to put the Client ID - but I am not sure about the Secret (from the Gitlab Applications...)

davidjgraph · 2019-08-05T16:50:55Z

Refreshing to see some constructive solutions in issues, but this one is a lot easier for us to deal with :). With 11.1.2 you can add a URL parameter "gitlab=yourURL" to set the path to your custom instance.

fhemberger · 2019-08-05T17:15:56Z

@davidjgraph Awesome, thanks!

m4r10k · 2019-08-05T18:42:59Z

@davidjgraph Many many thanks!

yetanothern · 2019-08-06T05:39:27Z

A stupid question: where should I add this parameter?

fhemberger · 2019-08-06T05:54:34Z

@stepkh http://draw.io?gitlab=https://mygitlab.example.com/

yetanothern · 2019-08-06T06:09:43Z

When I put url like http://my.drawio.local/?gitlab=http://my.gitlab.local I get "Save diagram to:" window, the same if I put drawio url without parameters.

yetanothern · 2019-08-06T06:16:58Z

This url works: http://my.drawio.local/?mode=gitlab&gitlab=http://my.gitlab.local. But when I authorize in gitlab, I get an error "Client authentication failed due to unkown client, no client authentication included, or unsupported authentication method". The same error was got by me when I manually changed 'gitlab.com' to 'my.gitlab.local' in previous version 11.1.1.

davidjgraph · 2019-08-06T07:27:03Z

OK, the main question is can anyone get this to work with the URL parameter? I don't have a local gitlab instance. If yes, authorizing in gitlab sounds off-topic for draw.io. If no, we need to debug.

agaget · 2019-08-06T07:44:13Z

Hi,
Come back from holiday and I see there already is a solution. Thanks for your prompt reaction :)
I encountered the same problem as @stepkh "client authentification blabla..." I'm not the administrator of our private instance so I will talk to my admin see if there is any right to unlock.

yetanothern · 2019-08-06T08:04:34Z

If it will help to debug, i glad to provide a couple test public containers with self-hosted draw.io and gitlab.

davidjgraph · 2019-08-06T08:06:02Z

Actually, I assumed with the above script that someone had this working, but probably not, reading it again. I should have explained the client ID.

At the top of GitLabClient.js there's a client ID. That's the ID for our application in Gitlab. It has custom redirect URLs for the authenication, so it won't work on your local URL.

I can tell you the setup using gitlab.com, I hope it's similiar on a local install. Under user settings there's an application option:

The redirect URI needs to point to the gitlab.html file in the root of the draw.io installation and you need to select the following scopes:

btw, if anyone can find a set of working scopes that doesn't include "API", please let us know.

When the app is saved, you need to swap out the clientID mentioned, but we'll add a URL parameter for that in a minute.

m4r10k · 2019-08-06T08:09:29Z

As I wrote in #493 (comment) , there is a setting for the ClientID in GitLabClient.js . You can generate this ID by creating an Application in the GitLab Admin Settings under "Applications" - but I have currently no idea where to put the Secret that is also provided by GitLab during Application creation :-)

m4r10k · 2019-08-06T08:10:58Z

@davidjgraph ups, ninja'd :)

davidjgraph · 2019-08-06T08:25:03Z

Added as d6c66c4

URL parameter is "gitlab-id"

Note that if you can inject these global variables in the docker image:

window.DRAWIO_GITLAB_URL
window.DRAWIO_GITLAB_ID

you don't have to use the URL parameters

agaget · 2019-08-06T08:48:34Z

In your example I think it shouldn't be MyGitlabPath but MyDrawIOPath.
I'm using the online version so I've configured the application as you said using as redirectURI https://draw.io/gitlab.html

When I entered this url with the client-ID given :
https://www.draw.io/?gitlab=https://mygitlab.fr&gitlab-id=94b55953cd3865a78dd6903e2adbd6095f2ac5646ec58aa46d0a76d848d18fa1
I create a new diagram, wanted to store to gitlab, when I "Authorize"

The url sent is the following, the id doesn't seem to be used ?
https://myGitlab.fr/oauth/authorize?client_id=5cdc018a32acddf6eba37592d9374945241e644b8368af847422d74c8709bc44&scope=api%20read_repository%20write_repository&redirect_uri=https%3A%2F%2Fwww.draw.io%2Fgitlab.html&response_type=token&state=123

davidjgraph · 2019-08-06T08:50:37Z

That won't work, because we haven't deployed this to the production version at draw.io yet...

agaget · 2019-08-06T08:51:37Z

That is a good reason :D

davidjgraph · 2019-08-06T08:52:17Z

Try the github pages version of this project for testing, https://jgraph.github.io/drawio/src/main/webapp/

agaget · 2019-08-06T08:57:53Z

Url for draw :
https://jgraph.github.io/drawio//src/main/webapp/?gitlab=https://mygitlab.fr&gitlab-id=94b55953cd3865a78dd6903e2adbd6095f2ac5646ec58aa46d0a76d848d18fa1

https://mygitlab.fr/oauth/authorize?client_id=5cdc018a32acddf6eba37592d9374945241e644b8368af847422d74c8709bc44&scope=api%20read_repository%20write_repository&redirect_uri=https%3A%2F%2Fjgraph.github.io%2Fgitlab.html&response_type=token&state=123

Still don't work, still a different id, I imagine that the client_id should be similar to the gitlab-id I indicate in the url, is it ?

davidjgraph · 2019-08-06T08:59:34Z

Ah, sorry my bad, it's not built. https://jgraph.github.io/drawio/src/main/webapp/?dev=1 , you need dev=1 in the URL parameters

agaget · 2019-08-06T09:01:50Z

your url result as "Page could not be loaded"

rickywu · 2019-08-06T12:50:04Z

Thanks, but I need an instance I can log into.

Docker image I don't know. https://github.com/fjudith/docker-draw.io is the recommended docker image, I think they process environment variables.

I think window.EXPORT_URL is a browser parameter not for docker container enviroment variable

Probally we need support read from enviroment variable?

davidjgraph · 2019-08-06T12:53:16Z

@rickywu Thanks. Yeah, it doesn't seem to be as easy as changing the URL and client ID.

@agaget That 404 is due to the draw.io domain not being on the jgraph.github.io root.

Let's try again with https://gitlab-test-dot-drawdotio.appspot.com/ . It's the github version, but the redirect will be at https://gitlab-test-dot-drawdotio.appspot.com/gitlab.html

davidjgraph · 2019-08-06T12:56:30Z

We don't use environment variables in our deployment. You're welcome to submit a PR, but it must work on Google App Engine, where our production draw.io runs.

agaget · 2019-08-06T13:07:52Z

🎉 it works !!
Just a strange details, in the selection menu of my repo it seems that it always select the last "projects" of the "groups"

For example if i click on ESS Modules /m-epics-PF704_Cpl_IPC it will select ESS Modules / PF704-dev (same for the other projects of the groups)

davidjgraph · 2019-08-06T13:15:42Z

@agaget can you open that as a new issue please? I'll close this one if the basics work.

agaget · 2019-08-06T13:28:54Z

@davidjgraph done #510

droub · 2019-10-22T15:01:16Z

I am facing the same kind of issues but it's hard to understand what setup is finally working. Could someone summarize the solution please ?

agaget · 2019-10-24T08:22:59Z

I can try.. As it seems that it's not in production yet I will use the https://gitlab-test-dot-drawdotio.appspot.com url.
In your Gitlab instance :

The redirect URI needs to point to the gitlab.html file in the root of the draw.io installation and you need to select the following scopes:

(use https://gitlab-test-dot-drawdotio.appspot.com not draw.io)

Then it will give you an Application-id paste it.

Use
https://gitlab-test-dot-drawdotio.appspot.com/?gitlab=https://MyGitLab.com&gitlab-id=Application_id_just_pasted
Authorize the app

It should work.

rickywu · 2019-10-24T08:38:31Z

The latest version add PreConfig.js to let you config DRAWIO_GITLAB_URL and DRAWIO_GITLAB_ID before compile or before start docker container

window.DRAWIO_GITLAB_URL = 'blabla';
window.DRAWIO_GITLAB_ID = 'blabla';

droub · 2019-10-25T13:12:18Z

Thanks, I still get the : "The redirect URI included is not valid." after authorization.

My draw.io docker is not served on a root url i.e. https://myapps.mycomp.com/draw could that be a problem ? My redirection does point to https://myapps.myconm.com/draw/gitlab.html

Also, I can see a 404ed request to the gitlab:
/oauth/undefined#search:1 Failed to load resource: the server responded with a status of 404 ()

I can't tell if my problem is on the draw.io side or on the gitlab side :(.

rickywu · 2019-10-26T03:02:14Z

You have to modify this line

drawio/src/main/webapp/js/diagramly/GitLabClient.js

Line 42 in c823b4f

var redirectUri = encodeURIComponent(window.location.origin + '/gitlab.html');

change window.location.origin to DRAWIO_GITLAB_URL
and set DRAWIO_GITLAB_URL = http://host.com/draw

see #515

davidjgraph · 2019-11-06T11:40:59Z

We'll change that to:

var href = window.location.href;
var dir = href.substring(0, href.lastIndexOf('/'));

in the next release.

davidjgraph · 2019-11-06T17:11:25Z

In 12.2.1

LilTrublMakr · 2020-07-29T13:22:07Z

I am unable to get a self hosted Draw.io and GitLab to work. I have tried all the suggestions I have seen and @agaget's instructions.

I have gone in to my gitlab > created new application > Named it > Set redirect uri to "https://draw.mydomain.com/gitlab.html" > Unticked "Confidential" (Maybe new since this feature was released?) > Ticked "api", "read_repository", and "write_repository" > Clicked on "Save application"

I then went to https://draw.mydomain.com.com/?gitlab=https://gitlab.mydomain.com&gitlab-id=(application key from newly created application in GitLab)

It opens up draw.io and gives me the option to create new or open existing. I create new > Blank > Click on "Authorize" > (GitLab window opens) and all I get is an error in GitLab that says "The redirect URI included is not valid."

I have tried doing everything fresh (deleted both containers for Draw.io and Gitlab) and deleted the config folder for GitLab (it does not seem that Draw.io has one for the unraid version at least) and started as fresh as I could to reach the same place.

Any help to get this working would be appreciated.

davidjgraph · 2020-07-29T14:14:54Z

What is the redirect URL used that is invalid? Is your custom application ID being used? How did you set it?

LilTrublMakr · 2020-07-29T15:16:44Z

What is the redirect URL used that is invalid?

https://draw.mydomain.com/gitlab.html
If I visit the page manually, I get This window will be closed automatically. which is expected but the page never closes.

Is your custom application ID being used?

Yes.

How did you set it?

I have tried both doing it by going through the menu that first pops up > selecting GitLab > and clicking on the Authorize button and by going to https://draw.mydomain.com.com/?gitlab=https://gitlab.mydomain.com&gitlab-id=(application key from GitLab) which basically takes me to the startup screen and I follow the process mentioned above.

apmikroel · 2020-09-02T11:57:14Z

We came upon the same error as @LilTrublMakr, and the error is from the redirect URL that draw.io generates.

When opening local instance by opening URL from browser https://<draw.io-CUSTOM-URL>/?gitlab=https://<CUSTOM-GITLAB-ADDRESS>&gitlab-id=<GENERATED-APP-ID> we cannot sign into our Gitlab installation.

Gitlab URL that is generated from draw.io when clicking Authenticate:
https://<CUSTOM-GITLAB-ADDRESS>/oauth/authorize?client_id=<GENERATED-APP-ID>&scope=api%20read_repository%20write_repository&redirect_uri=https%3A%2F%2F<draw.io-CUSTOM-URL>%2F%3Fgitlab%3Dhttps%3A%2F%2Fgitlab.html&response_type=token&state=123

And it should be:
https://<CUSTOM-GITLAB-ADDRESS>/oauth/authorize?client_id=<GENERATED-APP-ID>&scope=api%20read_repository%20write_repository&redirect_uri=https%3A%2F%2F<draw.io-CUSTOM-URL>%2Fgitlab.html&response_type=token&state=123

draw.io was installed from the dockerhub image jgraph/drawio, image id ca3338cc1f0d

I have used the same sign-up process (create a gitlab application with the scope api, read_repository, write_repository). After fixing the URL I can add an app, edit diagrams from repositories and create new diagrams.

davidjgraph · 2020-09-02T17:44:04Z

URL parameters must be URL encoded, the specification states this. Encode your parameters and it should work.

apmikroel · 2020-09-03T05:48:34Z

Right, thank you for the reply, after testing the opening instance by entering a URL

https://<draw.io-CUSTOM-URL>/?gitlab=https%3A%2F%2F<CUSTOM-GITLAB-ADDRESS>&gitlab-id=<GENERATED-ID>

I can confirm that integration works properly.

eremyja · 2020-12-29T03:54:24Z

Im not sure if this should be a new issue but it seems similar.

Description:
selfhosted drawio (v13.8.2) (docker container jgraph/drawio on rancher) returns "access denied" from selfhosted gitlab (v13.7.1) (proxmox container from turnkeylinux)

Steps I took:

Created application in gitlab admin area with trusted, confidential, api, read_repo, write_repo selected and callback uri http://<docker_host>:<drawio_port>/gitlab.html
Copied application key to environment variable "DRAWIO_GITLAB_ID" and gitlab url to "DRAWIO_GITLAB_URL"
In drawio; save to > gitlab, create new, blank template create, authorize
Logged into my gitlab
returned to drawio but Access denied
Back in gitlab applications, a new client is shown for drawio application.

Troubleshooting:

Tried creating the gitlab application from the user settings menu vice admin area.
Results: Additional screen in gitlab showing application creator and scope with another authorize button. Access still denied
Tried authorizing as a non-admin user.
Result: no change.
Added every combination of scope, trusted, and confidential
Result: no change
Tried https://<draw.io-CUSTOM-URL>/?gitlab=https%3A%2F%2F<CUSTOM-GITLAB-ADDRESS>&gitlab-id=<GENERATED-ID>
Result: no change

pepsifan · 2020-12-29T15:56:10Z

@eremyja It's probably a problem with CSP (Content Security Policy) which was introduced in drawio since 13.1.7 (29-MAY-2020). The CSP forbids the browser to make HTTP-Requests to other domains. The allowed domains are defined in src/main/webapp/js/diagramly/Devel.js

To define your custom allowed domain list for CSP you can use the Docker environment variable DRAWIO_CSP_HEADER.
Just have a look into docker-entrypoint.sh next to the Dockerfile in jgraph/docker-drawio.

Here an example for setting the environment variable in docker-compose.xml:

version: '3.2'
services:
  drawio:
    image: jgraph/drawio
    container_name: drawio.example.com
    environment:
      - DRAWIO_BASE_URL=https://drawio.example.com
      - DRAWIO_GITLAB_URL=https://git.example.com
      - DRAWIO_GITLAB_ID=abcdef123456
      - DRAWIO_CSP_HEADER=default-src \'self\'; script-src \'self\' \'unsafe-inline\'; connect-src \'self\' https://git.example.com; img-src * data:; media-src * data:; font-src * about:; style-src \'self\' \'unsafe-inline\';

In this example, the most sources are restricted to same origin except images, media and fonts. But the browser is also allowed to invoke requests to https://git.example.com by scripts.

eremyja · 2020-12-30T03:03:14Z

That was it! Thank you! Is there some documentation for these kinds of things? This ticket is the only place I could find any relevant information about setting up a self hosted instance.

agaget · 2021-12-06T09:42:18Z

Hi,
Come back to test this solution for various reasons (that was working when I wrote my last message #493 (comment))

Is this feature was included on the public instance app.diagrams.net/ ?

Cause when I use:
https://app.diagrams.net//?gitlab=https%3A%2F%2F&gitlab-id=

I got the redirection URI error.

I can't change nothing on this instance and I don't really want to install and manage my own instance.

Any idea ? Do i reopen the issue ?

davidjgraph added feature request notstale Stop the stale bot from processing labels Jul 30, 2019

davidjgraph added this to Todo Low Priority in drawio via automation Jul 30, 2019

davidjgraph moved this from Todo Low Priority to Triage in drawio Jul 30, 2019

davidjgraph closed this as completed Aug 6, 2019

drawio automation moved this from Triage to Done Aug 6, 2019

agaget mentioned this issue Aug 6, 2019

Bad git projects selection #510

Closed

nic-6443 mentioned this issue Mar 11, 2020

Fix GitLabClient.js get wrong draw.io host path when window.location href contains "/" #805

Closed

tafkos mentioned this issue Aug 19, 2021

Gitlab repo authorization fails on self-hosted instances #2192

Closed

B0rner mentioned this issue Oct 25, 2021

Unable to authorize draw.io in gitlab #2348

Closed

1 task

jgraph locked as resolved and limited conversation to collaborators Dec 6, 2021

Add private gitlab storage #493

Add private gitlab storage #493

Comments

agaget commented Jul 30, 2019

davidjgraph commented Jul 30, 2019

fhemberger commented Aug 4, 2019

m4r10k commented Aug 4, 2019

fhemberger commented Aug 4, 2019 • edited Loading

m4r10k commented Aug 4, 2019 • edited Loading

davidjgraph commented Aug 5, 2019

fhemberger commented Aug 5, 2019

m4r10k commented Aug 5, 2019

yetanothern commented Aug 6, 2019

fhemberger commented Aug 6, 2019

yetanothern commented Aug 6, 2019

yetanothern commented Aug 6, 2019

davidjgraph commented Aug 6, 2019

agaget commented Aug 6, 2019

yetanothern commented Aug 6, 2019 • edited Loading

davidjgraph commented Aug 6, 2019

m4r10k commented Aug 6, 2019

m4r10k commented Aug 6, 2019

davidjgraph commented Aug 6, 2019

agaget commented Aug 6, 2019

davidjgraph commented Aug 6, 2019

agaget commented Aug 6, 2019

davidjgraph commented Aug 6, 2019 • edited Loading

agaget commented Aug 6, 2019 • edited Loading

davidjgraph commented Aug 6, 2019

agaget commented Aug 6, 2019

rickywu commented Aug 6, 2019

davidjgraph commented Aug 6, 2019

davidjgraph commented Aug 6, 2019

agaget commented Aug 6, 2019

davidjgraph commented Aug 6, 2019

agaget commented Aug 6, 2019

droub commented Oct 22, 2019

agaget commented Oct 24, 2019

rickywu commented Oct 24, 2019 • edited Loading

droub commented Oct 25, 2019

rickywu commented Oct 26, 2019 • edited Loading

davidjgraph commented Nov 6, 2019 • edited Loading

davidjgraph commented Nov 6, 2019

LilTrublMakr commented Jul 29, 2020

davidjgraph commented Jul 29, 2020

LilTrublMakr commented Jul 29, 2020 • edited Loading

apmikroel commented Sep 2, 2020 • edited Loading

davidjgraph commented Sep 2, 2020

apmikroel commented Sep 3, 2020 • edited Loading

eremyja commented Dec 29, 2020

pepsifan commented Dec 29, 2020

eremyja commented Dec 30, 2020

agaget commented Dec 6, 2021

fhemberger commented Aug 4, 2019 •

edited

Loading

m4r10k commented Aug 4, 2019 •

edited

Loading

yetanothern commented Aug 6, 2019 •

edited

Loading

davidjgraph commented Aug 6, 2019 •

edited

Loading

agaget commented Aug 6, 2019 •

edited

Loading

rickywu commented Oct 24, 2019 •

edited

Loading

rickywu commented Oct 26, 2019 •

edited

Loading

davidjgraph commented Nov 6, 2019 •

edited

Loading

LilTrublMakr commented Jul 29, 2020 •

edited

Loading

apmikroel commented Sep 2, 2020 •

edited

Loading

apmikroel commented Sep 3, 2020 •

edited

Loading