Unable to create workspace from self-hosted gitlab

[a-elie]

Describe the bug

"Unexpected end of file from server" on Gitlab OAuth (state=oauth_provider=gitlab)

GET /api/oauth/callback?code=__REDACTED__&state=oauth_provider%3Dgitlab%26scope%3Dapi%2Bwrite_repository%2Bopenid%26request_method%3DPOST%26signature_method%3Drsa%26redirect_after_login%3Dhttps%3A%2F%2F HTTP/1.1
Host: eclipse-che.apps.lab.okd.local
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: http://gitlab.localdomain/
Connection: keep-alive
Cookie: JSESSIONID=__REDACTED__
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: cross-site
Sec-Fetch-User: ?1
Pragma: no-cache
Cache-Control: no-cache

HTTP/1.1 500 Internal Server Error
content-encoding: gzip
content-type: text/plain
date: Mon, 18 Jul 2022 13:21:39 GMT
gap-auth: ael@cluster.local
gap-upstream-address: 127.0.0.1:8081
jaxrs-body-provided: Error-Message
vary: accept-encoding
content-length: 60

kind: Secret
apiVersion: v1
metadata:
  name: gitlab-oauth-config
  labels:
    app.kubernetes.io/component: oauth-scm-configuration
    app.kubernetes.io/part-of: che.eclipse.org
  annotations:
    che.eclipse.org/oauth-scm-server: gitlab
    che.eclipse.org/scm-server-endpoint: http://gitlab.localdomain

Che version

7.50@latest

Steps to reproduce

1. Follow https://www.eclipse.org/che/docs/stable/administration-guide/configuring-oauth-2-for-gitlab/
2. Create workspace : https://eclipse-che.apps.lab.okd.local/#http://gitlab.localdomain/user/project
3. Redirected to openshift login => Sign in
4. Redirected to gitlab login => Sign in
5. Authorize eclipse che to use your account

Expected behavior

1. Recognize gitlab oauth_provider (github in che pod logs)
2. Save login informations
3. Redirect to https://eclipse-che.apps.lab.okd.local/#http://gitlab.localdomain/user/project

Runtime

OpenShift

Screenshots

Installation method

OperatorHub

Environment

other (please specify in additional context)

Eclipse Che Logs

2022-07-18 12:49:48,231[main] [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 182] - Configured factories for environments: '[openshift, kubernetes, no-environment]'
2022-07-18 12:49:48,232[main] [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 183] - Registered infrastructure 'openshift'
2022-07-18 12:49:48,316[main] [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 694] - Infrastructure is tracking 0 active runtimes that need to be stopped
2022-07-18 12:49:48,426[main] [INFO ] [o.e.c.a.c.u.ApiInfoLogInformer 36] - Eclipse Che Api Core: Build info '7.50.0' scmRevision 'ce5b3155c339e212d35ed7c33554adff98917478' implementationVersion '7.50.0'
2022-07-18 12:49:48,455[main] [WARN ] [p.s.AdminPermissionInitializer 69] - Admin admin not found yet.
2022-07-18 12:49:48,500[main] [INFO ] [o.e.c.c.metrics.MetricsServer 46] - Metrics server started at port 8087 successfully
18-Jul-2022 12:49:49.002 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/home/user/eclipse-che/tomcat/webapps/api.war] has finished in [24,698] ms
18-Jul-2022 12:49:49.004 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/home/user/eclipse-che/tomcat/webapps/swagger.war]
18-Jul-2022 12:49:49.306 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/home/user/eclipse-che/tomcat/webapps/swagger.war] has finished in [302] ms
18-Jul-2022 12:49:49.323 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
18-Jul-2022 12:49:49.344 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in [32537] milliseconds
2022-07-18 13:13:14,164[nio-8080-exec-1] [WARN ] [o.e.c.s.oauth.EmbeddedOAuthAPI 159] - Unsupported OAuth provider github
2022-07-18 13:13:15,749[nio-8080-exec-2] [WARN ] [o.e.c.s.oauth.EmbeddedOAuthAPI 159] - Unsupported OAuth provider github

Additional context

OpenShift version 4.10.0-0.okd-2022-03-07-131213
Eclipse Che 7.50
GitLab Community Edition 14.9.3

Still doesn't work with a github provider registred (and the warning is gone)
Works fine with github : https://eclipse-che.apps.lab.okd.local/#https://github.com/a-elie/web-app-test

[svor]

@vinokurig could you take a look please

[vinokurig]

Could not reproduce the error. I've setup a self hosted gitlab server and the authentication flow worked well. Tipically the Unexpected end of file from server error means that the remote server accepted and closed the connection without sending a response, so I suppose this is an infrustructure issue.
@a-elie Do you still see the error? Can you confirm that you self hosted gitlab works fine?

[a-elie]

Gitlab instance works fine, but that's the first application that is connected to the API

Unexpected end of file from server error means that the remote server accepted and closed the connection without sending a response, so I suppose this is an infrustructure issue.

Can you help me to narrow down the issue ?
I created a che user on gitlab (not admin) for the OAuth secret
I have the "Authorize eclipse che to use your account" redirection page on gitlab each time I try to create the workspace
My gitlab server is http-only

Can I add more logs to che server pod ?
For example :

spec:
  server:
    customCheProperties:
      CHE_LOGGER_CONFIG: "org.eclipse.che.security.oauth.EmbeddedOAuthAPI=DEBUG"

Thanks !

[vinokurig]

@a-elie Can you check your redirect URL of your gitlab application? It must be <che endpoint>/api/oauth/callback

[a-elie]

Redirect URL :

[vinokurig]

@a-elie Have you tried to test the flow with the SAAS gitlab?

[a-elie]

So ... it works fine on gitlab.com
This is the request callback :

GET /api/oauth/callback?code=__REDACTED__&state=oauth_provider%3Dgitlab%26scope%3Dapi%2Bwrite_repository%2Bopenid%26request_method%3DPOST%26signature_method%3Drsa%26redirect_after_login%3Dhttps%3A%2F%2F HTTP/1.1
Host: eclipse-che.apps.lab.okd.local
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://gitlab.com/
Connection: keep-alive
Cookie: JSESSIONID=__REDACTED__
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: cross-site
Sec-Fetch-User: ?1

HTTP/1.1 307 Temporary Redirect
content-length: 0
date: Wed, 20 Jul 2022 16:55:47 GMT
gap-auth: ael@cluster.local
gap-upstream-address: 127.0.0.1:8081
location: https://eclipse-che.apps.lab.okd.local/f?url=https%3A%2F%2Fgitlab.com%2Fa-elie%2Ftest-project

I removed user-che namespace on OKD and tried again on self-hosted, same error : Unexpected end of file from server

[a-elie]

Thanks for your help, the issue was not with gitlab auth_provider
che pod can't reach gitlab instance directly (network issue)

Got some helpful logs with :

apiVersion: org.eclipse.che/v2
kind: CheCluster
  name: eclipse-che
spec:
  components:
    cheServer:
      debug: true
      logLevel: DEBUG