Unable to view .ipynb files in VSCode in Insecured cluster

[rkpattnaik780]

What steps did you take and what happened:

Unable to view .ipynb files in vscode in Insecured cluster.

Steps to reproduce:

1. Login to an insecured cluster (example: OSIA)
2. Start a vscode notebook.
3. Create a file sample.ipynb
4. You will see the following error after certain amount of loading time:

Could not initialize webview: Error: Could not register service worker: SecurityError: Failed to register a ServiceWorker for scope 
('https://vscode-test.apps.hnalla.dev.datahub.redhat.com/vscode/stable-
2ccd690cbff1569e4a83d7c43d45101f817401dc/static/out/vs/workbench/contrib/webview/browser/pre/') with script 
('https://vscode-test.apps.hnalla.dev.datahub.redhat.com/vscode/stable-
2ccd690cbff1569e4a83d7c43d45101f817401dc/static/out/vs/workbench/contrib/webview/browser/pre/service-worker.js?
v=4&vscode-resource-base-authority=vscode-resource.vscode-cdn.net&remoteAuthority=vscode-
test.apps.hnalla.dev.datahub.redhat.com'): An SSL certificate error occurred when fetching the script..}

What did you expect to happen:

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• OpenDatahub Version: (please check the operator version)
• Workbench: (all, data-science, etc)
• Workbench Version: (2023.1, etc)
• Specific tool: (jupyterlab, rstudio, vscode, elyra-pipelines,etc)
• Notebook-Controller Version: (please check the image version in notebook-controller deployment)

Logs/Screenshots

JIRA: https://issues.redhat.com/browse/RHOAIENG-234

[guimou]

This does not seem to be related to disconnected environments as mentioned in #349, but to a self-signed certificate error, which is already known in the base code: coder/code-server#5671
There are three solutions offered at the end of above bug.

[harshad16]

There are some similar known issue on code-server side.
Adding here for references: coder/code-server#3410, coder/code-server#4800, coder/code-server#3754

[harshad16]

Based on the suggestions from the above comments, I have tried a few of the suggestions.

1. Pass on certs to code-server with option suggested here

• Set up certs ( for example: used the TLS certs created via service-ca)
• Made the change

  notebooks/codeserver/ubi9-python-3.9/run-code-server.sh

  Line 20 in bf2bf3c

  start_process /usr/bin/code-server \

# Start server
if [ -d "/etc/tls/private" ]; then
  start_process /usr/bin/code-server \
  --bind-addr 0.0.0.0:8787 \
  --disable-telemetry \
  --cert /etc/tls/private/tls.crt \
  --cert-key /etc/tls/private/tls.key \
  --auth none \
  --disable-update-check \
  /opt/app-root/src
else
  start_process /usr/bin/code-server \
    --bind-addr 0.0.0.0:8787 \
    --disable-telemetry \
    --auth none \
    --disable-update-check \
    /opt/app-root/src
fi

• Mount the certs to the path /etc/tls/private

To use this a notebook , use this image: quay.io/harshad16/code-server:arg
This breaks the nginx routing.

2. Tried passing the certificate on nginx level.

• Provided the self-signed certs at:

  notebooks/codeserver/ubi9-python-3.9/nginx/serverconf/proxy.conf.template_nbprefix

  Line 68 in bf2bf3c

# ssl self-signed
    proxy_ssl_trusted_certificate /var/run/secrets/kubernetes.io/serviceaccount/ca.crt;
    proxy_ssl_verify       on;
    proxy_ssl_verify_depth 2;

• https://docs.nginx.com/nginx/admin-guide/security-controls/securing-http-traffic-upstream/
• This doesn't fix the issue.
  To use this a notebook , use this image: quay.io/harshad16/code-server:nginx

[atheo89]

Following Harshad's first attempt and changing the html protocol on the nginx config from http to https we were able to connect on vscode, however the webview error still persists.

Adding here some logs to help with the investigating:

spawn-fcgi: child spawned successfully: PID: 10
Running command: /usr/bin/code-server --bind-addr 0.0.0.0:8787 --cert /etc/tls/private/tls.crt --cert-key /etc/tls/private/tls.key --disable-telemetry --auth none --disable-update-check /opt/app-root/src
[2023-11-22T19:45:01.673Z] info code-server 4.16.1 94ef3776ad7bebfb5780dfc9632e04d20d5c9a6c
[2023-11-22T19:45:01.674Z] info Using user-data-dir ~/.local/share/code-server
[2023-11-22T19:45:01.692Z] info Using config file ~/.config/code-server/config.yaml
[2023-11-22T19:45:01.692Z] info HTTPS server listening on https://0.0.0.0:8787/
[2023-11-22T19:45:01.692Z] info - Authentication is disabled
[2023-11-22T19:45:01.692Z] info - Using certificate for HTTPS: /etc/tls/private/tls.crt
[2023-11-22T19:45:01.692Z] info Session server listening on /opt/app-root/src/.local/share/code-server/code-server-ipc.sock
[19:45:15]
[19:45:15] Extension host agent started.
127.0.0.1 - user-adri [22/Nov/2023:19:45:15 +0000] "GET /notebook/test/new3/ HTTP/1.1" 302 145 "https://odh-dashboard-opendatahub.apps.atheodor.dev.datahub.redhat.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36" "10.39.193.192, 10.129.6.1"
[19:45:16] [<unknown>][a387a88a][ManagementConnection] Unknown reconnection token (never seen).
File not found: /usr/lib/code-server/lib/vscode/out/vsda_bg.wasm
File not found: /usr/lib/code-server/lib/vscode/out/vsda.js
[19:45:26] [<unknown>][982af50e][ManagementConnection] New connection established.
[19:45:27] Using the in-memory credential store as the operating system's credential store could not be accessed. Please see https://aka.ms/vscode-server-keyring on how to set this up. Details: libsecret-1.so.0: cannot open shared object file: No such file or directory
[19:45:28] [<unknown>][39850f26][ExtensionHostConnection] New connection established.
[19:45:28] [<unknown>][39850f26][ExtensionHostConnection] <104> Launched Extension Host Process.
127.0.0.1 - user-adri [22/Nov/2023:19:52:56 +0000] "GET /notebook/test/new3/ HTTP/1.1" 302 145 "https://odh-dashboard-opendatahub.apps.atheodor.dev.datahub.redhat.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36" "10.39.193.192, 10.129.6.1"
File not found: /usr/lib/code-server/lib/vscode/out/vsda_bg.wasm
File not found: /usr/lib/code-server/lib/vscode/out/vsda.js
[19:53:02] [<unknown>][e6f1a8b5][ManagementConnection] New connection established.
[19:53:04] [<unknown>][fa540c75][ExtensionHostConnection] New connection established.
[19:53:04] [<unknown>][fa540c75][ExtensionHostConnection] <1478> Launched Extension Host Process.
2023/11/22 19:53:04 [error] 29#29: *606 open() "/opt/app-root/src/vs/workbench/api/worker/extensionHostWorker.js.map" failed (2: No such file or directory), client: 127.0.0.1, server: new3-test.apps.atheodor.dev.datahub.redhat.com, request: "GET /vs/workbench/api/worker/extensionHostWorker.js.map HTTP/1.1", host: "new3-test.apps.atheodor.dev.datahub.redhat.com"
127.0.0.1 - user-adri [22/Nov/2023:19:53:04 +0000] "GET /vs/workbench/api/worker/extensionHostWorker.js.map HTTP/1.1" 404 555 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36" "10.39.193.192, 10.129.6.1"
2023/11/22 19:53:05 [error] 29#29: *606 open() "/opt/app-root/src/vs/workbench/api/worker/extensionHostWorker.nls.js.map" failed (2: No such file or directory), client: 127.0.0.1, server: new3-test.apps.atheodor.dev.datahub.redhat.com, request: "GET /vs/workbench/api/worker/extensionHostWorker.nls.js.map HTTP/1.1", host: "new3-test.apps.atheodor.dev.datahub.redhat.com"
127.0.0.1 - user-adri [22/Nov/2023:19:53:05 +0000] "GET /vs/workbench/api/worker/extensionHostWorker.nls.js.map HTTP/1.1" 404 555 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36" "10.39.193.192, 10.129.6.1"

Probably it worth's a shot to check the missing directories for the api/worker/extensionHostWorker.nls.js.map

---

Another attempt was to pass the certs on the nginxconf.sed by adding :

    ssl_certificate /etc/tls/private/tls.crt;
    ssl_certificate_key /etc/tls/private/tls.key;

and changing the liveness & readiness probe schemas to HTTPS. But doesn't work, the pod is unable to start

[atheo89]

The issue appears to be browser-related. Specifically, it does not function properly on Chrome but works correctly on Firefox.

[atheo89]

Continuing from the previous message:

It's important to note that service workers can only operate over an HTTPS connection with a valid SSL certificate.

Chrome the last years has become more restrictive, by rejecting invalid certificates, even after users have accepted the prompt warning them about the certificate. In contrast, other browsers like Firefox impose fewer restrictions, allowing the Jupyter extension to function smoothly in that environment.

For users encountering issues with Chrome, there is a workaround available.

dnf install google-chrome
google-chrome --ignore-certificate-errors --unsafely-treat-insecure-origin-as-secure=${URL}

Furthermore, apart the browser environment this behavior varies also across different clusters due to the certificate assignments being domain-based. For instance, in a cluster where certificates are set up at the cluster level, like the OSD cluster, all subdomains within that domain should operate seamlessly.

[harshad16]

Thanks for all the work, everyone.
Based on the discussion, the sharing the workarounds for this issue:

1. Use a different browser then chrome, seems like the restrictions are coming from chrome.
2. If you wanted to keep using chrome, probably use:

dnf install google-chrome
google-chrome --ignore-certificate-errors --unsafely-treat-insecure-origin-as-secure=${URL}

with these, we would continue find more solution as we moved forward.
for now, we are marking this a completed.