[Bug]: Authenticated API not correctly attaching self-signed certificates

[felix-appsmith]

Is there an existing issue for this?

• I have searched the existing issues

Description

Currently, when we have an API that requires authentication through self-signed certificates, even if we attach our PEM file, Appsmith is not able to read it and displays the following error:

org.springframework.web.reactive.function.client.WebClientRequestException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.

Example video:

certificates.mp4

Steps To Reproduce

1. Connect to a self-signed certificate API. You can use this Python script to generate the server with its PEM certificates.

   
from flask import Flask, request, jsonify
import ssl
import os
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography import x509
from cryptography.x509.oid import NameOID
from cryptography.hazmat.primitives import serialization
from datetime import datetime, timedelta
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.backends import default_backend

app = Flask(__name__)

cert_path = os.path.join(os.path.dirname(__file__), 'cert.pem')
key_path = os.path.join(os.path.dirname(__file__), 'key.pem')
if not os.path.exists(cert_path) or not os.path.exists(key_path):
   print("Generando certificado y clave...")
   
   private_key = rsa.generate_private_key(
       public_exponent=65537,
       key_size=2048
   )

   subject = issuer = x509.Name([
       x509.NameAttribute(NameOID.COMMON_NAME, u"localhost")
   ])
   cert = x509.CertificateBuilder().subject_name(
       subject
   ).issuer_name(
       issuer
   ).public_key(
       private_key.public_key()
   ).serial_number(
       x509.random_serial_number()
   ).not_valid_before(
       datetime.utcnow()
   ).not_valid_after(
       datetime.utcnow() + timedelta(days=365)
   ).add_extension(
       x509.SubjectAlternativeName([x509.DNSName(u"localhost")]),
       critical=False,
   ).sign(private_key, hashes.SHA256(), default_backend())

   with open(cert_path, 'wb') as f:
       f.write(cert.public_bytes(encoding=serialization.Encoding.PEM))
   with open(key_path, 'wb') as f:
       f.write(private_key.private_bytes(
           encoding=serialization.Encoding.PEM,
           format=serialization.PrivateFormat.TraditionalOpenSSL,
           encryption_algorithm=serialization.NoEncryption()
       ))
   print("Certificate and key generated at the following location:")
   print(cert_path)
   print(key_path)

context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.load_cert_chain(cert_path, key_path)

users = [
   {"username": "user1", "password": "pass1"},
   {"username": "user2", "password": "pass2"}
]

def authenticate(username, password):
   for user in users:
       if user["username"] == username and user["password"] == password:
           return True
   return False

@app.route('/protected')
def protected():
   auth = request.authorization
   if not auth or not authenticate(auth.username, auth.password):
       return jsonify({'error': 'Unauthorized access'}), 401
   return jsonify({'message': 'Welcome to the protected area!'})

if __name__ == '__main__':
   app.run(ssl_context=context)

Note: install these libraries: pip install flask && pip install cryptography
2. Run this curl command from the directory where you executed the Python code to verify a successful connection.
3. Expose port 5000 through Ngrok to connect to it from Appsmith.
4. Create your data source in Appsmith with an authenticated API.
5. In the URL field, put the URL that Ngrok gave you, plus this endpoint: /protected.
6. Attach the cert.pem certificate generated by the Python code.

Public Sample App

No response

Environment

Production

Issue video log

No response

Version

Appsmith v1.9.12-SNAPSHOT/ Cloud Appsmith Community v1.9.12

[Rytorcen]

Is there any option to disable SSL check within appsmith?

Can confirm the error.
System: self-hosted appsmith on docker tested with self-hosted API (self-signed cert)
API is fully functioning with Python or Postman.

Execution failed with status PE-RST-5000 -org.springframework.web.reactive.function.client.WebClientRequestException

TEST via CLI

docker-compose exec appsmith curl -v https://X.X.X.X
*   Trying X.X.X:X:443...
* TCP_NODELAY set
* Connected to X.X.X.X (X.X.X.X) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to X.X.X.X:443
* Closing connection 0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to X.X.X.X:443

DOCKER LOG

[2023-03-30 12:09:01,778] userEmail=XX, sessionId=723b199e-e188-4eec-af9b-0282da2c6ded, thread=nioEventLoopGroup-3-6, requestId=52a88b1c-d046-45b9-b499-c997af47a74f - Datasource context doesn't exist. Creating connection.
2023-03-30 14:09:01 backend stdout | 
2023-03-30 14:09:01 backend stdout | [2023-03-30 12:09:01,856]  - [28661514, L:/172.19.0.2:38894 ! R:omitted/omitted:443] The connection observed an error
2023-03-30 14:09:01 backend stdout | java.nio.channels.ClosedChannelException: null
2023-03-30 14:09:01 backend stdout |    at io.netty.handler.ssl.SslHandler.channelInactive(SslHandler.java:1065)
2023-03-30 14:09:01 backend stdout |    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:305)
2023-03-30 14:09:01 backend stdout |    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)
2023-03-30 14:09:01 backend stdout |    at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:274)
2023-03-30 14:09:01 backend stdout |    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)
2023-03-30 14:09:01 backend stdout |    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:301)
2023-03-30 14:09:01 backend stdout |    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)
2023-03-30 14:09:01 backend stdout |    at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)
2023-03-30 14:09:01 backend stdout |    at io.netty.channel.AbstractChannel$AbstractUnsafe$7.run(AbstractChannel.java:813)
2023-03-30 14:09:01 backend stdout |    at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)
2023-03-30 14:09:01 backend stdout |    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)
2023-03-30 14:09:01 backend stdout |    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
2023-03-30 14:09:01 backend stdout |    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:403)
2023-03-30 14:09:01 backend stdout |    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
2023-03-30 14:09:01 backend stdout |    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
2023-03-30 14:09:01 backend stdout |    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
2023-03-30 14:09:01 backend stdout |    at java.base/java.lang.Thread.run(Thread.java:833)

[sumitsum]

This issue does not seem valid with v1.9.19. To verify I have followed all the steps as provided in the description, except Appsmith instance was running on my local. I think there is a minor error in the way the test is described. Apart from attaching the self signed certificate we also need to provide the username and password info via the REST API header which turns out to be Authorization: Basic dXNlcjE6cGFzczE= , e.g. the curl request would look like:

curl --cacert cert.pem --header "Authorization: Basic dXNlcjE6cGFzczE=" https://localhost:5000/protected 

Also, as per my understanding, we cannot use ngrok with the python server since ngrok would expose a different endpoint for which we don't have the self signed certificate (the Python script provided creates the self signed certificate for the localhost endpoint).

The same works with Appsmith as well: