SSL Certificate - certificate_unknown - Android

[nicexavi]

Describe the bug
Hi,
I'm trying to run the BrowserMobProxyServer with an Android device to monitor the traffic.

The problem, it's that I can't monitor the traffic that runs over HTTPS, ONLY for the apps.
The Chrome in my Android device, works fine with the BrowserUp Proxy, but I when I try to access to any app (like Play Store) it won't work, and it throws me a "javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown" exception.

I'm using Selenium to automate the tests and I'm using the BrowserUpProxy version 1.2.1.

I've already installed the ca-certificate-rsa.cer on my Android Device. Still won't work.

To Reproduce
Steps to reproduce the behavior:

1. Create the Embedded BrowserUpProxy with setTrustAllServers to true.
2. Start the Proxy, connect Android to the proxy.

Screenshots
Throws me the error:
[LittleProxy-0-ClientToProxyWorker-0] ERROR org.littleshoot.proxy.impl.ClientToProxyConnection - (NEGOTIATING_CONNECT) [id: 0x53e1bf22, L:0.0.0.0/0.0.0.0:8080 ! R:/10.10.40.77:46362]: Caught an exception on ClientToProxyConnection io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:682) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:617) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:534) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at java.base/java.lang.Thread.run(Thread.java:835) Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131) at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:307) at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:285) at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:180) at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164) at java.base/sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:681) at java.base/sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:636) at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:454) at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:433) at java.base/javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:634) at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:295) at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1332) at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1227) at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1274) at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ... 16 more [LittleProxy-0-ProxyToServerWorker-3] INFO org.littleshoot.proxy.impl.ProxyToServerConnection - (HANDSHAKING) [id: 0x74048cdf, L:0.0.0.0/0.0.0.0:63811]: Connection to upstream server failed javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131) at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:307) at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:285) at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:180) at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164) at java.base/sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:681) at java.base/sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:636) at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:454) at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:433) at java.base/javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:634) at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:295) at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1332) at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1227) at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1274) at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:682) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:617) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:534) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at java.base/java.lang.Thread.run(Thread.java:835)

• Running the BrowserUp Proxy 1.2.1 from Gradle with Java8
• Android 9.0

[ericbeland]

Can you try installing ca-certificate-ec.cer ?

[nicexavi]

I've installed it.

Throws the same error and in ChromeWebBrowser on Android, says me that the connection it's not private.

[ericbeland]

You wrote that you installed "ca-certificate-rsa.cer" not "ca-certificate-ec.cer" which are two different files, so I just wanted to clarify. We ship with both.

[nicexavi]

Sorry about the confusion.

First, I tried to install the RSA and didn't work.

Now, you said me to try the EC and I tried it with the same result.

[ericbeland]

Can you point me to the steps you followed to install it?

[nicexavi]

1. I downloaded the certificates from: https://github.com/browserup/browserup-proxy/tree/master/browserup-proxy-core/src/main/resources/sslSupport
2. I sended the certificates via Telegram to my Android
3. I opened it in my Android and the Android asked me to confirm my pattern
4. I named the certificate with name "rsa-cert-littleproxy"
5. In the "Credential use" i selected "VPN and apps"
6. The certificate it's fully installed into the phone.

(Same steps for the "ca" certificate

Then I started the embebed proxy in the java.
The ChromeBrowser in android works well.
When I use the apps, throws me the "certificate_unknown" error.

[nicexavi]

Any updates on this @ericbeland ?

Do you know what it could be?

Thanks!

[ericbeland]

I'm sorry @nicexavi I don't know off-hand. It sounds like the android apps are maybe separate from the certificate acceptance for chrome. I don't really have any ideas how to fix that, though--I haven't worked on it with chrome apps as that isn't our use-case. We focus on browser based testing. If anyone else has ideas, please chime in.

[Ardesco]

I'm going to guess it's related to Androids new cert policies, you now need to add an exclusion in the app codebase for custom certificates:

https://android-developers.googleblog.com/2016/07/changes-to-trusted-certificate.html

Also look at the Android section here, it gives you all the info you need to fix general Android 7+ certificate errors:

https://www.charlesproxy.com/documentation/using-charles/ssl-certificates/

[nicexavi]

Hi,

It's strange because with CharlesProxy there isn't any problem.

I mean... I can decrypt SSL and do Network Throttling with Charles certificate and all seems be working well.

With Browserup nope.

Where is the difference...?

[Ardesco]

Charles proxy and and browserup proxy use different certificates. I would guess that the exclusion has been added for the Charles proxy certificate, but not for the browserup certificate.

[Ardesco]

You could always try making browserup use your Charles proxy certs and see if that works. Go to Help -> SSL Proxying -> Export Charles root certificate and private key... and then use it like below:

https://github.com/browserup/browserup-proxy/blob/master/browserup-proxy-mitm/src/test/java/com/browserup/bup/mitm/example/CustomCAKeyStoreExample.java

[nicexavi]

Solution:

Include the directive and generate my own certificates.

Network security config:
https://android-developers.googleblog.com/2016/07/changes-to-trusted-certificate.html

Generate certs:
https://github.com/browserup/browserup-proxy/blob/master/browserup-proxy-mitm/src/test/java/com/browserup/bup/mitm/example/SaveGeneratedCAExample.java

Use certs:
https://github.com/browserup/browserup-proxy/blob/master/browserup-proxy-mitm/src/test/java/com/browserup/bup/mitm/example/CustomCAKeyStoreExample.java

Thanks a lot folks!

[ericbeland]

@nicexavi Would you be willing to consider making a README on this topic we can ship and link to from the main readme? It seems like something we could document to leave future breadcrumbs for ourselves and others. Would you be interested in creating a PR with that info, some description of the problem and what you did?

[nicexavi]

Hey!

Yes, I'll do a README to help the community.

I'll to the PR with my other github account named "suitux". It's my personal account instead of the enterprise account.

Thanks!