Error: Bad Certificate

[software-engineer-mj]

I successfully deployed the app-autoscaler-release on AWS.

Everything works well except for the metricscollector and scalingengine APIs.

I can use the apiserver APIs.

apiserver/740d157f-8e3f-43fc-bd0b-28d3b43075aa:~$ curl https://apiserver.service.cf.internal:6100/v1/policies/45c39971-41c6-4fb2-b999-a4fc33068329 --insecure
{"instance_max_count":4,"instance_min_count":1,"scaling_rules":[{"adjustment":"-1","breach_duration_secs":600,"cool_down_secs":300,"metric_type":"memoryused","operator":"<","stat_window_secs":300,"threshold":30},{"adjustment":"+1","breach_duration_secs":600,"cool_down_secs":300,"metric_type":"memoryused","operator":">=","stat_window_secs":300,"threshold":90}],"schedules":{"recurring_schedule":[{"days_of_week":[1,2,3],"end_time":"18:00","initial_min_instance_count":5,"instance_max_count":10,"instance_min_count":1,"start_time":"10:00"},{"days_of_month":[5,15,25],"end_date":"2099-07-23","end_time":"19:30","initial_min_instance_count":5,"instance_max_count":10,"instance_min_count":3,"start_date":"2099-06-27","start_time":"11:00"},{"days_of_week":[4,5,6],"end_time":"18:00","instance_max_count":10,"instance_min_count":1,"start_time":"10:00"},{"days_of_month":[10,20,30],"end_time":"19:30","instance_max_count":10,"instance_min_count":1,"start_time":"11:00"}],"specific_date":[{"end_date_time":"2099-06-15T13:59","initial_min_instance_count":2,"instance_max_count":4,"instance_min_count":1,"start_date_time":"2099-06-02T10:00"},{"end_date_time":"2099-02-19T23:15","initial_min_instance_count":3,"instance_max_count":5,"instance_min_count":2,"start_date_time":"2099-01-04T20:00"}],"timezone":"Asia/Shanghai"}}

When I try to access the metricscollector, it says that the certificate is not valid.

apiserver/740d157f-8e3f-43fc-bd0b-28d3b43075aa:~$ curl https://metricscollector.service.cf.internal:6103 --insecure
curl: (35) error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate

curl -v https://metricscollector.service.cf.internal:6103 --cacert ca.crt 
* Rebuilt URL to: https://metricscollector.service.cf.internal:6103/
* Hostname was NOT found in DNS cache
*   Trying 10.244.4.7...
* Connected to metricscollector.service.cf.internal (10.244.4.7) port 6103 (#0)
* successfully set certificate verify locations:
*   CAfile: ca.crt
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Request CERT (13):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS alert, Server hello (2):
* error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate
* Closing connection 0
curl: (35) error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate

I attached my yml file.

app-autoscaler-release.zip

Any help would be really appreciated!

[kanekoh]

Hi @kmj940101,

I tried to execute same command line and then, curl command version 7.35 returns same error, when k/insecure option is specified. I haven't found any cause of this error.

If you want to access metrics collector and scaling engine, please try to use client certification. The following is an example.

$  curl https://metricscollector.service.cf.internal:6103/v1/apps/88ead269-e996-4811-96e6-7f6d6111b026/metric_histories/memoryused --cacert ca.crt --cert client.crt --key client.key -s | python -m json.tool | head
[
    {
        "app_id": "88ead269-e996-4811-96e6-7f6d6111b026",
        "collected_at": 1496223864818097647,
        "instance_index": 0,
        "name": "memoryused",
        "timestamp": 1496223860321666557,
        "unit": "megabytes",
        "value": "20"
    },

Metricscollector and Scalingengine certification is where the following server and path.

• Metricscollector certification file and private key file (client.crt, client.key)
  • eventgenerator server: /var/vcap/jobs/eventgenerator/config/certs/metricscollector
• Scalingengine certification file and private key file (client.crt, client.key)
  • eventgenerator server: /var/vcap/jobs/eventgenerator/config/certs/scalingengine/
  • scheduler server: /var/vcap/jobs/scheduler/config/certs/scalingengine/

[software-engineer-mj]

@kanekoh Thank you for your reply.

I followed your instruction and it worked well. I want to use the APIs from the web.

The bosh-lite that I am using forwards ports through a linux gateway with iptables.

This is what it looks like, when I execute "sudo iptables -t nat -L PREROUTING"

DNAT       tcp  --  anywhere             anywhere             tcp dpt:6100 to:< apiserver/0 IP >:6100
DNAT       tcp  --  anywhere             anywhere             tcp dpt:6101 to:< servicebroker/0 IP >:6101
DNAT       tcp  --  anywhere             anywhere             tcp dpt:6102 to:< scheduler/0 IP >:6102
DNAT       tcp  --  anywhere             anywhere             tcp dpt:6103 to:< metricscollector/0 IP >:6103
DNAT       tcp  --  anywhere             anywhere             tcp dpt:6104 to:< scalingengine/0 IP >:6104

When tried to access the metricscollector and scalingengine from the web, it did not work.

Any idea? I really appreciate your help!

[apiserver] https://< bosh-lite IP >:6100

[servicebroker] https://< bosh-lite IP >:6101

[scheduler] https://< bosh-lite IP >:6102

[metricscollector] https://< bosh-lite IP >:6103

[scalingengine] https://< bosh-lite IP >:6104

This site can’t provide a secure connection

< bosh-lite IP> didn’t accept your login certificate, or one may not have been provided.
Try contacting the system admin.
ERR_BAD_SSL_CLIENT_AUTH_CERT

[kanekoh]

Currently, public endpoint of app-autoscaler is not exposed.
if that is what you are looking for.

[software-engineer-mj]

@kanekoh Thank you for your comment.

I have another question, which is similar to the first one.

If there is no public endpoint, I want to access the APIs through the internal IPs.

This is what it looks like when I execute bosh vms app-autoscaler-release.

Deployment 'app-autoscaler-release'

Director task 1062

Task 1062 done

+-----------------------------------------------------------+---------+----+---------+------------+
| VM                                                        | State   | AZ | VM Type | IPs        |
+-----------------------------------------------------------+---------+----+---------+------------+
| apiserver/0 (740d157f-8e3f-43fc-bd0b-28d3b43075aa)        | running | z1 | default | 10.244.4.3 |
| eventgenerator/0 (7d9dfaba-58d5-44b6-957e-5751bdd9f6f4)   | running | z1 | default | 10.244.4.8 |
| metricscollector/0 (7acec1c7-d1ef-4d42-8fed-40056be97174) | running | z1 | default | 10.244.4.7 |
| postgres/0 (d1ba65c0-be26-4f8e-a15f-30b92b80bd43)         | running | z1 | default | 10.244.4.2 |
| pruner/0 (81ae6650-9cd6-40cc-80cf-3ecaeebdb0cc)           | running | z1 | default | 10.244.4.6 |
| scalingengine/0 (95da74c6-d647-42ef-a5f9-48748f0ff2b0)    | running | z1 | default | 10.244.4.9 |
| scheduler/0 (f64dfdb3-b62f-4d01-9345-63d227ac5935)        | running | z1 | default | 10.244.4.4 |
| servicebroker/0 (54db3272-f28c-489a-9232-58b41e6aa845)    | running | z1 | default | 10.244.4.5 |
+-----------------------------------------------------------+---------+----+---------+------------+

[apiserver]

ubuntu@agent-id-bosh-0:~$ curl https://10.244.4.3:6100/v1/policies/487fb8cb-ff6c-4e75-b8e0-8bc9df07f8de -s -k | python -m json.tool | head 
{
    "instance_max_count": 5,
    "instance_min_count": 1,
    "scaling_rules": [
        {
            "adjustment": "+1",
            "breach_duration_secs": 60,
            "cool_down_secs": 60,
            "metric_type": "memoryused",
            "operator": ">",

[scheduler]

ubuntu@agent-id-bosh-0:~$ curl https://10.244.4.4:6102/v2/schedules/45c39971-41c6-4fb2-b999-a4fc33068329 -s -k | python -m json.tool | head
{
    "instance_max_count": null,
    "instance_min_count": null,
    "schedules": {
        "recurring_schedule": [
            {
                "app_id": "45c39971-41c6-4fb2-b999-a4fc33068329",
                "days_of_month": null,
                "days_of_week": [
                    1,

[metricscollector]

ubuntu@agent-id-bosh-0:~$ curl https://10.244.4.7:6103/v1/apps/487fb8cb-ff6c-4e75-b8e0-8bc9df07f8de/metric_histories/memoryused -k
curl: (35) error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate

Any idea what causes this error to occur when trying to use the metricscollector APIs?

Thank you.

[kanekoh]

Please specify certificates as I mentioned before.

[software-engineer-mj]

I tried that one a couple of days ago and it worked well.

[https://metricscollector.service.cf.internal:6103]

eventgenerator/7d9dfaba-58d5-44b6-957e-5751bdd9f6f4:/var/vcap/jobs/eventgenerator/config/certs/metricscollector$ curl https://metricscollector.service.cf.internal:6103/v1/apps/487fb8cb-ff6c-4e75-b8e0-8bc9df07f8de/metric_histories/memoryused --cacert ca.crt --cert client.crt --key client.key -s | python -m json.tool | head
[
    {
        "app_id": "487fb8cb-ff6c-4e75-b8e0-8bc9df07f8de",
        "collected_at": 1496643256315000531,
        "instance_index": 0,
        "name": "memoryused",
        "timestamp": 1496643228032619352,
        "unit": "megabytes",
        "value": "683"
    }

What I want to do is to access the APIs through the internal IPs.

[https://10.244.4.7:6103]

curl https://10.244.4.7:6103/v1/apps/487fb8cb-ff6c-4e75-b8e0-8bc9df07f8de/metric_histories/memoryused 
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: http://curl.haxx.se/docs/sslcerts.html

curl https://10.244.4.7:6103/v1/apps/487fb8cb-ff6c-4e75-b8e0-8bc9df07f8de/metric_histories/memoryused --cacert ca.crt --cert client.crt --key client.key
curl: (51) SSL: certificate subject name 'metricscollector.service.cf.internal' does not match target host name '10.244.4.7'

[Certificate Decoder]

eventgenerator/7d9dfaba-58d5-44b6-957e-5751bdd9f6f4:/var/vcap/jobs/eventgenerator/config/certs/metricscollector$ openssl x509 -in client.crt -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            50:c3:f2:2e:d7:74:6e:78:48:eb:fc:3d:4a:8b:29:51
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=autoscalerCA
        Validity
            Not Before: Mar 12 23:41:50 2017 GMT
            Not After : Mar 12 23:41:50 2019 GMT
        Subject: CN=metricscollector

Thank you in advance for any help you can provide!

[kanekoh]

Please try to specify -k/--insecure and client certification (--cert, --key) options. The following is a sample.

$ curl https://10.244.4.7:6103/v1/apps/88ead269-e996-4811-96e6-7f6d6111b026/metric_histories/memoryused --insecure --cert client.crt --key client.key -s | python -m json.tool | head
[
    {
        "app_id": "88ead269-e996-4811-96e6-7f6d6111b026",
        "collected_at": 1496223864818097647,
        "instance_index": 0,
        "name": "memoryused",
        "timestamp": 1496223860321666557,
        "unit": "megabytes",
        "value": "20"
    },