Skip to content
This repository has been archived by the owner on Sep 21, 2022. It is now read-only.

how to deploy cf-mysql with or without setting Load Balancer and p-mysql service bindable ? #226

Closed
zhangrl-mark opened this issue May 23, 2019 · 1 comment
Closed

how to deploy cf-mysql with or without setting Load Balancer and p-mysql service bindable ? #226

zhangrl-mark opened this issue May 23, 2019 · 1 comment
Labels
accepted

Comments

@zhangrl-mark
Copy link

zhangrl-mark commented May 23, 2019 •
edited
Loading

My Evn: vSphere-ESXi.6.5 + VCSA-all.6.7.0 + bosh-deployment(bosh.268.6.0) + cf-deployment(cf.v8.1.0) + cf-mysql-release-36.19.0.tgz

Question1:

Is Mysql service flow: "App/CF-cli <--> Broker <--> Proxy <--> Mysql" Feasible wihtout setting a LB-haproxy in front of the proxy?

cf-mysql-release/docs/proxy.md :
Note: To use MySQL as a bindable service on Cloud Foundry the operator must configure a load balancer, Consul will not be acceessible from applications.

cf_mysql.host:
description: 'Host (DNS) or IP address of the load balancer, which sits above the mysql proxy, used by the broker and bound applications to reach the service'

Question2:

if Mysql service flow is: "App/CF-cli <--> Broker <--> LB-haproxy(ip:236) <--> Proxy(ip:246) <--> Mysql",
and deploy haproxy-boshrelease with -v "haproxy-backend-servers=[192.168.22.246]" point to first Proxy instace(only one),
and deploy cf-mysql-deployment with --var cf_mysql_host = 192.168.22.236 point to LB-haproxy(ip:236),
Is this way OK?
How do i verify the LB-haproxy works OK ?

Case1. Without setting load balancer: haproxy-boshrelease.

cf-mysql-deployment instance: Broker=1, Proxy=1, Mysql=1, arbitrator=1; (only one instance and all locate in azs: [z1])

broker: static_ips: 192.168.22.245
proxy: static_ips: 192.168.22.246
mysql: static_ips: 192.168.22.247
arbitrator: static_ips: 192.168.22.248

I hope the Mysql service flow: App/CF-cli <--> Broker <--> Proxy <--> Mysql.

this flow with only one proxy instance, so donnot config load balancer,
Is this way Feasible?
Is this way Feasible?
Is this way Feasible?

And i am confused about cf_mysql.host:
description: 'Host (DNS) or IP address of the load balancer, which sits above the mysql proxy, used by the broker and bound applications to reach the service'
How shoud i setup cf_mysql_host if i donnot config lb?

// --var cf_mysql_host = Host/IP of BL.
// --var cf_mysql_host=$BOSH_ENVIRONMENT # Why is BOSH_ENVIRONMENT ip address?

my following cmd deploy OK, without: --var cf_mysql_host, SYSTEM_DOMAIN=cfront.com,
bosh -d cf-mysql deploy
cf-mysql-deployment.yml
--vars-store cf-mysql-certs.yml
-o operations/register-proxy-route.yml
-o operations/add-broker.yml
--var cf_mysql_external_host=p-mysql.cfront.com
--var cf_admin_password=16yato05lb491vdgknkf
--var cf_api_url=https://api.cfront.com
--var cf_skip_ssl_validation=true

bosh -d cf-mysql run-errand broker-registrar

But Try Many times And Failed !!!
sometimes, "API endpoint not found at 'https://api.cfront.com' "
sometimes, "creating broker, FAILED, Server error, status code: 502, error code: 10001, message: The service broker returned an invalid response for the request to https..."
What should i do next?

zhangrongliang@zrl-System-Product-Name:~/workdir/CloudFoundry/stemcells/cf-mysql-deployment$ bosh -d cf-mysql run-errand broker-registrar
Using environment '192.168.22.85' as client 'admin'

Using deployment 'cf-mysql'

Task 1224

Task 1224 | 07:47:26 | Preparing deployment: Preparing deployment
Task 1224 | 07:47:27 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 1224 | 07:47:27 | Preparing deployment: Preparing deployment (00:00:01)
Task 1224 | 07:47:27 | Creating missing vms: broker-registrar-vm/9a8673f9-3218-47d4-aa65-3769cab42681 (0) (00:01:13)
Task 1224 | 07:48:40 | Updating instance broker-registrar-vm: broker-registrar-vm/9a8673f9-3218-47d4-aa65-3769cab42681 (0) (canary) (00:00:37)
Task 1224 | 07:49:17 | Running errand: broker-registrar-vm/9a8673f9-3218-47d4-aa65-3769cab42681 (0) (00:00:03)
Task 1224 | 07:49:20 | Fetching logs for broker-registrar-vm/9a8673f9-3218-47d4-aa65-3769cab42681 (0): Finding and packing log files (00:00:01)

Task 1224 Started Thu May 23 07:47:26 UTC 2019
Task 1224 Finished Thu May 23 07:49:21 UTC 2019
Task 1224 Duration 00:01:55
Task 1224 done

Instance broker-registrar-vm/9a8673f9-3218-47d4-aa65-3769cab42681
Exit Code 1
Stdout CF_API_URL=https://api.cfront.com
CF_SKIP_SSL_VALIDATION=true
CF_ADMIN_USERNAME=admin
BROKER_NAME=p-mysql
BROKER_URL=https://p-mysql.cfront.com:443
BROKER_USERNAME=admin
Setting api endpoint to https://api.cfront.com...
FAILED

Stderr API endpoint not found at 'https://api.cfront.com'

1 errand(s)

Errand 'broker-registrar' completed with error (exit code 1)

Exit code 1

zhangrongliang@zrl-System-Product-Name:~/workdir/CloudFoundry/stemcells/cf-mysql-deployment$ bosh -d cf-mysql run-errand broker-registrar
Using environment '192.168.22.85' as client 'admin'

Using deployment 'cf-mysql'

Task 1225

Task 1225 | 07:50:57 | Preparing deployment: Preparing deployment
Task 1225 | 07:50:58 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 1225 | 07:50:58 | Preparing deployment: Preparing deployment (00:00:01)
Task 1225 | 07:50:58 | Creating missing vms: broker-registrar-vm/9a8673f9-3218-47d4-aa65-3769cab42681 (0) (00:01:27)
Task 1225 | 07:52:25 | Updating instance broker-registrar-vm: broker-registrar-vm/9a8673f9-3218-47d4-aa65-3769cab42681 (0) (canary) (00:00:36)
Task 1225 | 07:53:01 | Running errand: broker-registrar-vm/9a8673f9-3218-47d4-aa65-3769cab42681 (0) (00:00:02)
Task 1225 | 07:53:03 | Fetching logs for broker-registrar-vm/9a8673f9-3218-47d4-aa65-3769cab42681 (0): Finding and packing log files (00:00:01)

Task 1225 Started Thu May 23 07:50:57 UTC 2019
Task 1225 Finished Thu May 23 07:53:04 UTC 2019
Task 1225 Duration 00:02:07
Task 1225 done

Instance broker-registrar-vm/9a8673f9-3218-47d4-aa65-3769cab42681
Exit Code 1
Stdout CF_API_URL=https://api.cfront.com
CF_SKIP_SSL_VALIDATION=true
CF_ADMIN_USERNAME=admin
BROKER_NAME=p-mysql
BROKER_URL=https://p-mysql.cfront.com:443
BROKER_USERNAME=admin
Setting api endpoint to https://api.cfront.com...
OK

       api endpoint:   https://api.cfront.com  
       api version:    2.134.0  
       Not logged in. Use 'cf login' to log in.  
       API endpoint: https://api.cfront.com  
       Authenticating...  
       OK  
       Use 'cf target' to view or set your target org and space.  
       Service broker does not exist - creating broker  
       Creating service broker p-mysql as admin...  
       FAILED  
       Server error, status code: 502, error code: 10001, message: The service broker returned an invalid response for the request to https://p-mysql.cfront.com/v2/catalog. Status Code: 302 Found, Body: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">  
       <html><head>  
       <title>302 Found</title>  
       </head><body>  
       <h1>Found</h1>  
       <p>The document has moved <a href="http://cfront.com/index.php/v2/catalog">here</a>.</p>  
       </body></html>  
         
       Service broker already exists - updating broker  
       FAILED  
       Service Broker p-mysql not found

Stderr -

1 errand(s)

Errand 'broker-registrar' completed with error (exit code 1)

Exit code 1

Case2. With setting load balancer: haproxy-boshrelease.

cf-mysql-deployment: Broker=1, LB-haproxy=1, Proxy=1, Mysql=1, arbitrator=1;

broker: static_ips: 192.168.22.245
proxy: static_ips: 192.168.22.246
mysql: static_ips: 192.168.22.247
arbitrator: static_ips: 192.168.22.248

LB-haproxy: static_ips: 192.168.22.236

I hope the Mysql service flow: App/CF-cli <--> Broker <--> LB-haproxy <--> Proxy <--> Mysql.

For Case1 faied after try 'bosh -d cf-mysql run-errand broker-registrar' many times,
and accodrding to the description within cf-mysql-release/docs/proxy.md :

Setting a load balancer in front of the proxies
Note: To use MySQL as a bindable service on Cloud Foundry the operator must configure a load balancer, Consul will not be acceessible from applications.

So, i re-deploy cf-mysql through cf-mysql-deployment.
first, deploy haproxy-boshrelease, with -v haproxy-backend-servers=[192.168.22.246], it point to cf-mysql.Proxy instance IP.
second, deploy cf-mysql-deployment, with --var cf_mysql_host = 192.168.22.236, point to LB-haproxy(ip:236),

Is this way OK?
How do i verify the LB-haproxy works OK ?

//// deploy haproxy-boshrelease and cf-mysql-deployment
LB-haproxy deploy: point to only one proxy instace: 192.168.22.246, and port=3306.
bosh -d haproxy deploy manifests/haproxy.yml
-v haproxy-backend-port=3306
-v "haproxy-backend-servers=[192.168.22.246]"

bosh -d cf-mysql deploy
cf-mysql-deployment.yml
--vars-store cf-mysql-2th-certs.yml
-o operations/register-proxy-route.yml
-o operations/add-broker.yml
-o operations/configure-broker-load-balancer.yml
--var cf_mysql_external_host=p-mysql.cfront.com
--var cf_mysql_host=192.168.22.236
--var cf_admin_password=$CF_ADMIN_PASSWORD
--var cf_api_url=https://api.$SYSTEM_DOMAIN
--var cf_skip_ssl_validation=true

But errors like:
Error executing 'quota violator'.All: dial tcp 192.168.22.236:3306: connect: connection refused"}}
./broker/cf-mysql-broker/quota_enforcer.stdout.log:433:{"timestamp":"1558175908.190212965","source":"Quota Enforcer","message":"Quota Enforcer.Enforcing Failed","log_level":2,"data":{"error":"Finding violators: Error executing 'quota violator'.All: dial tcp 192.168.22.236:3306: connect: connection refused"}}
./broker/cf-mysql-broker/quota_enforcer.stdout.log:435:{"timestamp":"1558175909.190588474","source":"Quota Enforcer","message":"Quota Enforcer.Enforcing Failed","log_level":2,"data":{"error":"Finding violators: Error executing 'quota violator'.All: dial tcp 192.168.22.236:3306: connect: connection refused"}}
./broker/cf-mysql-broker/quota_enforcer.stdout.log:437:{"timestamp":"1558175910.191226959","source":"Quota Enforcer","message":"Quota Enforcer.Enforcing Failed","log_level":2,"data":{"error":"Finding violators: Error executing 'quota violator'.All: dial tcp 192.168.22.236:3306: connect: connection refused"}}
./broker/cf-mysql-broker/db_migrate.combined.log:1:#<Mysql2::Error: Can't connect to MySQL server on '192.168.22.236' (107)>
./broker/cf-mysql-broker/db_migrate.combined.log:2:Couldn't create database for {"adapter"=>"mysql2", "encoding"=>"utf8", "database"=>"mysql_broker", "pool"=>5, "connect_timeout"=>20, "read_timeout"=>20, "reconnect"=>true, "host"=>"192.168.22.236", "port"=>3306, "username"=>"cf-mysql-broker", "password"=>"rgeved3iv8nvlo3b7bhi"}, {:charset=>"utf8", :collation=>"utf8_unicode_ci"}
./broker/cf-mysql-broker/db_migrate.combined.log:5:Mysql2::Error: Can't connect to MySQL server on '192.168.22.236' (107)
./broker/cf-mysql-broker/db_migrate.combined.log:32:#<Mysql2::Error: Can't connect to MySQL server on '192.168.22.236' (107)>
./broker/cf-mysql-broker/db_migrate.combined.log:33:Couldn't create database for {"adapter"=>"mysql2", "encoding"=>"utf8", "database"=>"mysql_broker", "pool"=>5, "connect_timeout"=>20, "read_timeout"=>20, "reconnect"=>true, "host"=>"192.168.22.236", "port"=>3306, "username"=>"cf-mysql-broker", "password"=>"rgeved3iv8nvlo3b7bhi"}, {:charset=>"utf8", :collation=>"utf8_unicode_ci"}
./broker/cf-mysql-broker/db_migrate.combined.log:36:Mysql2::Error: Can't connect to MySQL server on '192.168.22.236' (107)
./broker/cf-mysql-broker/db_migrate.combined.log:63:#<Mysql2::Error: Can't connect to MySQL server on '192.168.22.236' (107)>
@cf-gitbot
Copy link
Collaborator

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/166207442

The labels on this github issue will be updated when the story is started.

@zhangrl-mark zhangrl-mark changed the title how to deploy cf-mysql with or without setting Load Balancer ? how to deploy cf-mysql with or without setting Load Balancer and p-mysql service bindable ? May 23, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
accepted
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@cf-gitbot @zhangrl-mark