apb CLI command returns status 404 for all sub commands

[alesz]

What happened:
apb CLI command returns status 404 for all sub commands

apb --debug list
Contacting the ansible-service-broker at: https://asb-1338-openshift-ansible-service-broker.apps.oc.example/openshift-automation-service-broker/v2/catalog
Error: Attempt to list APBs in the broker returned status: 404
Unable to list APBs in Ansible Service Broker

Environment:

oc version
oc v3.10.0+dd10d17
kubernetes v1.10.0+b81c8f8
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://oc.example:443
openshift v3.10.0+a5e4ac9-10
kubernetes v1.10.0+b81c8f8

apb version
Version: apb-1.2.12

oc get all -n openshift-ansible-service-broker
NAME              READY     STATUS    RESTARTS   AGE
pod/asb-1-vkvg8   1/1       Running   0          2h

NAME                          DESIRED   CURRENT   READY     AGE
replicationcontroller/asb-1   1         1         1         2h

NAME          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
service/asb   ClusterIP   172.30.216.241   <none>        1338/TCP,1337/TCP   2h

NAME                                     REVISION   DESIRED   CURRENT   TRIGGERED BY
deploymentconfig.apps.openshift.io/asb   1          1         1         config

NAME                                HOST/PORT                                                            PATH      SERVICES   PORT      TERMINATION   WILDCARD
route.route.openshift.io/asb-1338   asb-1338-openshift-ansible-service-broker.apps.oc.example                      asb        1338      reencrypt     None

curl -X GET -H "Authorization: Bearer fEmCaDvbMP31Jkue3_ecB-ycnrKFO8pfZp2js0_4EKU" https://asb-1338-openshift-ansible-service-broker.apps.oc.example/
{
  "paths": [
    "/apis",
    "/healthz",
    "/healthz/ping",
    "/healthz/poststarthook/generic-apiserver-start-informers",
    "/metrics",
    "/osb/"
  ]
}

curl -X GET -H "Authorization: Bearer fEmCaDvbMP31Jkue3_ecB-ycnrKFO8pfZp2js0_4EKU" https://asb-1338-openshift-ansible-service-broke.apps.oc.example/osb/
404 page not found

oc get configmap broker-config -o yaml
apiVersion: v1
data:
  broker-config: "registry:\n  - type: dockerhub\n    name: dh\n    url:  \n    org:
    \ ansibleplaybookbundle\n    tag:  latest\n    white_list: []\n\n    auth_type:
    \"\"\n    auth_name: \"\"\n  - type: local_openshift\n    name: localregistry\n
    \   namespaces: ['openshift']\n    white_list: []\n\ndao:\n  type: crd\nlog:\n
    \ stdout: true\n  level: info\n  color: true\nopenshift:\n  host: \"\"\n  ca_file:
    \"\"\n  bearer_token_file: \"\"\n  namespace: openshift-ansible-service-broker\n
    \ sandbox_role: edit\n  image_pull_policy: Always\n  keep_namespace: false\n  keep_namespace_on_error:
    true\nbroker:\n  dev_broker: false\n  bootstrap_on_startup: true\n  refresh_interval:
    600s\n  launch_apb_on_bind: false\n  output_request: false\n  recovery: true\n
    \ ssl_cert_key: /etc/tls/private/tls.key\n  ssl_cert: /etc/tls/private/tls.crt\n
    \ auto_escalate: False\n  auth:\n    - type: basic\n      enabled: false\n"
kind: ConfigMap
metadata:
  creationTimestamp: 2018-09-06T01:06:58Z
  labels:
    app: openshift-ansible-service-broker
  name: broker-config
  namespace: openshift-ansible-service-broker
  resourceVersion: "59430679"
  selfLink: /api/v1/namespaces/openshift-ansible-service-broker/configmaps/broker-config
  uid: 277843ae-b171-11e8-be2a-0050568936f3

Installed using openshift-ansible/release-3.10:

oc delete project openshift-ansible-service-broker
ansible-playbook -i ../hosts playbooks/openshift-service-catalog/config.yml

asb pod log:

Using config file mounted to /etc/ansible-service-broker/config.yaml
============================================================
==           Creating Ansible Service Broker...           ==
============================================================
time="2018-09-06T01:07:06Z" level=info msg="Ansible Service Broker Version: 1.3.8"
time="2018-09-06T01:07:06Z" level=info msg="Initializing clients..."
time="2018-09-06T01:07:06Z" level=info msg="OpenShift version: v3.10.0+a5e4ac9-10"
time="2018-09-06T01:07:06Z" level=info msg="Kubernetes version: v1.10.0+b81c8f8"
time="2018-09-06T01:07:06Z" level=info msg="Empty AuthType. Assuming credentials are defined in the config... "
time="2018-09-06T01:07:06Z" level=info msg="== REGISTRY CX == "
time="2018-09-06T01:07:06Z" level=info msg="Name: dh"
time="2018-09-06T01:07:06Z" level=info msg="Type: dockerhub"
time="2018-09-06T01:07:06Z" level=info msg="Url: "
time="2018-09-06T01:07:06Z" level=info msg="Empty AuthType. Assuming credentials are defined in the config... "
time="2018-09-06T01:07:06Z" level=info msg="== REGISTRY CX == "
time="2018-09-06T01:07:06Z" level=info msg="Name: localregistry"
time="2018-09-06T01:07:06Z" level=info msg="Type: local_openshift"
time="2018-09-06T01:07:06Z" level=info msg="Url: "
============================================================
==           Starting Ansible Service Broker...           ==
============================================================
time="2018-09-06T01:07:06Z" level=info msg="Initiating Recovery Process"
time="2018-09-06T01:07:06Z" level=info msg="Recovery complete"
time="2018-09-06T01:07:06Z" level=info msg="recover called"
time="2018-09-06T01:07:06Z" level=info msg="Broker configured to bootstrap on startup"
time="2018-09-06T01:07:06Z" level=info msg="Attempting bootstrap..."
time="2018-09-06T01:07:06Z" level=info msg="AnsibleBroker::Bootstrap"
time="2018-09-06T01:07:06Z" level=info msg="0 specs deleted"
time="2018-09-06T01:07:09Z" level=info msg="Bundles filtered by white/blacklist filter:\n\t-> ansibleplaybookbundle/mariadb-apb\n\t-> ansibleplaybookbundle/hello-world-apb\n\t-> ansibleplaybookbundle/rocketchat-apb\n\t-> ansibleplaybookbundle/rds-postgres-apb\n\t-> ansibleplaybookbundle/etherpad-apb\n\t-> ansibleplaybookbundle/wordpress-ha-apb\n\t-> ansibleplaybookbundle/hello-world-db-apb\n\t-> ansibleplaybookbundle/hastebin-apb\n\t-> ansibleplaybookbundle/pyzip-demo-apb\n\t-> ansibleplaybookbundle/jenkins-apb\n\t-> ansibleplaybookbundle/pyzip-demo-db-apb\n\t-> ansibleplaybookbundle/thelounge-apb\n\t-> ansibleplaybookbundle/manageiq-apb\n\t-> ansibleplaybookbundle/nginx-apb\n\t-> ansibleplaybookbundle/mediawiki-apb\n\t-> ansibleplaybookbundle/postgresql-apb\n\t-> ansibleplaybookbundle/mysql-apb\n\t-> ansibleplaybookbundle/photo-album-demo-app-apb\n\t-> ansibleplaybookbundle/s2i-apb\n\t-> ansibleplaybookbundle/es-apb\n\t-> ansibleplaybookbundle/kibana-apb\n\t-> ansibleplaybookbundle/prometheus-apb\n\t-> ansibleplaybookbundle/awx-apb\n\t-> ansibleplaybookbundle/photo-album-demo-ext-api-apb\n\t-> ansibleplaybookbundle/photo-album-demo-api-apb\n\t-> ansibleplaybookbundle/virtualmachines-apb\n\t-> ansibleplaybookbundle/iscsi-demo-target-apb\n\t-> ansibleplaybookbundle/mongodb-apb\n\t-> ansibleplaybookbundle/keycloak-apb\n\t-> ansibleplaybookbundle/mssql-remote-apb\n\t-> ansibleplaybookbundle/proxy-config-apb\n\t-> ansibleplaybookbundle/dynamic-apb\n\t-> ansibleplaybookbundle/mssql-apb\n\t-> ansibleplaybookbundle/eclipse-che-apb\n\t-> ansibleplaybookbundle/origin-recycler\n\t-> ansibleplaybookbundle/apb-tools\n\t-> ansibleplaybookbundle/manageiq-runner-apb\n\t-> ansibleplaybookbundle/import-vm-apb\n\t-> ansibleplaybookbundle/kubevirt-apb\n\t-> ansibleplaybookbundle/v2v-apb\n\t-> ansibleplaybookbundle/vnc-desktop-apb\n\t-> ansibleplaybookbundle/galera-apb\n\t-> ansibleplaybookbundle/openshift-logging-apb\n\t-> ansibleplaybookbundle/rhpam-apb\n\t-> ansibleplaybookbundle/standalone-cinder-apb\n\t-> ansibleplaybookbundle/
gluster-s3object-apb\n\t-> ansibleplaybookbundle/tiller-apb\n\t-> ansibleplaybookbundle/origin-ansible-service-broker\n\t-> ansibleplaybookbundle/hello-world\n\t-> ansibleplaybookbundle/mediawiki123\n\t-> ansibleplaybookbundle/apb-base\n\t-> ansibleplaybookbundle/ansible-service-broker\n\t-> ansibleplaybookbundle/photo-album-demo-api\n\t-> ansibleplaybookbundle/manageiq-apb-runner\n\t-> ansibleplaybookbundle/py-zip-demo\n\t-> ansibleplaybookbundle/mediawiki\n\t-> ansibleplaybookbundle/asb-installer\n\t-> ansibleplaybookbundle/apb-assets-base\n\t-> ansibleplaybookbundle/photo-album-demo-app\n\t-> ansibleplaybookbundle/helm-bundle-base\n\t-> ansibleplaybookbundle/origin\n\t-> ansibleplaybookbundle/vnc-desktop\n\t-> ansibleplaybookbundle/deploy-broker\n\t-> ansibleplaybookbundle/vnc-client\n\t-> ansibleplaybookbundle/origin-service-catalog\n\t-> ansibleplaybookbundle/origin-docker-registry\n\t-> ansibleplaybookbundle/origin-deployer\n\t-> ansibleplaybookbundle/origin-haproxy-router\n\t-> ansibleplaybookbundle/origin-pod\n\t-> ansibleplaybookbundle/origin-sti-builder\n\t-> ansibleplaybookbundle/kubevirt-ansible\n"
time="2018-09-06T01:07:09Z" level=info msg="Validating specs..."
time="2018-09-06T01:07:09Z" level=info msg="All specs passed validation!"
time="2018-09-06T01:07:09Z" level=info msg="Bundles filtered by white/blacklist filter:\n\t-> openshift/alert-bot-apb\n\t-> openshift/dotnet\n\t-> openshift/dotnet-runtime\n\t-> openshift/httpd\n\t-> openshift/jenkins\n\t-> openshift/keycloak-apb\n\t-> openshift/mariadb\n\t-> openshift/mongodb\n\t-> openshift/mysql\n\t-> openshift/nginx\n\t-> openshift/nodejs\n\t-> openshift/oxidized-apb\n\t-> openshift/perl\n\t-> openshift/php\n\t-> openshift/postgresql\n\t-> openshift/python\n\t-> openshift/redis\n\t-> openshift/ruby\n\t-> openshift/web-portal-apb\n\t-> openshift/wildfly\n"
time="2018-09-06T01:07:09Z" level=info msg="Validating specs..."
time="2018-09-06T01:07:09Z" level=info msg="All specs passed validation!"
time="2018-09-06T01:07:09Z" level=info msg="Broker successfully bootstrapped on startup"
time="2018-09-06T01:07:10Z" level=info msg="Listening on https://[::]:1338"
time="2018-09-06T01:07:10Z" level=info msg="Ansible Service Broker Starting"
time="2018-09-06T01:17:09Z" level=info msg="Broker configured to refresh specs every 10m0s seconds"
time="2018-09-06T01:17:09Z" level=info msg="Attempting bootstrap at 2018-09-06 01:17:09.777459633 +0000 UTC"
time="2018-09-06T01:17:09Z" level=info msg="AnsibleBroker::Bootstrap"
time="2018-09-06T01:17:09Z" level=info msg="0 specs deleted"
time="2018-09-06T01:17:13Z" level=info msg="Bundles filtered by white/blacklist filter:\n\t-> ansibleplaybookbundle/kubevirt-ansible\n\t-> ansibleplaybookbundle/hello-world-apb\n\t-> ansibleplaybookbundle/rocketchat-apb\n\t-> ansibleplaybookbundle/rds-postgres-apb\n\t-> ansibleplaybookbundle/etherpad-apb\n\t-> ansibleplaybookbundle/wordpress-ha-apb\n\t-> ansibleplaybookbundle/hello-world-db-apb\n\t-> ansibleplaybookbundle/hastebin-apb\n\t-> ansibleplaybookbundle/pyzip-demo-apb\n\t-> ansibleplaybookbundle/jenkins-apb\n\t-> ansibleplaybookbundle/pyzip-demo-db-apb\n\t-> ansibleplaybookbundle/thelounge-apb\n\t-> ansibleplaybookbundle/manageiq-apb\n\t-> ansibleplaybookbundle/nginx-apb\n\t-> ansibleplaybookbundle/mediawiki-apb\n\t-> ansibleplaybookbundle/postgresql-apb\n\t-> ansibleplaybookbundle/mysql-apb\n\t-> ansibleplaybookbundle/mariadb-apb\n\t-> ansibleplaybookbundle/s2i-apb\n\t-> ansibleplaybookbundle/es-apb\n\t-> ansibleplaybookbundle/kibana-apb\n\t-> ansibleplaybookbundle/prometheus-apb\n\t-> ansibleplaybookbundle/awx-apb\n\t-> ansibleplaybookbundle/photo-album-demo-ext-api-apb\n\t-> ansibleplaybookbundle/photo-album-demo-api-apb\n\t-> ansibleplaybookbundle/photo-album-demo-app-apb\n\t-> ansibleplaybookbundle/iscsi-demo-target-apb\n\t-> ansibleplaybookbundle/mongodb-apb\n\t-> ansibleplaybookbundle/keycloak-apb\n\t-> ansibleplaybookbundle/virtualmachines-apb\n\t-> ansibleplaybookbundle/proxy-config-apb\n\t-> ansibleplaybookbundle/mssql-remote-apb\n\t-> ansibleplaybookbundle/dynamic-apb\n\t-> ansibleplaybookbundle/mssql-apb\n\t-> ansibleplaybookbundle/eclipse-che-apb\n\t-> ansibleplaybookbundle/manageiq-runner-apb\n\t-> ansibleplaybookbundle/import-vm-apb\n\t-> ansibleplaybookbundle/kubevirt-apb\n\t-> ansibleplaybookbundle/v2v-apb\n\t-> ansibleplaybookbundle/vnc-desktop-apb\n\t-> ansibleplaybookbundle/galera-apb\n\t-> ansibleplaybookbundle/openshift-logging-apb\n\t-> ansibleplaybookbundle/rhpam-apb\n\t-> ansibleplaybookbundle/standalone-cinder-apb\n\t-> ansibleplaybookbundle/gluster-s3object-apb\n\t-> ansiblepla
ybookbundle/tiller-apb\n\t-> ansibleplaybookbundle/origin-ansible-service-broker\n\t-> ansibleplaybookbundle/hello-world\n\t-> ansibleplaybookbundle/mediawiki123\n\t-> ansibleplaybookbundle/apb-base\n\t-> ansibleplaybookbundle/ansible-service-broker\n\t-> ansibleplaybookbundle/apb-tools\n\t-> ansibleplaybookbundle/manageiq-apb-runner\n\t-> ansibleplaybookbundle/py-zip-demo\n\t-> ansibleplaybookbundle/mediawiki\n\t-> ansibleplaybookbundle/asb-installer\n\t-> ansibleplaybookbundle/apb-assets-base\n\t-> ansibleplaybookbundle/photo-album-demo-app\n\t-> ansibleplaybookbundle/helm-bundle-base\n\t-> ansibleplaybookbundle/origin\n\t-> ansibleplaybookbundle/photo-album-demo-api\n\t-> ansibleplaybookbundle/deploy-broker\n\t-> ansibleplaybookbundle/vnc-client\n\t-> ansibleplaybookbundle/origin-service-catalog\n\t-> ansibleplaybookbundle/vnc-desktop\n\t-> ansibleplaybookbundle/origin-deployer\n\t-> ansibleplaybookbundle/origin-docker-registry\n\t-> ansibleplaybookbundle/origin-haproxy-router\n\t-> ansibleplaybookbundle/origin-pod\n\t-> ansibleplaybookbundle/origin-sti-builder\n\t-> ansibleplaybookbundle/origin-recycler\n"
time="2018-09-06T01:17:13Z" level=info msg="Validating specs..."
time="2018-09-06T01:17:13Z" level=info msg="All specs passed validation!"
time="2018-09-06T01:17:13Z" level=info msg="Bundles filtered by white/blacklist filter:\n\t-> openshift/alert-bot-apb\n\t-> openshift/dotnet\n\t-> openshift/dotnet-runtime\n\t-> openshift/httpd\n\t-> openshift/jenkins\n\t-> openshift/keycloak-apb\n\t-> openshift/mariadb\n\t-> openshift/mongodb\n\t-> openshift/mysql\n\t-> openshift/nginx\n\t-> openshift/nodejs\n\t-> openshift/oxidized-apb\n\t-> openshift/perl\n\t-> openshift/php\n\t-> openshift/postgresql\n\t-> openshift/python\n\t-> openshift/redis\n\t-> openshift/ruby\n\t-> openshift/web-portal-apb\n\t-> openshift/wildfly\n"
time="2018-09-06T01:17:13Z" level=info msg="Validating specs..."
time="2018-09-06T01:17:13Z" level=info msg="All specs passed validation!"
time="2018-09-06T01:17:13Z" level=info msg="Broker successfully bootstrapped"

[dymurray]

I've tested with latest tooling and everything seems to be working for me. I believe this is probably because the version of the apb tool is using a different suffix to contact the broker than your broker expects (this has changed a few times in the past) so we have provided a command apb config which allows you to set these fields manually. I am closing this issue since it should be resolved if you enter the proper values in apb config