NFS Broker doesn't currently support Postgres

[siennathesane]

The nfsbroker itself has CLI support baked into it for Postgres, but when you try and leverage it, you get this error:

   2019-05-30T16:07:55.29-0600 [APP/PROC/WEB/0] ERR panic: sql: unknown driver "postgres" (forgotten import?)
   2019-05-30T16:07:55.29-0600 [APP/PROC/WEB/0] ERR goroutine 1 [running]:
   2019-05-30T16:07:55.29-0600 [APP/PROC/WEB/0] ERR code.cloudfoundry.org/lager.(*logger).Fatal(0xc420064600, 0x91893a, 0x19, 0xb5b7c0, 0xc420047c30, 0x0, 0x0, 0x0)
   2019-05-30T16:07:55.29-0600 [APP/PROC/WEB/0] ERR 	/tmp/build/1110d2e9/release-src/src/code.cloudfoundry.org/lager/logger.go:162 +0x611
   2019-05-30T16:07:55.29-0600 [APP/PROC/WEB/0] ERR code.cloudfoundry.org/service-broker-store/brokerstore.NewStore(0xb648e0, 0xc420064600, 0x7ffe50a5b562, 0x8, 0xc420016c40, 0x10, 0xc420014900, 0x19, 0xc42001c4b0, 0x4d, ...)
   2019-05-30T16:07:55.29-0600 [APP/PROC/WEB/0] ERR 	/tmp/build/1110d2e9/release-src/src/code.cloudfoundry.org/service-broker-store/brokerstore/store.go:54 +0x645
   2019-05-30T16:07:55.29-0600 [APP/PROC/WEB/0] ERR main.createServer(0xb648e0, 0xc420064600, 0xc420064600, 0x90b08e)
   2019-05-30T16:07:55.29-0600 [APP/PROC/WEB/0] ERR 	/tmp/build/1110d2e9/release-src/src/code.cloudfoundry.org/nfsbroker/main.go:288 +0x38f
   2019-05-30T16:07:55.29-0600 [APP/PROC/WEB/0] ERR main.main()
   2019-05-30T16:07:55.29-0600 [APP/PROC/WEB/0] ERR 	/tmp/build/1110d2e9/release-src/src/code.cloudfoundry.org/nfsbroker/main.go:159 +0x110

Upon further investigation, the cause is because there is no driver support for Postgres in cloudfoundry/service-broker-store.

In order to resolve this, cloudfoundry/goshims#5 needs to first be addressed to add core shim support. Once that's been resolved, cloudfoundry/service-broker-store#4 needs to be addressed. Once those are handled, in that order, this issue should go away and a new release can be cut.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/166368428

The labels on this github issue will be updated when the story is started.

[siennathesane]

@paulcwarren & @davewalter can you please provide some commentary on this issue? This is an issue for the cloud.gov platform and is blocking the deployment of volumes for customers.

[julian-hj]

@mxplusb sorry for the slow response. We are a bit conflicted about this issue because we are trying to move away from direct SQL support for our broker backing store, and instead standardize on credhub as our storage backend. We currently have migration code to move MySql data into CredHub, but if we pulled this PR, we'd have to develop and test a migration for Postgres also.

Is it possible for you to use CredHub instead of SQL for your storage? That would be more future proof, and would also allow you to use Postgres under the covers without requiring direct support from nfsbroker.

[julian-hj]

this ops file provides an example for how to establish a credhub identity and plumb it through to nfsbroker.

[siennathesane]

Thanks for the heads up on the migration.

Is it possible for you to use CredHub instead of SQL for your storage?

No it is not. We do not use Credhub at the moment, just a vars store for now. There are eventually plans for us to move to a Config Server implementation, but it may or may not be Credhub.

we'd have to develop and test a migration for Postgres also.

I think this extra step is going to be necessary, imo. The broker was marketed with and had basic plumbing for both MySQL and PostgreSQL, so it seems (to me) to be a reasonable expectation that there should be a tested migration plan for PostrgreSQL.

Fixing the bug in PostgreSQL support should take priority so the software can support it as marketed and written, then figure out the rest of the migration story. I appreciate your recommendation of future proofing but I'm more interested in fixing the bug now than a future proof design that I can't use.

[julian-hj]

Hi @mxplusb,

Given all the other issues you've posted to nfsbroker, it looks like your planned usage of the broker diverges quite significantly from the standard cf-deployment path that we actually build and test, so while I take your point that it isn't great that postgres capabilities regressed, we've already moved away from SQL in cf-deployment, quite some time ago. We are a tiny team, with very limited capacity to test non-standard configurations, so it's not really practical for us to invest in supporting capabilities that we soon plan to remove and that haven't been functional for at least a year.

I hate to suggest it, but given your various requirements, probably your best bet is going to be to maintain your own fork of the broker, so that you can make it work with your specific build and deployment requirements.

[julian-hj]

Closing this, as we plan to remove all SQL support from nfsbroker in coming months, and standardize on CredHub.