Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kong does not work with Azure Cosmos Db with Cassandra API #188

Closed
tomkerkhove opened this Issue Oct 30, 2018 · 7 comments

Comments

Projects
None yet
4 participants
@tomkerkhove
Copy link

tomkerkhove commented Oct 30, 2018

I'm trying to get Kong running with Cassandra database via Azure Cosmos Db with Cassandra API. This is using Cassandra drivers which are compatible with CQL v4.

However, while running the migrations it fails because "the host seems unhealthy" while it works fine in the Azure portal. Turning off SSL validation doesn't help as well.

I'm probably doing something wrong here, if not it would be great to have this support.

More information about Cosmos Db support for Cassandra:

Repro

  1. Create an Azure Cosmos Db instance with Cassandra API (docs)
  2. Run Kong migrations as in sample below

Error details

 ⚡ tkerkhove@CLT-TKERKHOVE C:\Kong master
❯ docker run --rm -e "KONG_DATABASE=cassandra" -e "KONG_CASSANDRA_CONTACT_POINTS=kong-datastore.cassandra.cosmosdb.azure.com" -e "KONG_CASSANDRA_SSL=false" -e "KONG_CASSANDRA_KEYSPACE=kongdatastore" -e "KONG_CASSANDRA_USERNAME=kong-datastor
e" -e "KONG_CASSANDRA_PASSWORD=<password>" -e "KONG_CASSANDRA_PORT=10350" kong:latest kong migrations up --v                                                      
2018/10/30 08:58:24 [verbose] Kong: 0.14.1                                                                                                                                                                                                      
2018/10/30 08:58:24 [warn] You are using Cassandra but your 'db_update_propagation' setting is set to '0' (default). Due to the distributed nature of Cassandra, you should increase this value.                                                
Error:                                                                                                                                                                                                                                          
/usr/local/share/lua/5.1/kong/cmd/migrations.lua:33: [cassandra error] all hosts tried for query failed. kong-datastore.cassandra.cosmosdb.azure.com: host seems unhealthy, considering it down (timeout)                                       
stack traceback:                                                                                                                                                                                                                                
        [C]: in function 'assert'                                                                                                                                                                                                               
        /usr/local/share/lua/5.1/kong/cmd/migrations.lua:33: in function 'cmd_exec'                                                                                                                                                             
        /usr/local/share/lua/5.1/kong/cmd/init.lua:87: in function </usr/local/share/lua/5.1/kong/cmd/init.lua:87>                                                                                                                              
        [C]: in function 'xpcall'                                                                                                                                                                                                               
        /usr/local/share/lua/5.1/kong/cmd/init.lua:87: in function </usr/local/share/lua/5.1/kong/cmd/init.lua:44>                                                                                                                              
        /usr/local/bin/kong:7: in function 'file_gen'                                                                                                                                                                                           
        init_worker_by_lua:48: in function <init_worker_by_lua:46>                                                                                                                                                                              
        [C]: in function 'xpcall'                                                                                                                                                                                                               
        init_worker_by_lua:55: in function <init_worker_by_lua:53>                                                                                                                                                                              
2018/10/30 08:58:24 [verbose] no config file found at /etc/kong/kong.conf                                                                                                                                                                       
2018/10/30 08:58:24 [verbose] no config file found at /etc/kong.conf                                                                                                                                                                            
2018/10/30 08:58:24 [verbose] no config file, skipping loading                                                                                                                                                                                  
2018/10/30 08:58:24 [verbose] prefix in use: /usr/local/kong                                                                                                                                                                                    
@coopr

This comment has been minimized.

Copy link

coopr commented Oct 30, 2018

Thanks for your report. We recently did a quick test of Kong + Azure Cosmos DB, and came to the conclusion, after some discussion with Microsoft staff, that Kong does not currently support Cosmos DB, and that the work necessary to cause Kong to support Cosmos DB is more than minor.

In the long term, we do hope to support Cassandra "compatible" datastores like Cosmos DB, but I don't have any timeline to offer.

@coopr coopr closed this Oct 30, 2018

@thibaultcha

This comment has been minimized.

Copy link
Member

thibaultcha commented Oct 30, 2018

Just to clarify a point here: Kong is not compatible with CosmosDB because CosmosDB does not implement several (more than a couple) of the Cassandra features that Kong depends on.
There is no or very little amount of work to be done on Kong itself (in case someone stumbling upon this issue is wondering what a contribution would take), but work to be done by the CosmosDB team itself. Hope that clarifies the situation.

@tomkerkhove

This comment has been minimized.

Copy link
Author

tomkerkhove commented Oct 30, 2018

@tomkerkhove tomkerkhove referenced this issue Nov 2, 2018

Merged

Add install for Microsoft Azure #1018

3 of 4 tasks complete
@tomkerkhove

This comment has been minimized.

Copy link
Author

tomkerkhove commented Nov 13, 2018

Any update on the lacking features that you need to get this supported @thibaultcha? Would love to get people involved so that we can use this as well.

@tomkerkhove

This comment has been minimized.

Copy link
Author

tomkerkhove commented Mar 21, 2019

@thibaultcha what features are lacking? Would love to get it supported on Cosmos.

Looping in @markjbrown

@thibaultcha

This comment has been minimized.

Copy link
Member

thibaultcha commented Mar 21, 2019

@tomkerkhove Hi there,

The specific features that Kong needs as of 1.0 are:

  • Secondary indexes
  • Lightweight transactions
  • Truncate
  • Support for ALTER TABLE
  • Logged batched (starting with 1.1)

We are in touch with some folks on the CosmoDB team since last September. Given their current timeline, it'll be a few more months until CosmoDB implements the above feature, at which point, we'll give the Kong <-> CosmoDB test another go.

Cheers

@markjbrown

This comment has been minimized.

Copy link

markjbrown commented Mar 21, 2019

Hi @thibaultcha I'm going to review this with our Cassandra API team on Cosmos. See where we're at on these.

thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.