Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cassandra Assets: improve examples and had astra-keyspace asset #375

Merged
merged 16 commits into from
Sep 7, 2023

Conversation

eolivelli
Copy link
Member

@eolivelli eolivelli commented Sep 7, 2023

Summary:

  • move the example applications to use the new assets
  • implement a new astra-keyspace, as for AstraDB we need some special handling with a different SDK and you also need the AstraCS token and the database name
  • in the Astra DataSource we support now both username/password and clientId/secret, this is because by default the Astra UI calls them that way, it shows 3 fields: clientId, secret and token.
  • Change the AssetManager API in order to allow to cache the DataSource

The new astra-keyspace works this way:

assets:
 - name: "products-keyspace"
    asset-type: "astra-keyspace"
    creation-mode: create-if-not-exists
    config:
      keyspace: "products"
      datasource: "AstraDBDatasource"

Unfortunately in order to make it work you must provide more configurations to the AstraDBDatasource

secrets.yaml

secrets:
  - name: astra
    id: astra
    data:
      secureBundle: "<file:secure-connect-database.zip>"            
      clientId: "qCZpZkEXtKiDGQlffHIhNAHT"
      secret: "xxxx"
      token: "AstraCS:xxxxxx"
      database: "xxxxxx"

configuration.yaml

configuration:
  resources:
    - type: "vector-database"
      name: "AstraDatasource"
      configuration:
        service: "astra"
        clientId: "{{{ secrets.astra.clientId }}}"
        secret: "{{{ secrets.astra.secret }}}"
        secureBundle: "{{{ secrets.astra.secureBundle }}}"
        token: "{{{ secrets.astra.token }}}"
        database: "{{{ secrets.astra.database }}}"

The values for "token" and for "database" are required only by the astra-keyspace asset, you don't need to configure them normally

@eolivelli eolivelli changed the title Cassandra Assents: improve examples and had astradb-keyspace asset Cassandra Assets: improve examples and had astradb-keyspace asset Sep 7, 2023
@eolivelli eolivelli changed the title Cassandra Assets: improve examples and had astradb-keyspace asset Cassandra Assets: improve examples and had astra-keyspace asset Sep 7, 2023
@eolivelli eolivelli marked this pull request as ready for review September 7, 2023 16:14
@eolivelli eolivelli merged commit 547187d into main Sep 7, 2023
8 checks passed
@eolivelli eolivelli deleted the impl/cassandra-assets-2 branch September 7, 2023 19:24
benfrank241 pushed a commit to vectorize-io/langstream that referenced this pull request May 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant