Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev patch #11

Closed
wants to merge 11 commits into from
Closed

Dev patch #11

wants to merge 11 commits into from

Conversation

pflooky
Copy link

@pflooky pflooky commented Sep 30, 2023

  • Column relationships defined at dataset level
    • Use pattern ?(<dataset_uuid>.)<table_name>.<column_name>?(.<nested_column_name>+) to allow for cross dataset relationships
  • Generic connection options
    • Similar to Spark style connections where a map of string -> string is defined
    • Valid connection options defined by connection type
    • Allows for other data source definitions to be defined such as JMS, non-JDBC databases, HTTP
  • Standardise primaryKey, partitionKey and clusterKey
  • Add in primaryKeyPosition, partitionKeyPosition, clusterKeyPosition and isUnique for column definition

jgperrin and others added 11 commits July 27, 2023 10:40
As requested
…aset level, standardise primaryKey, partitionKey and clusterKey names, add in key position, isUnique and nested columns for column definition, suggestion for generic connection type and options to accommodate for other data sources
# Conflicts:
#	docs/README.md
@kopparam kopparam mentioned this pull request Oct 2, 2023
@jgperrin
Copy link

jgperrin commented Oct 2, 2023

See my messages on Slack. Great ideas, but you introduced breaking changes at the same time.

@jgperrin jgperrin closed this Oct 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants