v4.0.0

Features

• React to new setting in platform, ditating whether to stream edge properties or not.

Chores

• Updated to .NET6
• Disable MARS (multiple active result sets).
  This is to mirror change in platform, and to improve performance.

Fixes

• Prepend tablename with underscore, if it starts with a digit, since table names cannot start with digits in mssql.
• Cap length column names, and append hash value of name, to ensure uniqueness

Changes:

• b987ded Merge branch 'release/4.0.0'
• ecc7902 Merge remote-tracking branch 'origin/master' into release/4.0.0
• 75bb604 chore: Update references (#113)
• dc69866 chore: Update the readme file (#112)
• ba46fd7 chore: Add index for EntityId in edge properties tables (#110)
• 69352e0 fix: Set max of 101 characters for main table name, to leave enough space for suffixes (#109)
• 11e6946 feat: Use option for streaming edge properties (#103)
• 0cd225b fix: Cap length of column name (#104)
• d40aba1 chore: Add 4.0.0 release notes (#105)
• da2c666 fix: Prepend table names starting with digit (#102)

See More

• 207f497 fix: Explicitly dispose readers (#101)
• 58304fe chore: Disable multiple active result sets in connection string (#100)
• b7f2cff Merge branch 'support/3.7' into develop
• 3113473 feat: Update to .NET 6 (#98)
• 28ffd2e chore: Clean up build
• 7e0cedd chore: Bump version to next major
• f4bae2b chore: Update platform dependencies
• 55c0776 chore: Update to .NET 6
• 8326222 Merge branch 'master' into develop
• 26570e4 Merge branch 'release/3.7.0' into develop

This list of changes was auto generated.

v3.7.1

Features

• Introduce new column on code table IsDataPartOriginEntityCode

  IsDataPartOriginEntityCode indicates whether the code is the origin entity code in any dataparts, of the entity.
  No action is required to upgrade to this version, but certain actions can be taken to populate the column with values faster then it otherwise would.
  See: UpgradeNotes/3.7.1/3.7.1-upgrade-notes.md

Changes:

• 0dc1e77 Merge branch 'support/3.7'
• 27c7aad feat: Add IsDataPartOriginEntityCode column on code table (#99)

This list of changes was auto generated.

Release

3.7.0 introduces a new schema, that will support more data to be exported, and make it easier to run quries related to edges.

Features

• Introduce new schema

  • Main table now always contains persist version.
  • Code table is unchanged.
  • Edge table has been replaced by two tables, one for outgoing edges, and one for incoming edges.
    Edge tables now contains Id of edge, id of entity it belongs to, edge type and either ToCode (for outgoing edges) or FromCode (for incoming edges)
  • Edge properties tables have been introduced, which will contain the properties on outgoing and incoming edges respectivly.
    The edge properties tables will the id of the edge they belong to, the name of the key, and the value of the property.

• Export will use custom table types
  All export of multirow data (code, edge and edge properties) will make use of custom table typed parameters.
  This is to ensure that an arbitrary amount of rows can be inserted, instead of being limited by the maximum about of parameters supported by sql (2100).

Upgrade story

In order to upgrade to this version of the connector, if you already have a running stream using an earlier verion,
do the following for all streams using the connector:

• Disable the stream
• Upgrade the connector (ie. the nuget package)
• Reprocess the stream
• Reenable the stream

This will result in the existing tables being arhived, and new tables being created.
These tables will be empty, so everything will need to be exported anew, to populate the tables again.
If you have not reprocessed the stream, an error will be logged, providing instructions to do the upgrade.

For more detailed upgrade notes, including screenshots, see 3.7.0-upgrade-notes.md

Changes:

• b4aed84 Merge branch 'release/3.7.0'
• ed9d0fe Merge branch 'develop' into release/3.7.0
• c08d0bd Merge branch 'master' into develop
• a39c53e chore: Remove dependency on connector common (#97)
• 4842121 chore: Add entity id to edge properties table (#96)
• a882cb6 fix: Set correct column types in event mode (#95)
• d324dc3 chore: Add upgrade notes for release 3.7.0 (#94)
• a6167fb Merge pull request #93 from CluedIn-io/release/3.7.0
• 96f3ad3 Merge pull request #92 from CluedIn-io/chore/merge-develop-to-release370
• a7fc334 Merge branch 'develop' into chore/merge-develop-to-release370

See More

• d46ab71 chore: Remove wrong reading of row (#89)
• 623684d chore: Use simpler command for syncing edges, when none is present (#88)
• 228b097 fix: Filter out default columns from properties (#87)
• 82ff91e Merge pull request #90 from CluedIn-io/chore/revert-reverts
• 26b8296 Revert "Revert "fix: Rework VerifyExistingContainer, to log exception if table is incompatible (#82)""
• e7d1e6a Revert "Revert "feat: Throw exception if old table exists (#78)""
• 6783708 Revert "feat: Throw exception if old table exists (#78)"
• 0991c61 Revert "fix: Rework VerifyExistingContainer, to log exception if table is incompatible (#82)"
• 18a67cc chore: Add release notes for 3.7.0 (#85)
• 1a24b8f fix: Rework VerifyExistingContainer, to log exception if table is incompatible (#82)
• 87299cf chore: Change EntityType column type to NVARCHAR(1024) (#84)
• 2da02ca fix: Add index on entity id in edge tables (#83)
• aa052a8 feat: Throw exception if old table exists (#81)
• c657c82 chore: Simplify passing of timestamp (#80)
• de16b52 chore: Simplify passing of timestamp (#79)
• 019b0a5 feat: Throw exception if old table exists (#78)
• 009b4a4 feat: Implement IConnectorV2 (#77)

This list of changes was auto generated.

Release

Features

• Supports CluedIn 3.5.3

Fixes

• Fixes unnecessary SQL permission grants causing create container to fail with a restricted write user

Changes:

• e1371f1 Merge branch 'release/3.6.0'
• c34677b fix: Remove grants execution on created type
• e911690 chore: Grant reference and view permissions on CodeTableType (#75)
• 7db3131 Merge branch 'master' into develop
• 46c7f2d Merge pull request #74 from CluedIn-io/release/3.5.3

This list of changes was auto generated.

Release

Features

• Supports CluedIn 3.5.3

Changes:

• f23b717 Merge branch 'release/3.5.3'
• d6a71bf chore: Update Common connector reference to 3.5.3 (#73)
• e45c02e Merge branch 'master' into develop
• d1ea009 Merge pull request #72 from CluedIn-io/release/3.4.0

This list of changes was auto generated.

Release

The 3.4.0 update focuses mainly on stabiltity and error handling, and has no semantic changes.

Features

Add ability to configure database schema
Allow configuring the default max column size

Chores

Set id to be unique identifier
Update reference to common connector package
Add indexes on primary table (see upgrade notes)
Execute commands in transactions

Fixes

Add retrying of failed commands
Insert edges with a constant number of parameters

Changes:

• ffb9618 Merge branch 'release/3.4.0'
• bad8c83 chore: Update common connector reference to 3.4.4 (#71)
• 0f1cd4d fix: Copy datatypes when creating tables, to avoid duplicates on retry (#70)
• 6ff9ac4 fix: Specify type when granting EXEC on CodeTableType (#69)
• 1394406 chore: Pass caught exceptions as inner exceptions of thrown exceptions (#68)
• 04c98c4 chore: Update common connector reference (#67)
• 90db5f7 feat: Allow configuration of default collumn size (#65)
• d15a8b1 fix: Grant EXEC rights on CodeTableType (#66)
• 7985faf fix: Set max pool size to 200 (#64)
• f416c22 chore: Add 3.4.0 release notes (#61)

See More

• c1d6f28 fix: Set nvarchar limit to max (#63)
• 9198ba7 fix: Add custom types when creating container (#62)
• 7c1ea04 fix: Insert edges via table (#60)
• 7e94e29 feat: Execute commands in transactions (#58)
• 8d99923 fix: Filter on table when verifying indexes (#59)
• 54a94f6 fix: Bail out of VerifyExistingContainer, if tables does not exist (#57)
• 5e48bcb feat: Change indexes for primary table (#55)
• 26a780f chore: Add basic retry logic (#54)
• 0e5636e chore: Give better name to upgrade feature (#52)
• e3802c2 feat: Introduce support for database schema configuration (#51)
• 2df5ce3 chore: Update Connector.Common to support underscore (#50)
• 6f92888 Merge pull request #48 from CluedIn-io/fix/uniqueidentifier-type
• 9f51029 Update src/Connector.SqlServer/Features/DefaultBuildCreateIndexFeature.cs
• ebf0f62 Update src/Connector.SqlServer/Features/DefaultBuildCreateIndexFeature.cs
• 0cd01d6 improvement: use uniqueidentifer for entity guid to improve performance
• 338a839 improvement: use uniqueidentifer for entity guid to improve performance
• 64f9948 Merge pull request #45 from CluedIn-io/fix/limit-nvarchar-size
• be8ed85 fix: tests to support nvarchar(1024)
• 044c380 chore: add limit of 1024 to nvarchar field
• 3a3a84d Merge pull request #44 from CluedIn-io/feature/configurable-edges-syncing
• c8bb745 workaround: option to disable syncing Edges table
• 171ee17 chore: Bump version - gitversion issues (#42)
• ff3b428 chore: Prepare develop for 3.4 (#41)

This list of changes was auto generated.

Release

Fixes

• CLR types that do not map to SQL types will be serialized as JSON

Changes:

• b258583 Merge branch 'release/3.3.0'
• e1b1859 chore: Update version numbers
• 29d03ea fix: Correct casing of config model AB#7136 (#40)
• e3fabd4 fix: Config requires camelCasing (#38)
• c1dddf1 fix: Ensure schedule and api limit set for provider (#37) [ #6966 ]
• 03b1b5f Matrix (#35)
• 2cfdbe8 Merge pull request #36 from CluedIn-io/fix/330
• 4a65552 fix: for not showing supported modes correctly in 330
• 3846c33 Merge pull request #34 from CluedIn-io/chore/master-to-dev
• b33df14 Merge remote-tracking branch 'origin/develop' into chore/master-to-dev

See More

• 160f069 Merge pull request #31 from CluedIn-io/chore/325-into-develop
• eadb800 fix: correct broken test post merge
• 073aa0d fix: ensure parameter value is serialized if unknown type
• 4c7c56e Merge branch 'release/3.2.5' into develop
• 9c05614 feat: CLR types that do not map to SQL types will be serialized as JSON (#27)
• 23bfe0d Nuget package version is behind head creating commit to start build
• a0b75c3 Nuget package version is behind head creating commit to start build
• 9cd9297 Nuget package version is behind head creating commit to start build
• fcc9d59 Nuget package version is behind head creating commit to start build
• f6568b4 Nuget package version is behind head creating commit to start build
• 0219ac8 Nuget package version is behind head creating commit to start build
• 5b63e98 Nuget package version is behind head creating commit to start build
• f406d76 Updated logo to svg
• 1c57d1a Merge pull request #7 from CluedIn-io/feature/master-into-dev
• 958c098 Merge tag '3.2.2' into feature/master-into-dev
• e8e8fc3 (Temporary change to be reverted back) Disable Unit tests
• 0b93c2c (Temporary change to be reverted back) Just to get Streams going at Lederne
• e84a21f chore: bump version

This list of changes was auto generated.

Release

Features

• Supports Streams multi modes (Event Stream, Sync)
• Serialize complex property values (e.g. collections) as JSON rather than as strings

Fixes

• Sql Server Connector table upgrade causing error when not existing (AB#3496)

Changes:

• d779358 Merge branch 'release/3.2.5'
• e3b0f5f fix: Ensure parameter value is serialized if unknown type (#33)
• e0799e8 fix: automate nuget auth for build (#32)
• d06b1ec Merge pull request #30 from CluedIn-io/fix/Verify-Schema-tries-to-add-TimeStamp-column-to-non-existent-table [ #3496 ]
• b88970f For - Sql Server Connector table upgrade causing error when not existing (AB#3496)
• 5e1d15c Merge pull request #29 from CluedIn-io/bugFix/update-displayname-authmethods
• 760caae chore: Change the display name for the Database Name
• e83496e chore: configure codeowners (#28)
• 44fda19 feat: Add time stamp column to sync mode (#25) [ #2762 ]
• fbf1b6a chore: version bump

See More

• 9e3f216 feat: Stream dual mode (#23)

This list of changes was auto generated.

Release

Features

• Exposes IBuildStoreDataFeature abstraction which can be replaced by extension authors to modify sql persistance statements
• Exposes IBuildCreateContainerFeature abstraction which can be replaced by extension authors to modify sql container creation statements
• Exposes IBuildCreateIndexFeature abstraction which can be replaced by extension authors to modify sql index creation statements
• Exposes IBuildDeleteDataFeature abstraction which can be replaced by extension authors to modify sql delete statements
• Exposes IBulkStoreDataFeature abstraction which can be replaced by extension authors to
  modify bulk sql persistence statements. NOTE this is experimental at present
• Exposes IBulkDeleteDataFeature abstraction which can be replaced by extension authors to modify bulk sql persistence statements. NOTE this is experimental at present

Fixes

• When provided data cannot be mapped to a sql type it is not written to the table
• When no data can be mapped to a sql type an error is thrown

Changes:

• 1b3a16b Merge branch 'release/3.2.3'
• c62d5bd fix: correctly generate inserts for codes table
• 7a6dda5 feature: Support delete and code table exapansion (#21)
• 3e5c16a Merge pull request #18 from CluedIn-io/feature/323_Performance-Improvements
• 54cd389 Updated release notes
• 3faec76 feat(bulksql): use bulk sql features within connector if enabled
• a5ae90c feat(bulksql): switch default client for bulk sql client
• a468c07 feat(bulksql): include bulksql in feature store
• c9fdf9b fix(bulksql): remove breaking changes
• 21f4de5 feat(bulksql): Implement feature for bulk data persistence

See More

• 78759e9 feat(bulksql): implement IBulkSqlClient to prevent breaking changes
• 322a4fb chore: bump to cluedin 3.2.3
• 3a0c29a Fixed connection string tests
• 38f85a6 Removed unwanted logging and fixed bad field name in default key fields
• 6607f61 Basics of Bulk Inserts added
• fa392d9 Added logging and connection pooling
• f84d6a2 Revert "POC:Switch from MERGE INTO to INSERT"
• 858669d Revert "POC:Switch from MERGE INTO to INSERT"
• d8596d9 POC:Switch from MERGE INTO to INSERT
• 1c6562d POC:Switch from MERGE INTO to INSERT
• c7e4043 Adds indexes to generated tables (#13) [ #12 ]
• 1c6b4c0 fix(container): make create container feature consistent (#10)
• 3e3cba2 Merge pull request #9 from CluedIn-io/feature/sql-feature
• 6a99cc6 Added feature for Create Container, and ability to specify Key Fields used when merging in StoreData
• 8615b6c Merge pull request #8 from CluedIn-io/feature/sql-feature
• c05a2bc fix: have invalid fields typed as strings if unknown
• 752d5d0 chore: added release notes
• 85436db fix: when selected column cannot be mapped remove from persistance
• 80ac762 feature(features): implement interfaces to support feature override

This list of changes was auto generated.

Release

Fixes

• Ensure port number is used when creating a connection

Changes:

• 1559673 Merge pull request #6 from CluedIn-io/release/3.2.2
• c7b3e4a Temporarily disable vocab key datatype resolution as it needs reviewing to make sure all types are supported
• 6051427 Merge pull request #4 from CluedIn-io/feature/stream-edges
• 6756e4d fix: bump targeted version for cluedin
• e41068b wip: support local debugging/building
• 1f318c5 feat: support persistance of edge data
• 13afe45 feat: support crud for edge tables
• 8916943 fix: Ensure port number default is used when getting job data
• 3a593ba fix: error handling for port value
• 7508989 fix: use port number in connectionstring [ #2 ]

This list of changes was auto generated.