Repository navigation
π Add batch to insert records in MSSQL (2100 params limit) - #5272
Conversation
|
/test connector=connectors/destination-mssql
|
Sherif A. Nada (sherifnada)
left a comment
There was a problem hiding this comment.
Can you add a test to protect against regressions?
Subodh Chaturvedi (subodh1810)
left a comment
There was a problem hiding this comment.
I think we should start using BULK INSERT to load data in sql server like we do with other destinations. Not specific to this PR but just wanted to bring it up.
| try { | ||
| SqlOperationsUtils.insertRawRecordsInSingleQuery(insertQueryComponent, recordQueryComponent, database, record); | ||
| } catch (SQLException e) { | ||
| e.printStackTrace(); |
There was a problem hiding this comment.
Shouldn't we throw a RuntimeException right here?
There was a problem hiding this comment.
The function insertRawRecordsInSingleQuery throws a SQLException
There was a problem hiding this comment.
I think he means this suggestion below:
| e.printStackTrace(); | |
| throw new RuntimeException(e); |
Thanks for the suggestion Subodh Chaturvedi (@subodh1810). I'll release this patch version to allow users the MSSQL destination for now. |
|
/test connector=connectors/destination-mssql
|
|
/test connector=connectors/destination-mssql
|
|
/publish connector=connectors/destination-mssql
|
What
Closes #4181
How
Break the insert function to work with batch limited to 700 records.
Recommended reading order
SqlServerOperations.javaPre-merge Checklist
Expand the relevant checklist and delete the others.
New Connector
Community member or Airbyter
airbyte_secret./gradlew :airbyte-integrations:connectors:<name>:integrationTest.README.mddocs/SUMMARY.mddocs/integrations/<source or destination>/<name>.mdincluding changelog. See changelog exampledocs/integrations/README.mdairbyte-integrations/builds.mdAirbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>command is passing./publishcommand described hereUpdating a connector
Community member or Airbyter
airbyte_secret./gradlew :airbyte-integrations:connectors:<name>:integrationTest.README.mddocs/integrations/<source or destination>/<name>.mdincluding changelog. See changelog exampleAirbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>command is passing./publishcommand described hereConnector Generator
-scaffoldin their name) have been updated with the latest scaffold by running./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplatesthen checking in your changes