Skip to content

Dest-MSSQL: MSSQL Uses BulkLoader Interface - #56353

Merged
Johnny Schmidt (johnny-schmidt) merged 1 commit into
masterfrom
jschmidt/dest-mssql/mssql-uses-bulk-load
Apr 2, 2025
Merged

Johnny Schmidt (johnny-schmidt) merged 1 commit into
masterfrom
jschmidt/dest-mssql/mssql-uses-bulk-load

Conversation

@johnny-schmidt

@johnny-schmidt Johnny Schmidt (johnny-schmidt) commented Mar 23, 2025 •

Copy link
Copy Markdown
Contributor

What

MSSQL Uses the BulkLoader interface described here .

This basically just moves the work that was being done for bulk load in the BulkLoaderStreamLoader into MSSQLBulkLoader. Start/close still live in the stream loader. A small statestore is broken out to pass info from start/close to the factory. Additionally there's some micronaut magic to make it possible to conditionally inject a bean based on whether we're configured for BulkLoad.

This has been heavily tested, all relevant fixes and improvements are in underlying PRs. Expect a slight perf improvement in the cloud. (Also I verified that I didn't break StandardInsert.)

@vercel

vercel Bot commented Mar 23, 2025 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
airbyte-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 1, 2025 10:15pm

@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch from 05373cc to 86edc20 Compare March 23, 2025 21:30
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch from 86edc20 to fdf2ed1 Compare March 23, 2025 22:25
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch from fdf2ed1 to eddfa04 Compare March 23, 2025 22:34
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) marked this pull request as ready for review March 23, 2025 22:37
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch from eddfa04 to fff4705 Compare March 23, 2025 23:36
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch from fff4705 to bf0e6be Compare March 24, 2025 16:05
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/load-cdk/bulk-load-interface branch from 521ecaa to e495fdd Compare March 24, 2025 19:41
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch from bf0e6be to a2c9ed0 Compare March 24, 2025 19:46
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch from a2c9ed0 to 7136bb5 Compare March 24, 2025 20:32
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/load-cdk/bulk-load-interface branch from e495fdd to e277af2 Compare March 24, 2025 20:53
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch from 7136bb5 to ad6bf14 Compare March 24, 2025 20:59
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/load-cdk/bulk-load-interface branch 2 times, most recently from e0ef829 to f84a68f Compare March 25, 2025 21:44
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch from ad6bf14 to 10cf83c Compare March 25, 2025 21:53
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/load-cdk/bulk-load-interface branch from f84a68f to 1287217 Compare March 27, 2025 17:56
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch from 10cf83c to a7b5381 Compare March 27, 2025 18:18
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch from a7b5381 to 95f429e Compare March 27, 2025 20:11
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch 4 times, most recently from bb164b4 to 258d44c Compare March 27, 2025 20:20
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/load-cdk/bulk-load-interface branch from 31913e5 to 2ab32ca Compare March 27, 2025 20:31
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch from 258d44c to ef6d9ca Compare March 27, 2025 20:32
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/load-cdk/bulk-load-interface branch from 2ab32ca to 364c547 Compare March 28, 2025 17:51
Base automatically changed from jschmidt/load-cdk/bulk-load-interface to master March 28, 2025 17:57
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch from ef6d9ca to c7c25f3 Compare March 28, 2025 17:59
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch from c7c25f3 to 0d82af4 Compare March 28, 2025 19:05
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch 2 times, most recently from e650f75 to 4bac608 Compare March 28, 2025 20:03
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) force-pushed the jschmidt/dest-mssql/mssql-uses-bulk-load branch from c9b831a to 0ed47b2 Compare April 1, 2025 22:04
@johnny-schmidt
Johnny Schmidt (johnny-schmidt) deleted the jschmidt/dest-mssql/mssql-uses-bulk-load branch April 2, 2025 23:48

This branch was successfully deployed

1 active deployment
Preview — 0ed47b26 Deployed Apr 1, 2025 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants