🐛 Destination MSSQL: fix issue with unicode symbols handling (ex.\u2028 showed as ?) #3671
Conversation
There was a problem hiding this comment.
Thanks for also updating the docs!
Once you've addressed the one comment, please bump the connector version like described here and update your PR with the new version: https://docs.airbyte.io/contributing-to-airbyte/building-new-connector#updating-a-connector
then publish the connector like described in the link.
| @@ -53,7 +53,7 @@ public String createTableQuery(String schemaName, String tableName) { | |||
| + "WHERE s.name = '%s' AND t.name = '%s') " | |||
| + "CREATE TABLE %s.%s ( \n" | |||
| + "%s VARCHAR(64) PRIMARY KEY,\n" | |||
| + "%s VARCHAR(MAX),\n" | |||
There was a problem hiding this comment.
@ChristopheDuong do we also need to do the same change in DBT i.e: writing strings as NVARCHARs?
There was a problem hiding this comment.
Normalization is not handling MSSQL yet (so nothing to do there, unless implementing it...), otherwise I don't know how the mssql-dbt adapter plugin handle strings atm
| @@ -53,7 +53,7 @@ public String createTableQuery(String schemaName, String tableName) { | |||
| + "WHERE s.name = '%s' AND t.name = '%s') " | |||
| + "CREATE TABLE %s.%s ( \n" | |||
| + "%s VARCHAR(64) PRIMARY KEY,\n" | |||
| + "%s VARCHAR(MAX),\n" | |||
There was a problem hiding this comment.
@etsybaev can you leave a comment in the code describing why NVARCHAR is used since we typically use VARCHAR?
There was a problem hiding this comment.
@sherifnada done. Added comment to code and documentation. Thanks
|
Hi @sherifnada. |
|
@etsybaev to publish an image you need to:
|
|
/test connector=connectors/destination-mssql
|
|
/publish connector=connectors/destination-mssql
|
What
#3552 MSSQL Destination: Fix newline handling
How
Fixed MSSQL _airbyte_data field's type to support unicode.
This is for Microsoft SQL Server:
NVARCHAR is Unicode - 2 bytes per character, therefore max. of 1 billion characters; will handle East Asian, Arabic, Hebrew, Cyrillic etc. characters just fine.
VARCHAR is non-Unicode - 1 byte per character, max. capacity is 2 billion characters, but limited to the character set you're SQL Server is using, basically - no support for those languages mentioned before
Pre-merge Checklist
Recommended reading order
test.javacomponent.ts