Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BULK with Codepage Parameter on Linux #289

Open
leykevin opened this Issue Apr 16, 2018 · 16 comments

Comments

Projects
None yet
6 participants
@leykevin
Copy link

leykevin commented Apr 16, 2018

Hello,
i have the following Error in MSSQL Docker image:
Keyword or statement option 'CODEPAGE' is not supported on the 'Linux' platform. [16202] (severity 15) [(null)]
I have MSSQL installed directly on Debian System and this works but not in the Docker Container.
How can i use the Bulk command with the Parameter Codepage.
I have a CSV Data in ANSI Latin(1252) and want to import it. The docker have access to the files. If i delete the codepage parameter the import ist successfully but with letters like äöü looks bad.

@smartinsightsfromdata

This comment has been minimized.

Copy link

smartinsightsfromdata commented May 30, 2018

Same issue.

@eltermann

This comment has been minimized.

Copy link

eltermann commented Aug 28, 2018

For some reason, CODEPAGE is not available on mssql for Linux.

The documentation says:

CODEPAGE is not a supported option on Linux.
@twright-msft

This comment has been minimized.

Copy link
Collaborator

twright-msft commented Aug 29, 2018

We are going to update the documentation to be more precise. CODEPAGE does actually work on SQL Server on Linux but only for RAW and Unicode.

@eltermann

This comment has been minimized.

Copy link

eltermann commented Aug 31, 2018

@twright-msft, could you please provide an example of a simple BULK INSERT statement that works on SQL Server on Linux and makes use of CODEPAGE?

Also, this documentation update will be great for clarification.

@twright-msft

This comment has been minimized.

Copy link
Collaborator

twright-msft commented Sep 4, 2018

https://docs.microsoft.com/en-us/sql/t-sql/statements/bulk-insert-transact-sql?view=sql-server-2017#examples See example D.
Pasted here for convenience. I changed the example to use CODEPAGE = RAW since that is what works on Linux right now.

BULK INSERT MyTable FROM 'D:\data.csv' WITH ( CODEPAGE = 'RAW', DATAFILETYPE = 'char', FIELDTERMINATOR = ',' );

@DennisGlindhart

This comment has been minimized.

Copy link

DennisGlindhart commented Sep 6, 2018

@twright-msft Unfortunately this example does not seem to work for me - still the same error ('CODEPAGE' is not supported on the 'Linux' platform).

Tried in both Docker & and newest SQL-server installed directly on CentOS.

@twright-msft

This comment has been minimized.

Copy link
Collaborator

twright-msft commented Sep 6, 2018

Can you please run this on your SQL Server on Linux and share the output:
SELECT @@VERSION

@DennisGlindhart

This comment has been minimized.

Copy link

DennisGlindhart commented Sep 6, 2018

@twright-msft

Microsoft SQL Server 2017 (RTM-CU10) (KB4342123) - 14.0.3037.1 (X64) Jul 27 2018 09:40:27 Copyright (C) 2017 Microsoft Corporation Developer Edition (64-bit) on Linux (CentOS Linux 7 (Core))

Reproduce:

Install clean CentOS (7.5 / 1804) (followed by yum update && reboot)

$ cat /etc/locale.conf
LANG="en_DK.UTF-8"
$ curl -o /etc/yum.repos.d/mssql-server.repo https://packages.microsoft.com/config/rhel/7/mssql-server-2017.repo
$ curl -o /etc/yum.repos.d/msprod.repo https://packages.microsoft.com/config/rhel/7/prod.repo
$ yum install -y mssql-server mssql-tools unixODBC-devel
$ /opt/mssql/bin/mssql-conf setup
Edition: Developer, Language: English
$ /opt/mssql-tools/bin/sqlcmd -s localhost -U sa -P 'xxx'
1> SELECT @@VERSION;
2> GO
Microsoft SQL Server 2017 (RTM-CU10) (KB4342123) - 14.0.3037.1 (X64)
Jul 27 2018 09:40:27
Copyright (C) 2017 Microsoft Corporation
Developer Edition (64-bit) on Linux (CentOS Linux 7 (Core))

(1 rows affected)
1> BULK INSERT MyTable FROM '/srv/test.tbl' WITH ( CODEPAGE = 'RAW', DATAFILETYPE = 'char', FIELDTERMINATOR = ',' );
2> GO
Msg 16202, Level 15, State 3, Server srvfab-sql1, Line 1
Keyword or statement option 'CODEPAGE' is not supported on the 'Linux' platform.

@twright-msft

This comment has been minimized.

Copy link
Collaborator

twright-msft commented Sep 6, 2018

I wonder if this is something that is being caught in sqlcmd. Can you please try from a different tool like SQL Server Management Studio or SQL Operations Studio? We added support for RAW in cumulative update 4.

@DennisGlindhart

This comment has been minimized.

Copy link

DennisGlindhart commented Sep 6, 2018

@twright-msft Same issue from SQL Operations Studio (Version 0.28.6 if that makes a difference).
Can it be something with system locale or some related libraries since it (according to @leykevin) works on Debian?

@twright-msft

This comment has been minimized.

Copy link
Collaborator

twright-msft commented Sep 9, 2018

Turns out this was not actually included in a CU as we thought. I'm trying to get clarity on when it will be included. Is bcp an option for you instead of BULK INSERT?

@DennisGlindhart

This comment has been minimized.

Copy link

DennisGlindhart commented Sep 9, 2018

Thank you very much for clarification - that explains it.

BCP is unfortunately not an option in my case, no. I'll just have to wait till CODEPAGE support comes along before continuing with testing - An ETA would be awesome, if possible, yes. Looking forward to it - Thanks.

@DennisGlindhart

This comment has been minimized.

Copy link

DennisGlindhart commented Nov 11, 2018

@twright-msft Seems to be available in 2019 CTP (tested with 2.1) - at least for RAW, but with UNICODE i still get the same error.

Are there any plans for supporting other other codepages?

@twright-msft

This comment has been minimized.

Copy link
Collaborator

twright-msft commented Nov 11, 2018

Glad you got it working with 2.1. We don’t have any plans for other specific code pages right now. Which ones would you like to have?

@DennisGlindhart

This comment has been minimized.

Copy link

DennisGlindhart commented Nov 12, 2018

We are currently using 1252 (Latin) on Windows environment.

We might be able to do with Unicode but it doesn't seem to work on Linux so I haven't tested - only RAW works and that doesn't work due to some date formats.

@MasayukiOzawa

This comment has been minimized.

Copy link

MasayukiOzawa commented Mar 8, 2019

Do you plan to support CODEPAGE = 65001?
Support under UTF - 8 under Linux environment improves convenience.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.