Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load data from csv file #94

Open
smarwei opened this issue Jul 28, 2022 · 2 comments
Open

Load data from csv file #94

smarwei opened this issue Jul 28, 2022 · 2 comments

Comments

@smarwei
Copy link

smarwei commented Jul 28, 2022

I wanted to use "copy" to copy data from a csv file to a table.
The postgresql-simple docs do not mention copying from/to a file though. (only stdin)
I tried it nevertheless, and it actually works (all data is loaded into the table), but the copy function gives me an error:
QueryError {qeMessage = "Database.PostgreSQL.Simple.Copy.copy_ CommandOk", [...]

Is it just not intended to load directly from files or is there another way?

@cimmanon
Copy link

How are you copying from a file? This worked for me in older versions:

catch
	(
		-- IO ()
		PG.copy_ c "COPY table_name FROM STDIN (FORMAT CSV, HEADER 1)" >>
		-- IO ()
		(PG.putCopyData c =<< BS.readFile f) >>
		-- IO Int64
		Right <$> PG.putCopyEnd c
		)
	(\(_ :: SomeException) -> return $ Left "Invalid CSV format")

@smarwei
Copy link
Author

smarwei commented Jul 30, 2022

i just used

PG.copy_ c "COPY table_name (col_1, col_2, col_n) FROM '/path/to/file.csv' WITH (format 'csv', QUOTE '"', DELIMITER '|', escape '\', ENCODING 'UTF-8' );"

That's because I didn't think it was necessary to read the file first, since I don't need it for anything other than dumping it to the database.
I would also load csv files up to 10GB in size and thought it might be faster to let Postgres handle that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants