Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PG 1.5.0 may have broken COPY exports #526

Closed
askreet opened this issue Apr 25, 2023 · 2 comments · Fixed by #527
Closed

PG 1.5.0 may have broken COPY exports #526

askreet opened this issue Apr 25, 2023 · 2 comments · Fixed by #527

Comments

@askreet
Copy link

askreet commented Apr 25, 2023

The following example fails with:

.../lib/ruby/gems/3.2.0/gems/pg-1.5.1/lib/pg/connection.rb:406:in `sync_get_copy_data': no COPY in progress (PG::Error)
	from .../lib/ruby/gems/3.2.0/gems/pg-1.5.1/lib/pg/connection.rb:406:in `get_copy_data'
	from .../lib/ruby/gems/3.2.0/gems/pg-1.5.1/lib/pg/connection.rb:249:in `copy_data'
	from test.rb:8:in `<main>'
require 'pg'

conn = PG::Connection.new(host: 'localhost', user: 'postgres', password: 'password', port: 5432, dbname: 'dbname')

conn.copy_data('COPY (SELECT * FROM users) TO STDOUT WITH BINARY') do |res|
  loop do
    data = conn.get_copy_data
    break if data.nil?
  end
end

This works in 1.4.6. It seems that we call get_copy_data an additional time as of v1.5.0 to handle some expected padding from PG, but that does not happen on my system.

 PostgreSQL 14.6 (Debian 14.6-1.pgdg110+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
larskanis added a commit to larskanis/ruby-pg that referenced this issue Apr 26, 2023
... when not using an coder.

To keep compatibility the handling if header and trailer data is now only done when a coder is given.

Fixes ged#526
larskanis added a commit to larskanis/ruby-pg that referenced this issue Apr 26, 2023
... when not using an coder.

To keep compatibility, the handling of header and trailer data is now only done when a coder is given.

Fixes ged#526
@larskanis
Copy link
Collaborator

I added automatic handling of header and trailer data to binary COPY format, which is necessary for PG::BinaryEncoder::CopyRow and PG::BinaryDecoder::CopyRow. I think the best fix is to apply this only when a coder is given to copy_data. Then binary encoding and decoding can be done equally simple as in text format through the new coders and compatibility is preserved to use unmodified binary format with no en/decoder. This is implemented in #527 .

larskanis added a commit to larskanis/ruby-pg that referenced this issue Apr 26, 2023
... when not using an coder.

To keep compatibility, the handling of header and trailer data is now only done when a coder is given.

Fixes ged#526
@askreet
Copy link
Author

askreet commented Apr 26, 2023

Thank you for the rapid fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants