Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Support IO.copy_stream from a pipe IO instance. #2114

Merged
merged 6 commits into from

3 participants

Warren Seen Dirkjan Bussink Brian Shirai
Warren Seen

IO.copy_stream fails when the IO instance is backed by a pipe, as seek is not
supported on the underlying pipe.

This change required some modification to the copy_stream_spec, I'm open to
suggestions if there's a more preferred way of structuring the spec.

Warren Seen warrenseen Support IO.copy_stream from a pipe IO instance.
IO.copy_stream fails when the IO instance is backed by a pipe, as seek is not
supported on the underlying pipe.

This change required some modification to the copy_stream_spec, I'm open to
suggestions if there's a more preferred way of structuring the spec.
b717b95
spec/ruby/core/io/copy_stream_spec.rb
((8 lines not shown))
+ before :each do
+ @to_io = File.open @to_name, "wb"
+ end
+
+ after :each do
+ @to_io.close
+ end
+
+ it_behaves_like :io_copy_stream_to_io, nil, IOSpecs::CopyStream
+ it_behaves_like :io_copy_stream_to_io_with_offset, nil, IOSpecs::CopyStream
+ end
+ end
+
+ describe "from a pipe IO" do
+ before :each do
+ @from_io = IO.popen "cat #{@from_name}"
Brian Shirai Owner
brixen added a note

Is there a way to do this without cat?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
spec/ruby/core/io/copy_stream_spec.rb
@@ -177,6 +183,50 @@
describe "to a file name" do
it_behaves_like :io_copy_stream_to_file, nil, IOSpecs::CopyStream
+ it_behaves_like :io_copy_stream_to_file_with_offset, nil, IOSpecs::CopyStream
+ end
+
+ describe "to an IO" do
+ before :each do
+ @to_io = File.open @to_name, "wb"
Brian Shirai Owner
brixen added a note

We have a new_io helper. Can this be used here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
kernel/common/io19.rb
@@ -274,7 +274,11 @@ def run
@from.ensure_open_and_readable
@to.ensure_open_and_writable
- saved_pos = @from.pos if @from_io
+ begin
+ saved_pos = @from.pos if @from_io
+ rescue Errno::ESPIPE
Brian Shirai Owner
brixen added a note

Could we do this with a flag on IO instead of rescuing an exception? Causing exceptions in the normal flow is very bad for performance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Dirkjan Bussink
Owner

Do you think you could improve the pull request with the feedback provided? If there's stuff unclear, please let us know!

Warren Seen

@dbussink yep, I've got it sitting here nearly done, hoping to get a chance to finish it off over the weekend.

Warren Seen warrenseen Add pipe flag to IO.
Add a pipe flag to IO class so we can avoid calling #seek on the source IO in copy_stream.
2c7a44b
Warren Seen

@dbussink, it took me a little longer to get around to it, but I've replaced the call to cat with something that should be OS independent but still gives us a pipe to work with.

Brian Shirai brixen merged commit 36dc79f into from
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jan 3, 2013
  1. Warren Seen

    Support IO.copy_stream from a pipe IO instance.

    warrenseen authored
    IO.copy_stream fails when the IO instance is backed by a pipe, as seek is not
    supported on the underlying pipe.
    
    This change required some modification to the copy_stream_spec, I'm open to
    suggestions if there's a more preferred way of structuring the spec.
Commits on Jan 13, 2013
  1. Warren Seen
Commits on Jan 29, 2013
  1. Warren Seen

    Add pipe flag to IO.

    warrenseen authored
    Add a pipe flag to IO class so we can avoid calling #seek on the source IO in copy_stream.
Commits on Feb 3, 2013
  1. Warren Seen
  2. Warren Seen
  3. Warren Seen
This page is out of date. Refresh to see the latest.
19 kernel/common/io19.rb
View
@@ -274,7 +274,11 @@ def run
@from.ensure_open_and_readable
@to.ensure_open_and_writable
- saved_pos = @from.pos if @from_io
+ if @from_io && !@from.pipe?
+ saved_pos = @from.pos
+ else
+ saved_pos = 0
+ end
@from.seek @offset, IO::SEEK_CUR if @offset
@@ -410,6 +414,8 @@ def initialize(fd, mode=undefined, options=undefined)
@external = Encoding.default_external
end
end
+
+ @pipe = false
end
private :initialize
@@ -945,6 +951,14 @@ def close_on_exec?
(fcntl(F_GETFD) & FD_CLOEXEC) != 0
end
+ def pipe=(v)
+ @pipe = !!v
+ end
+
+ def pipe?
+ @pipe
+ end
+
def self.pipe(external=nil, internal=nil, options=nil)
lhs = allocate
rhs = allocate
@@ -961,6 +975,9 @@ def self.pipe(external=nil, internal=nil, options=nil)
lhs.sync = true
rhs.sync = true
+ lhs.pipe = true
+ rhs.pipe = true
+
if block_given?
begin
yield lhs, rhs
66 spec/ruby/core/io/copy_stream_spec.rb
View
@@ -20,11 +20,6 @@
@to_name.should have_data("Line one")
end
- it "copies only length bytes from the offset" do
- IO.copy_stream(@object.from, @to_name, 8, 4).should == 8
- @to_name.should have_data(" one\n\nLi")
- end
-
it "calls #to_path to convert on object to a file name" do
obj = mock("io_copy_stream_to")
obj.should_receive(:to_path).and_return(@to_name)
@@ -41,12 +36,19 @@
end
end
+ describe :io_copy_stream_to_file_with_offset, :shared => true do
+ it "copies only length bytes from the offset" do
+ IO.copy_stream(@object.from, @to_name, 8, 4).should == 8
+ @to_name.should have_data(" one\n\nLi")
+ end
+ end
+
describe :io_copy_stream_to_io, :shared => true do
it "copies the entire IO contents to the IO" do
IO.copy_stream(@object.from, @to_io)
@to_name.should have_data(@content)
IO.copy_stream(@from_bigfile, @to_io)
- @to_io.should have_data(@content + @content_bigfile)
+ @to_name.should have_data(@content + @content_bigfile)
end
it "returns the number of bytes copied" do
@@ -67,7 +69,7 @@
it "raises an IOError if the destination IO is not open for writing" do
@to_io.close
- @to_io = File.open @to_name, "r"
+ @to_io = new_io @to_name, "r"
lambda { IO.copy_stream @object.from, @to_io }.should raise_error(IOError)
end
@@ -80,7 +82,9 @@
IO.copy_stream(@object.from, @to_io, 8).should == 8
@to_name.should have_data("Line one")
end
+ end
+ describe :io_copy_stream_to_io_with_offset, :shared => true do
it "copies only length bytes from the offset" do
IO.copy_stream(@object.from, @to_io, 8, 4).should == 8
@to_name.should have_data(" one\n\nLi")
@@ -107,7 +111,7 @@
describe "from an IO" do
before :each do
- @from_io = File.open @from_name, "rb"
+ @from_io = new_io @from_name, "rb"
IOSpecs::CopyStream.from = @from_io
end
@@ -117,7 +121,7 @@
it "raises an IOError if the source IO is not open for reading" do
@from_io.close
- @from_io = File.open @from_name, "a"
+ @from_io = new_io @from_name, "a"
lambda { IO.copy_stream @from_io, @to_name }.should raise_error(IOError)
end
@@ -140,11 +144,12 @@
describe "to a file name" do
it_behaves_like :io_copy_stream_to_file, nil, IOSpecs::CopyStream
+ it_behaves_like :io_copy_stream_to_file_with_offset, nil, IOSpecs::CopyStream
end
describe "to an IO" do
before :each do
- @to_io = File.open @to_name, "wb"
+ @to_io = new_io @to_name, "wb"
end
after :each do
@@ -152,6 +157,7 @@
end
it_behaves_like :io_copy_stream_to_io, nil, IOSpecs::CopyStream
+ it_behaves_like :io_copy_stream_to_io_with_offset, nil, IOSpecs::CopyStream
end
end
@@ -177,11 +183,49 @@
describe "to a file name" do
it_behaves_like :io_copy_stream_to_file, nil, IOSpecs::CopyStream
+ it_behaves_like :io_copy_stream_to_file_with_offset, nil, IOSpecs::CopyStream
+ end
+
+ describe "to an IO" do
+ before :each do
+ @to_io = new_io @to_name, "wb"
+ end
+
+ after :each do
+ @to_io.close
+ end
+
+ it_behaves_like :io_copy_stream_to_io, nil, IOSpecs::CopyStream
+ it_behaves_like :io_copy_stream_to_io_with_offset, nil, IOSpecs::CopyStream
+ end
+ end
+
+ describe "from a pipe IO" do
+ before :each do
+ @from_io = IOSpecs.pipe_fixture(@content)
+ IOSpecs::CopyStream.from = @from_io
+ end
+
+ after :each do
+ @from_io.close
+ end
+
+ it "does not close the source IO" do
+ IO.copy_stream(@from_io, @to_name)
+ @from_io.closed?.should be_false
+ end
+
+ it "raises an error when an offset is specified" do
+ lambda { IO.copy_stream(@from_io, @to_name, 8, 4) }.should raise_error(Errno::ESPIPE)
+ end
+
+ describe "to a file name" do
+ it_behaves_like :io_copy_stream_to_file, nil, IOSpecs::CopyStream
end
describe "to an IO" do
before :each do
- @to_io = File.open @to_name, "wb"
+ @to_io = new_io @to_name, "wb"
end
after :each do
9 spec/ruby/core/io/fixtures/classes.rb
View
@@ -121,6 +121,15 @@ def self.closed_io
io.close
io
end
+
+ # Creates a pipe-based IO fixture containing the specified
+ # contents
+ def self.pipe_fixture(content)
+ source, sink = IO.pipe
+ sink.write content
+ sink.close
+ source
+ end
# Defines +method+ on +obj+ using the provided +block+. This
# special helper is needed for e.g. IO.open specs to avoid
Something went wrong with that request. Please try again.