Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Refactored invalid range validation for String#delete! and String#squeeze! #1736

Closed
wants to merge 7 commits into from

3 participants

@antekpiechnik

As suggested by @evanphx here 1835272

I moved the validation of the input for String#delete! and String#squeeze! methods into String#count_table. This allowed for unification of implementation of those methods across 1.8 and 1.9. The validation also now has some ruby-version-specific specs. I changed the validation messages to a more specific one, and added validation error message specs away from RubySpec (as suggested by @dbussink).

String#squeeze! validation itself was fixed (checking all provided arguments now), for which (and String#delete!) I added specific RubySpec specs in separate commits.

This also resolves/contains this pull request: #1735

antekpiechnik added some commits
@antekpiechnik antekpiechnik Added RubySpec for invalid range given as second parameter in String#…
…squeeze in 1.9
0aa720f
@antekpiechnik antekpiechnik Added RubySpec for invalid range given as second parameter in String#…
…delete!
68272ab
@antekpiechnik antekpiechnik Marked the new squeeze spec as failing for ci ea0f27f
@antekpiechnik antekpiechnik Added specs to cover different behaviour of String#count_table in 1.8…
… and 1.9.

Specs that had illegal input have been separated into two ruby versions, and
respective valid-input cases have been provided for both versions.
63a0509
@antekpiechnik antekpiechnik Refactored input validation for String#squeeze and String#delete in 1.9.
Moved the validation to String#count_tables from String#squeeze and String#delete
respectively and separated its implementations for both 1.8 and 1.9.
a4a119d
@antekpiechnik antekpiechnik Unified the implementation of String#delete! and String#squeeze! betw…
…een 1.8 and 1.9.
a688114
@travisbot

This pull request passes (merged a688114 into 95bb2fe).

kernel/common/string19.rb
@@ -22,6 +22,38 @@ def codepoints
alias_method :each_codepoint, :codepoints
+ def count_table(*strings)
+ strings.each do |string|
+ if string =~ /.+\-.+/
+ ranges_found = string.scan(/\w{1}\-\w{1}/)
+ ranges_found.map{ |range| range.gsub(/-/, '').split('') }.each do |range_array|
+ raise ArgumentError, "invalid range \"#{range_array.join('-')}\" in string transliteration" unless range_array == range_array.sort
+ end
+ end
+ end
+
+ table = String.pattern 256, 1
+
+ i, size = 0, strings.size
+ while i < size
@dbussink Owner

Why not validate here? Since here we already walk through the list of strings, now we walk it twice and that's probably more confusing.

Good point, I fixed that in antekpiechnik@2de7d1a

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@travisbot

This pull request passes (merged 2de7d1a into 95bb2fe).

@antekpiechnik

Hey @dbussink, is there anything else this pull request needs in order to get merged ?

@dbussink
Owner

I find the expression still very confusing. Isn't it a better idea to push it further down into tr_expand? There it already determines whether the expression is valid or not so the logic should be much simpler if it's integrated there.

@dbussink
Owner

If you push it own into tr_expand! it means that it is also fixed for String#tr, since that also has a very similar failing tag. It does mean you probably have to change it in the C++ primitive.

@dbussink dbussink referenced this pull request from a commit
@dbussink dbussink Cleanup String#squeeze! and String#delete!
With the change to the String#tr_expand! primitive these custom changes
are no longer necessary. This closes pull requests #1736 and #1665. Both
pull requests had useful stuff that I reused, but both also changed much
more than necessary and also in the wrong places.

Since multiple people apparently took this on and changed things in
different places, I thought it's a good idea to solve this problem
properly now and close the open pull requests.
831b842
@dbussink
Owner

The problem in #1665 was very similar to this one and the change to the primitive was better suited for solving this problem. I did take some of the spec changes from this request so the final solution could be as simple as possible. Therefore I'm closing this pull request.

@dbussink dbussink closed this
@LTe LTe referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on May 19, 2012
  1. @antekpiechnik
  2. @antekpiechnik
  3. @antekpiechnik
  4. @antekpiechnik

    Added specs to cover different behaviour of String#count_table in 1.8…

    antekpiechnik authored
    … and 1.9.
    
    Specs that had illegal input have been separated into two ruby versions, and
    respective valid-input cases have been provided for both versions.
  5. @antekpiechnik

    Refactored input validation for String#squeeze and String#delete in 1.9.

    antekpiechnik authored
    Moved the validation to String#count_tables from String#squeeze and String#delete
    respectively and separated its implementations for both 1.8 and 1.9.
  6. @antekpiechnik
Commits on May 20, 2012
  1. @antekpiechnik
This page is out of date. Refresh to see the latest.
View
70 kernel/common/string.rb
@@ -254,6 +254,30 @@ def delete(*strings)
str.delete!(*strings) || str
end
+ def delete!(*strings)
+ raise ArgumentError, "wrong number of arguments" if strings.empty?
+
+ self.modify!
+
+ table = count_table(*strings).__data__
+
+ i, j = 0, -1
+ while i < @num_bytes
+ c = @data[i]
+ unless table[c] == 1
+ @data[j+=1] = c
+ end
+ i += 1
+ end
+
+ if (j += 1) < @num_bytes
+ self.num_bytes = j
+ self
+ else
+ nil
+ end
+ end
+
def clear
Rubinius.check_frozen
self.num_bytes = 0
@@ -786,6 +810,29 @@ def squeeze(*strings)
str.squeeze!(*strings) || str
end
+ def squeeze!(*strings)
+ return if @num_bytes == 0
+ self.modify!
+
+ table = count_table(*strings).__data__
+
+ i, j, last = 1, 0, @data[0]
+ while i < @num_bytes
+ c = @data[i]
+ unless c == last and table[c] == 1
+ @data[j+=1] = last = c
+ end
+ i += 1
+ end
+
+ if (j += 1) < @num_bytes
+ self.num_bytes = j
+ self
+ else
+ nil
+ end
+ end
+
def start_with?(*prefixes)
prefixes.each do |prefix|
prefix = Rubinius::Type.check_convert_type prefix, String, :to_str
@@ -1088,29 +1135,6 @@ def compare_substring(other, start, size)
raise PrimitiveFailure, "String#compare_substring primitive failed"
end
- def count_table(*strings)
- table = String.pattern 256, 1
-
- i, size = 0, strings.size
- while i < size
- str = StringValue(strings[i]).dup
- if str.size > 1 && str.getbyte(0) == 94 # ?^
- pos, neg = 0, 1
- else
- pos, neg = 1, 0
- end
-
- set = String.pattern 256, neg
- str.tr_expand! nil, true
- j, chars = -1, str.size
- set.setbyte(str.getbyte(j), pos) while (j += 1) < chars
-
- table.apply_and! set
- i += 1
- end
- table
- end
-
def tr_expand!(limit, invalid_as_empty)
Rubinius.primitive :string_tr_expand
raise PrimitiveFailure, "String#tr_expand primitive failed"
View
71 kernel/common/string18.rb
@@ -12,6 +12,29 @@ def self.allocate
alias_method :bytesize, :size
+ def count_table(*strings)
+ table = String.pattern 256, 1
+
+ i, size = 0, strings.size
+ while i < size
+ str = StringValue(strings[i]).dup
+ if str.size > 1 && str.getbyte(0) == 94 # ?^
+ pos, neg = 0, 1
+ else
+ pos, neg = 1, 0
+ end
+
+ set = String.pattern 256, neg
+ str.tr_expand! nil, true
+ j, chars = -1, str.size
+ set.setbyte(str.getbyte(j), pos) while (j += 1) < chars
+
+ table.apply_and! set
+ i += 1
+ end
+ table
+ end
+
def upto(stop, exclusive=false)
stop = StringValue(stop)
return self if self > stop
@@ -36,31 +59,6 @@ def reverse!
self
end
- def delete!(*strings)
- raise ArgumentError, "wrong number of arguments" if strings.empty?
-
- self.modify!
-
- table = count_table(*strings).__data__
-
- i, j = 0, -1
- while i < @num_bytes
- c = @data[i]
- unless table[c] == 1
- @data[j+=1] = c
- end
- i += 1
- end
-
- if (j += 1) < @num_bytes
- self.num_bytes = j
- self
- else
- nil
- end
- end
-
-
def slice!(one, two=undefined)
# This is un-DRY, but it's a simple manual argument splitting. Keeps
# the code fast and clean since the sequence are pretty short.
@@ -90,29 +88,6 @@ def slice!(one, two=undefined)
result
end
- def squeeze!(*strings)
- return if @num_bytes == 0
- self.modify!
-
- table = count_table(*strings).__data__
-
- i, j, last = 1, 0, @data[0]
- while i < @num_bytes
- c = @data[i]
- unless c == last and table[c] == 1
- @data[j+=1] = last = c
- end
- i += 1
- end
-
- if (j += 1) < @num_bytes
- self.num_bytes = j
- self
- else
- nil
- end
- end
-
def sub!(pattern, replacement=undefined)
# Copied mostly from sub to keep Regexp.last_match= working right.
View
92 kernel/common/string19.rb
@@ -22,6 +22,37 @@ def codepoints
alias_method :each_codepoint, :codepoints
+ def count_table(*strings)
+ table = String.pattern 256, 1
+
+ i, size = 0, strings.size
+ while i < size
+ str = StringValue(strings[i]).dup
+
+ if str =~ /.+\-.+/
+ ranges_found = str.scan(/\w{1}\-\w{1}/)
+ ranges_found.map{ |range| range.gsub(/-/, '').split('') }.each do |range_array|
+ raise ArgumentError, "invalid range \"#{range_array.join('-')}\" in string transliteration" unless range_array == range_array.sort
+ end
+ end
+
+ if str.size > 1 && str.getbyte(0) == 94 # ?^
+ pos, neg = 0, 1
+ else
+ pos, neg = 1, 0
+ end
+
+ set = String.pattern 256, neg
+ str.tr_expand! nil, true
+ j, chars = -1, str.size
+ set.setbyte(str.getbyte(j), pos) while (j += 1) < chars
+
+ table.apply_and! set
+ i += 1
+ end
+ table
+ end
+
def encode!(to=undefined, from=undefined, options=nil)
Rubinius.check_frozen
@@ -51,39 +82,6 @@ def prepend(other)
self
end
- def delete!(*strings)
- raise ArgumentError, "wrong number of arguments" if strings.empty?
-
- strings.each do |string|
- if string =~ /.+\-.+/
- ranges_found = string.scan(/\w{1}\-\w{1}/)
- ranges_found.map{ |range| range.gsub(/-/, '').split('') }.each do |range_array|
- raise ArgumentError, "invalid range #{strings} in string transliteration" unless range_array == range_array.sort
- end
- end
- end
-
- self.modify!
-
- table = count_table(*strings).__data__
-
- i, j = 0, -1
- while i < @num_bytes
- c = @data[i]
- unless table[c] == 1
- @data[j+=1] = c
- end
- i += 1
- end
-
- if (j += 1) < @num_bytes
- self.num_bytes = j
- self
- else
- nil
- end
- end
-
def upto(stop, exclusive=false)
return to_enum :upto, stop, exclusive unless block_given?
stop = StringValue(stop)
@@ -121,34 +119,6 @@ def reverse!
self
end
- def squeeze!(*strings)
- if strings.first =~ /.+\-.+/
- range = strings.first.gsub(/-/, '').split('')
- raise ArgumentError, "invalid range #{strings} in string transliteration" unless range == range.sort
- end
-
- return if @num_bytes == 0
- self.modify!
-
- table = count_table(*strings).__data__
-
- i, j, last = 1, 0, @data[0]
- while i < @num_bytes
- c = @data[i]
- unless c == last and table[c] == 1
- @data[j+=1] = last = c
- end
- i += 1
- end
-
- if (j += 1) < @num_bytes
- self.num_bytes = j
- self
- else
- nil
- end
- end
-
def sub!(pattern, replacement=undefined)
# Copied mostly from sub to keep Regexp.last_match= working right.
View
58 spec/core/string/count_table_spec.rb
@@ -20,22 +20,70 @@
256.times { |i| t.getbyte(i).should == ("-|*\030 abcde".include?(i.chr) ? 1 : 0) }
end
- it "returns a String with entries corresponding to sequences set to 1" do
- t = "".count_table "-ab-g\\d-abhi--A\\"
+ it "returns a String with entries corresponding to sequences set to 1 without an invalid range" do
+ t = "".count_table "-ab-g\\f-hbhi--A\\"
256.times { |i|
- t.getbyte(i).should == ("-abcdefgbhA\\".include?(i.chr) ? 1 : 0)
+ t.getbyte(i).should == ("-abcdefgfghbhA\\".include?(i.chr) ? 1 : 0)
}
end
+ ruby_version_is "" ... "1.9" do
+ it "returns a String with entries corresponding to sequences set to 1 ignoring an invalid range" do
+ t = "".count_table "-ab-g\\d-abhi--A\\"
+ 256.times { |i|
+ t.getbyte(i).should == ("-abcdefgbhA\\".include?(i.chr) ? 1 : 0)
+ }
+ end
+ end
+
+ ruby_version_is "1.9" do
+ it "raises an exception for input provided with invalid range" do
+ lambda { "".count_table "-ab-g\\d-abhi--A\\" }.should raise_error(ArgumentError)
+ end
+ end
+
it "returns a String with entries corresponding included and excluded characters" do
t = "".count_table "ab", "^bc"
256.times { |i| t.getbyte(i).should == (i == 97 ? 1 : 0) }
end
it "returns a String with entries set to false for '^abc-e' inverse match strings" do
- t = "".count_table "^ab-g\\d-abhi--A\\"
+ t = "".count_table "^ab-g\\f-hbhi--A\\"
256.times { |i|
- t.getbyte(i).should == ("abcdefgbhA\\".include?(i.chr) ? 0 : 1)
+ t.getbyte(i).should == ("abcdefgfghbhA\\".include?(i.chr) ? 0 : 1)
}
end
+
+ ruby_version_is "" ... "1.9" do
+ it "returns a String with entries set to false for '^abc-e' inverse match strings ignoring the invalid range" do
+ t = "".count_table "^ab-g\\d-abhi--A\\"
+ 256.times { |i|
+ t.getbyte(i).should == ("abcdefgbhA\\".include?(i.chr) ? 0 : 1)
+ }
+ end
+ end
+
+ ruby_version_is "1.9" do
+ it "raises an exception for '^abc-e' inverse match strings with an invalid range" do
+ lambda { "".count_table "^ab-g\\d-abhi--A\\" }.should raise_error(ArgumentError)
+ end
+ end
+
+ describe "input validation" do
+ it "raises no exception when input is correct" do
+ lambda { "".count_table("abc") }.should_not raise_error
+ end
+
+ it "raises no exception when input contains a correct range" do
+ lambda { "".count_table("a-c") }.should_not raise_error
+ end
+
+ ruby_version_is "1.9" do
+ it "raises a proper ArgumentError when input contains a range out of sequence" do
+ lambda { "".count_table("h-e") }.should raise_error(ArgumentError, 'invalid range "h-e" in string transliteration')
+ lambda { "".count_table("^h-e") }.should raise_error(ArgumentError, 'invalid range "h-e" in string transliteration')
+ lambda { "".count_table("abc", "ash-e") }.should raise_error(ArgumentError, 'invalid range "h-e" in string transliteration')
+ end
+ end
+ end
end
View
1  spec/ruby/core/string/delete_spec.rb
@@ -64,6 +64,7 @@
it "raises if the given ranges are invalid" do
lambda { "hello".delete("h-e") }.should raise_error(ArgumentError)
lambda { "hello".delete("^h-e") }.should raise_error(ArgumentError)
+ lambda { "hello".delete("abc", "h-e") }.should raise_error(ArgumentError)
end
end
View
1  spec/ruby/core/string/squeeze_spec.rb
@@ -123,6 +123,7 @@
s = "--subbookkeeper--"
lambda { s.squeeze!("e-b") }.should raise_error(ArgumentError)
lambda { s.squeeze!("^e-b") }.should raise_error(ArgumentError)
+ lambda { s.squeeze!("abc", "e-b") }.should raise_error(ArgumentError)
end
end
Something went wrong with that request. Please try again.