Advent of Changelog: Day 13

zverok · zverok · commit bf77067833ca · 2023-12-13T23:58:27.000+02:00
diff --git a/_src/3.3.md b/_src/3.3.md
@@ -27,9 +27,12 @@ description: Ruby 3.3 full and annotated changelog
 * **Code:**
   ```ruby
   [1, 2, 3].pack('r*')
-  # Ruby 3.1: "", no warning
-  # Ruby 3.2: "", warning: unknown pack directive 'r' in 'r*'
-  # Ruby 3.3: in `pack': unknown pack directive 'r' in 'r*' (ArgumentError)
+  # Ruby 3.1:
+  #   => "", no warning
+  # Ruby 3.2:
+  #   => "", warning: unknown pack directive 'r' in 'r*'
+  # Ruby 3.3:
+  #   in `pack': unknown pack directive 'r' in 'r*' (ArgumentError)
   ```
 * **Notes:**
 
@@ -66,9 +69,11 @@ Two methods to accept an integer file descriptor as an argument: `for_fd` create
   #=> "/home/zverok/projects/ruby/doc" -- the current path have changed successfully
 
   # A block form of fchdir is available, like for a regular .chdir:
-  Dir.fchdir(Dir.new('NEWS').fileno) do
+  Dir.fchdir(Dir.new('NEWS').fileno) do |*args|
+    p args #=> [] -- no arguments are passed into the block
     p Dir.pwd #=> "/home/zverok/projects/ruby/doc/NEWS"
-  end
+    'return value'
+  end #=> "return value"
   Dir.pwd #=> "/home/zverok/projects/ruby/doc" -- back to the path before the block
   ```
 * **Notes:**
@@ -89,9 +94,11 @@ An instance method version of [Dir.chdir](https://docs.ruby-lang.org/en/master/D
   Dir.pwd #=> "/home/zverok/projects/ruby/doc"
 
   # The block form works, too:
-  Dir.new('NEWS').chdir do
+  Dir.new('NEWS').chdir do |*args|
+    p args #=> [] -- no arguments are passed into the block
     Dir.pwd #=> "/home/zverok/projects/ruby/doc/NEWS"
-  end
+    'return value'
+  end #=> "return value"
   Dir.pwd #=> "/home/zverok/projects/ruby/doc"
   ```
 
@@ -118,63 +125,63 @@ Allows to assign a string to be rendered as class/module's `#name`, without assi
 * **Documentation:** [Module#set_temporary_name](https://docs.ruby-lang.org/en/master/Module.html#method-i-set_temporary_name)
 * **Code:**
   ```ruby
-dynamic_class = Class.new do
-  def foo; end
-end
+  dynamic_class = Class.new do
+    def foo; end
+  end
 
-dynamic_class.name #=> nil
+  dynamic_class.name #=> nil
 
-# For dynamic classes, representation of related values is frequently unreadable:
-dynamic_class #=> #<Class:0x0...>
-instance = dynamic_class.new #=> #<#<Class:0x0...>:0x0...>
-instance.method(:foo) #=> #<Method: #<Class:0x0...>#foo() ...>
+  # For dynamic classes, representation of related values is frequently unreadable:
+  dynamic_class #=> #<Class:0x0...>
+  instance = dynamic_class.new #=> #<#<Class:0x0...>:0x0...>
+  instance.method(:foo) #=> #<Method: #<Class:0x0...>#foo() ...>
 
-dynamic_class::Nested = Module.new
-dynamic_class::Nested #=> #<Class:0x0...>::Nested
+  dynamic_class::Nested = Module.new
+  dynamic_class::Nested #=> #<Class:0x0...>::Nested
 
-# After assigning the temporary name, representation becomes more convenient:
-dynamic_class.set_temporary_name("MyDSLClass(with description)")
+  # After assigning the temporary name, representation becomes more convenient:
+  dynamic_class.set_temporary_name("MyDSLClass(with description)")
 
-dynamic_class #=> MyDSLClass(with description)
-instance #=> #<MyDSLClass(with description):0x0...>
-instance.method(:foo) #=> #<Method: MyDSLClass(with description)#foo() ...>
+  dynamic_class #=> MyDSLClass(with description)
+  instance #=> #<MyDSLClass(with description):0x0...>
+  instance.method(:foo) #=> #<Method: MyDSLClass(with description)#foo() ...>
 
-# Note that module constant names are assigned at the moment of their creation,
-# and don't change when the temporary name is assigned:
-dynamic_class::OtherNested = Module.new
+  # Note that module constant names are assigned at the moment of their creation,
+  # and don't change when the temporary name is assigned:
+  dynamic_class::OtherNested = Module.new
 
-dynamic_class::Nested #=> #<Class:0x0...>::Nested
-dynamic_class::OtherNested #=> MyDSLClass(with description)::OtherNested
+  dynamic_class::Nested #=> #<Class:0x0...>::Nested
+  dynamic_class::OtherNested #=> MyDSLClass(with description)::OtherNested
 
-# Assigning names that correspond to constant name rules is prohibited:
-dynamic_class.set_temporary_name("MyClass")
-# `set_temporary_name': the temporary name must not be a constant path to avoid confusion (ArgumentError)
-dynamic_class.set_temporary_name("MyClass::NestedName")
-# `set_temporary_name': the temporary name must not be a constant path to avoid confusion (ArgumentError)
+  # Assigning names that correspond to constant name rules is prohibited:
+  dynamic_class.set_temporary_name("MyClass")
+  # `set_temporary_name': the temporary name must not be a constant path to avoid confusion (ArgumentError)
+  dynamic_class.set_temporary_name("MyClass::NestedName")
+  # `set_temporary_name': the temporary name must not be a constant path to avoid confusion (ArgumentError)
 
-# When the module with a temporary name is put into a constant,
-# it receives a permanent name, which can't be changed anymore
-C = dynamic_class
+  # When the module with a temporary name is put into a constant,
+  # it receives a permanent name, which can't be changed anymore
+  C = dynamic_class
 
-# It affects all associated values (including modules)
+  # It affects all associated values (including modules)
 
-dynamic_class #=> C
-instance #=> #<C:0x0...>
-instance.method(:foo) #=> #<Method: C#foo() ...>
-dynamic_class::Nested #=> C::Nested
-dynamic_class::OtherNested #=> C::OtherNested
+  dynamic_class #=> C
+  instance #=> #<C:0x0...>
+  instance.method(:foo) #=> #<Method: C#foo() ...>
+  dynamic_class::Nested #=> C::Nested
+  dynamic_class::OtherNested #=> C::OtherNested
 
-dynamic_class.set_temporary_name("Can I have it back?")
-# `set_temporary_name': can't change permanent name (RuntimeError)
+  dynamic_class.set_temporary_name("Can I have it back?")
+  # `set_temporary_name': can't change permanent name (RuntimeError)
 
-# `nil` can be used to cleanup a temporary name:
-other_class = Class.new
-other_class.set_temporary_name("another one")
-other_class #=> another one
-other_class.set_temporary_name(nil)
-other_class #=> #<Class:0x0...>
+  # `nil` can be used to cleanup a temporary name:
+  other_class = Class.new
+  other_class.set_temporary_name("another one")
+  other_class #=> another one
+  other_class.set_temporary_name(nil)
+  other_class #=> #<Class:0x0...>
   ```
-* **Notes:** Any phrase that used as a temporary name would be used verbatim; this might create very confusing `#inspect` results and error messages; so it is advised to use strings somehow implying that the name belong to a module. Imagine we wrap into classes with temporary names RSpec-style examples, and then there is a typo in such example:
+* **Notes:** Any phrase that used as a temporary name would be used verbatim; this might create very confusing `#inspect` results and error messages; so it is advised to use strings somehow implying that the name belong to a module. Imagine we wrap into classes with temporary names RSpec-style examples, and then there is a typo in the body of such example:
   ```ruby
   it "works as a calculator" do
     expec(2+2).to eq 4
@@ -200,6 +207,20 @@ A new "weak map" concept implementation. Unlike `ObjectSpace::WeakMap`, it compa
 * **Discussion:** [Feature #18498]
 * **Documentation:** [ObjectSpace::WeakKeyMap](https://docs.ruby-lang.org/en/master/ObjectSpace/WeakKeyMap.html)
 * **Code:**
+  ```ruby
+  map = ObjectSpace::WeekMap.new
+
+  key = "foo"
+  map[key] = true
+  map["foo"] #=> true
+
+  key = nil
+  GC.start
+
+  map["foo"] #=> nil
+
+  TODO
+  ```
 * **Notes:** The class interface is significantly leaner than `WeakMap`'s, and doesn't provide any kind of iteration methods (which is very hard to implement and use correctly with weakly-referenced objects), so the new class is more like a black box with associations than a collection.
 
 ### `ObjectSpace::WeakMap#delete`
@@ -232,13 +253,14 @@ A new "weak map" concept implementation. Unlike `ObjectSpace::WeakMap`, it compa
 
 ### `Proc#dup` and `#clone` call `#initialize_dup` and `#initialize_copy`
 
-* **Reason:** A fix for an old inconsistency: `Object`'s `#dup` and `#clone` methods docs
+* **Reason:** A fix for an old inconsistency: `Object`'s `#dup` and `#clone` methods docs claimed that corresponding copying constructors would be called on object cloning/duplication, yet it was not true for `Proc`.
 * **Discussion:** [Feature #19362]
 * **Documentation:** — (Adheres to the behavior described for [Object#dup](https://docs.ruby-lang.org/en/master/Object.html#method-i-dup) and [#clone](https://docs.ruby-lang.org/en/master/Kernel.html#method-i-clone))
 * **Code:**
   ```ruby
-  # The examples would work the same with
+  # The examples would work the same way with
   # #dup/#initialize_dup and #clone/#initialize_copy
+
   class TaggedProc < Proc
     attr_reader :tag
 
@@ -279,19 +301,21 @@ A new "weak map" concept implementation. Unlike `ObjectSpace::WeakMap`, it compa
 
 ### `Thread::Queue#freeze` and `SizedQueue#freeze` raise `TypeError`
 
-* **Reason:** The discussion was started with a bug report about `Queue` not respecting `#freeze` in any way (`#push` and `#pop` were still working after `#freeze` call). It was then decided that allowing to freeze a queue like any other collection (leaving it immutable) would have questionable semantics: as `Queue` is meant to be an inter-thread communication utility, freezing a queue while some thread waits for it would either leave this thread hanging, or would require `#freeze`'s functionality to extend for communication with dependent threads. Neither is a good option, so the behavior of the method was changed to communicate that queue freezing doesn't make sense.
+* **Reason:** The discussion was started with a bug report about `Queue` not respecting `#freeze` in any way (`#push` and `#pop` were still working after `#freeze` call). It was then decided that allowing to freeze a queue like any other collection (leaving it immutable) would have questionable semantics. As `Queue` is meant to be an inter-thread communication utility, freezing a queue while some thread waits for it would either leave this thread hanging, or would require `#freeze`'s functionality to extend for communication with dependent threads. Neither is a good option, so the behavior of the method was changed to communicate that queue freezing doesn't make sense.
 * **Discussion:** [Bug #17146]
 * **Documentation:** [Thread::Queue#freeze](https://docs.ruby-lang.org/en/master/Thread/Queue.html#method-i-freeze) and [Thread::SizedQueue#freeze](https://docs.ruby-lang.org/en/master/Thread/SizedQueue.html#method-i-freeze)
 
 ### `Range#reverse_each`
 
 Specialized `Range#reverse_each` method is implemented.
 
-* **Reason:** Previously, `Range` didn't have a specialized `#reverse_each` method, so calling it would invoke a generic `Enumerable#reverse_each`. The latter works by  converting the object to array, and then enumerating this array. In case of a `Range` this can be inefficient (producing large arrays) or impossible (when only upper bound of the range is defined)
+* **Reason:** Previously, `Range` didn't have a specialized `#reverse_each` method, so calling it invoked a generic `Enumerable#reverse_each`. The latter works by converting the object to array, and then enumerating this array. In case of a `Range` this can be inefficient (producing large arrays) or impossible (when only upper bound of the range is defined). It also went into infinite loop with endless ranges, trying to enumerate it all to convert into array, while the range can say beforehand that it would be impossible.
 * **Discussion:** [Feature #18515]
 * **Documentation:** [Range#reverse_each](https://docs.ruby-lang.org/en/master/Range.html#method-i-reverse_each)
 * **Code:**
   ```ruby
+  # Efficient implementation for integers:
+
   (1..2**100).reverse_each.take(3)
   # Ruby 3.2: hangs on my machine, trying to produce an array
   # Ruby 3.3: #=> [1267650600228229401496703205376, 1267650600228229401496703205375, 1267650600228229401496703205374]
@@ -301,6 +325,8 @@ Specialized `Range#reverse_each` method is implemented.
   # Ruby 3.2: can't iterate from NilClass (TypeError)
   # Ruby 3.3: #=> [5, 4, 3]
 
+  # Explicit error for endless ranges:
+
   (1...).reverse_each
   # Ruby 3.2: hangs forever, trying to produce an array
   # Ruby 3.3: `reverse_each': can't iterate from NilClass (TypeError)
@@ -397,31 +423,92 @@ Allows to trace when some exception was `rescue`'d in the code of interest.
   ```
 * **Notes:**
   * The discussion was once [started](https://bugs.ruby-lang.org/issues/15973) from the proposal to make `lambda` change "lambiness" of a passed block, but it raises multiple issues (changing the block semantics mid-program is just one of them). In general, `lambda` as a _method_ is considered legacy, inferior to the `-> { }` lambda literal syntax, exactly due to problems like this: it looks like a regular method that receives a block, and therefore should be able accept _any_ block, but in fact it is "special" method. So in 3.0, there was a warning about `lambda(&proc_instance)`, and since 3.3, the warning finally turned into an error.
-  * There is exactly one occurrence in Ruby where block semantics _changes_ mid-flight:
 
 ### Deprecate subprocess creation with method dedicated to files
 
-* **Reason:**
+* **Reason:** Methods that are dedicated for opening/reading a file by name historically supported the special syntax of the argument: if it started with pipe character `|`, the subprocess was created and could've been used to communicate with an external command. The functionality is still explained in [Ruby 3.2 docs](https://docs.ruby-lang.org/en/3.2/Kernel.html#method-i-open). It, though, created a security vulnerability: even when the program's author didn't rely on that behavior, the malicious string could've been passed by the attacker instead of an innocent filename.
 * **Discussion:** [Feature #19630]
-* **Documentation:**
 * **Affected methods:**
-  * Kernel#open
-  * URI.open
-  * IO.binread
-  * IO.foreach
-  * IO.readlines
-  * IO.read
-  * IO.write
+  * [Kernel#open](https://docs.ruby-lang.org/en/master/Kernel.html#method-i-open)
+  * [IO.binread](https://docs.ruby-lang.org/en/master/IO.html#method-c-binread)
+  * [IO.foreach](https://docs.ruby-lang.org/en/master/IO.html#method-c-foreach)
+  * [IO.readlines](https://docs.ruby-lang.org/en/master/IO.html#method-c-readlines)
+  * [IO.read](https://docs.ruby-lang.org/en/master/IO.html#method-c-read)
+  * [IO.write](https://docs.ruby-lang.org/en/master/IO.html#method-c-write)
+  * [URI.open](https://docs.ruby-lang.org/en/master/URI.html#method-c-open) ([open-uri](https://github.com/ruby/open-uri) standard library)
 * **Code:**
+  ```ruby
+  IO.read('| ls')
+  #=> contents of the current folder
+
+  Warning[:deprecated] = true # Or pass -w command-line option
+  IO.read('| ls')
+  # warning: Calling Kernel#open with a leading '|' is deprecated and will be removed in Ruby 4.0; use IO.popen instead
+  #=> contents of the current folder
+  ```
 * **Notes:**
+  * The documentation for the corresponding methods was adjusted accordingly. Compare the documentation for `Kernel#open` from [3.2](https://docs.ruby-lang.org/en/3.2/Kernel.html#method-i-open) (explains and showcases the `|` trick) and [3.3](https://docs.ruby-lang.org/en/master/Kernel.html#method-i-open) (just mentions that there is a vulnerability to command injection attack).
+  * As advised by the warning, [IO.popen](https://docs.ruby-lang.org/en/master/IO.html#method-c-popen) is a specialized method when communicating with an external process is desired functionality:
+    ```ruby
+    IO.popen('ls')
+    #=> contents of the current folder
+    ```
+  * As the impact of the change might be big, note that target version for removal is set to **4.0**. To the best of my knowledge, there are no set date for major version yet.
 
 ### New `Warning` category: `:performance`
 
-* **Reason:**
+A new warning category was introduced for a code that is correct but is known to produces a performance problems. One new such warning was added for objects with too many "shape" variations.
+
 * **Discussion:** [Feature #19538]
 * **Documentation:** [Warning#[category]](https://docs.ruby-lang.org/en/master/Warning.html#method-c-5B-5D)
-* **Code:**
+* **Code:** Here is an example of the new warning in play:
+  ```ruby
+  class C
+    def initialize(i)
+      instance_variable_set("@var_#{i}", i**2)
+    end
+  end
+
+  Warning[:performance] = true # or pass `-W:performance` command-line argument
+
+  (1..10).map { C.new(_1) }
+  # warning: Maximum shapes variations (8) reached by C, instance variables accesses will be slower.
+  ```
+  The example is artificial, but it shows the principle: when we have more than 8 instances of the _same_ class, but with different _list of instance variables_ (shape), we might have a performance problem. This means, for example, that a frequently-used class that has many methods with a memoization idiom (`@var ||= value` on the first access) would create the same problem, unless all of them would be initialized in the `initialize`, making all instances having the same shape:
+  ```ruby
+  class C
+    # 9 different getters that create an instance varaible
+    # on the first access.
+    def var1 = @var1 ||= rand
+    def var2 = @var2 ||= rand
+    def var3 = @var3 ||= rand
+    def var4 = @var4 ||= rand
+    def var5 = @var5 ||= rand
+    def var6 = @var6 ||= rand
+    def var7 = @var7 ||= rand
+    def var8 = @var8 ||= rand
+    def var9 = @var9 ||= rand
+  end
+
+  Warning[:performance] = true
+  # Invoking different getters on different instances of the same class makes
+  # them have different set of instance variables.
+  (1..9).map { C.new.send("var#{_1}") }
+  # warning: Maximum shapes variations (8) reached by C, instance variables accesses will be slower.
+
+  # But if we add this to initialize:
+  class C
+    def initialize
+      @var1, @var2, @var3, @var4, @var5, @var5, @var6, @var7, @var8, @var9 = nil
+    end
+  end
+
+  (1..9).map { C.new.send("var#{_1}") }
+  # no warning. All objects have the same list of instance vars = the same shape
+  ```
 * **Notes:**
+  * The warning category should be turned on explicitly by providing `-W:performance` CLI option or `Warning[:performance] = true` from the program.
+* **Additional reading:** [Performance impact of the memoization idiom on modern Ruby](https://railsatscale.com/2023-10-24-memoization-pattern-and-object-shapes/) by Ruby core team member Jean Boussier.
 
 ### `Fiber#kill`
 
@@ -432,6 +519,7 @@ Terminates the Fiber by sending an exception inside it.
 * **Documentation:** [Fiber#kill](https://docs.ruby-lang.org/en/master/Fiber.html#method-i-kill)
 * **Code:**
   ```ruby
+  TODO: various behaviors
 
   # Semi-realistic usage example:
   reader = Fiber.new do
@@ -459,8 +547,6 @@ Terminates the Fiber by sending an exception inside it.
 * **Code:**
 * **Notes:**
 
- added for checking if two ranges overlap. [[Feature #19839]]
-
 ### `Range#overlap?`
 
 Checks for overlapping of two ranges.
@@ -500,6 +586,7 @@ In Ruby 3.3, it will just warn to prepare for a change.
 * **Code:** In the code below, where Ruby 3.3 currently produces a warning, Ruby 3.4 would treat `it` as an anonymous block argument; where Ruby 3.3 doesn't produce a warning, Ruby 3.4 would treat `it` as a local variable name or a method call (and would look for such names available in the scope).
   ```ruby
   # The cases that are warned:
+  # -------------------------
   # warning: `it` calls without arguments will refer to the first block param in Ruby 3.4; use it() or self.it
 
   (1..3).map { it }      # inside a block without explicit parameters
@@ -508,6 +595,7 @@ In Ruby 3.3, it will just warn to prepare for a change.
   (1..3).map { it }      # even if a method with name `it` exists in the scope
 
   # The cases that are not warned:
+  # -----------------------------
 
   it                        # not inside a block
   (1..3).map { |x| it }     # inside a block with named parameters
