Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Adding deep versions of stringify_keys and symbolize_keys (plain and bang) for nested hashes #6060

Merged
merged 1 commit into from May 23, 2012

Conversation

Projects
None yet
10 participants
Contributor

lucashungaro commented Apr 29, 2012

I have to carry around some snippets 'cause I always use these methods (specially deep_symbolize_keys) when I use YAML files with custom app configuration. Some people I know also have their own versions, so I thought adding them to ActiveSupport was a good idea.

@josevalim josevalim and 1 other commented on an outdated diff Apr 29, 2012

activesupport/lib/active_support/core_ext/hash/keys.rb
@@ -51,4 +51,56 @@ def assert_valid_keys(*valid_keys)
raise ArgumentError.new("Unknown key: #{k}") unless valid_keys.include?(k)
end
end
+
+ # Return a new hash with all keys converted to strings.
+ # This includes the keys from the root hash and from all
+ # nested hashes.
+ def deep_stringify_keys
+ result = {}
+ self.each do |key, value|
+ if value.is_a?(Hash)
+ result[(key.to_s)] = value.deep_stringify_keys
@josevalim

josevalim Apr 29, 2012

Contributor

Can we remove the parenthesis here in the hash access pls?

@lucashungaro

lucashungaro Apr 29, 2012

Contributor

Woops! My bad. Doing it.

@josevalim josevalim and 2 others commented on an outdated diff Apr 29, 2012

activesupport/lib/active_support/core_ext/hash/keys.rb
+ result[(key.to_s)] = value.deep_stringify_keys
+ else
+ result[(key.to_s)] = self[key]
+ end
+ end
+ result
+ end
+
+ # Destructively convert all keys to strings.
+ # This includes the keys from the root hash and from all
+ # nested hashes.
+ def deep_stringify_keys!
+ keys.each do |key|
+ self[key.to_s] = delete(key)
+ end
+ values.each{ |h| h.deep_stringify_keys! if h.is_a?(Hash) }
@josevalim

josevalim Apr 29, 2012

Contributor

Can we loop keys and values once?

@lucashungaro

lucashungaro Apr 29, 2012

Contributor

I think so. I did them separately to ease the understanding of what's going on, but it would be easy to do in one step.

@lucashungaro

lucashungaro Apr 29, 2012

Contributor

Ok, after further review, I can't change the own hash inside an each block (it throws an exception). I could copy the hash and do the conversions, then change self to reference the new hash or just keep iterating over keys and values separately. What do you think?

@gazay

gazay Apr 29, 2012

Contributor

Yes, I think it can be done similar to this #5939

@lucashungaro

lucashungaro Apr 29, 2012

Contributor

Using each_with_object?

@gazay

gazay Apr 29, 2012

Contributor

Yes, sorry, I was from phone, can't write more

Something like this:

each_with_object({}) do |(key, value), hash|
  hash[key.to_s] = value.is_a?(Hash) ? value.deep_stringify_keys! : value
end

It's the same as you described

@lucashungaro

lucashungaro Apr 29, 2012

Contributor

This would make a copy, right? If a use each_with_object(self) I'll have the same problem (modifying the object while iterating over it). I think I'm missing something...

@gazay

gazay Apr 29, 2012

Contributor

Oh, I was wrong, sorry. My example will works fine with non-destructive version of this method https://github.com/rails/rails/pull/6060/files#L0R58. In ruby you can't change value of self, so you need to do something like this:

def deep_stringify_keys!
  stringified = deep_stringify_keys
  self.clear.merge! stringified
end
@lucashungaro

lucashungaro Apr 29, 2012

Contributor

Yeah, that was what I thought at first. I'll think about a way to benchmark this and see which version is faster. Thanks!

Also, I'm repeating the logic in every method instead of reusing it (like in your example) because sometimes people will undef some of these methods. I found one case while writing this code, so I decided to just repeat the key conversion code in every method. Maybe extracting that part to a private method and reusing it would be fine.

@gazay

gazay Apr 29, 2012

Contributor

By my benchmark test separate iteration is faster than my example about 1.5 times :)

@lucashungaro

lucashungaro Apr 30, 2012

Contributor

I've found similar results. The code that iterates over keys and then values takes ~6.4s to do deep_stringify_keys! 1 million times. The code that loops over keys and values at once copying the array and "rebuilding" self after takes ~8.9s.

So, I think I'll leave the code as it is now. Anything else I can improve upon?

@gazay

gazay May 1, 2012

Contributor

I found better solution and only with one iteration:

def deep_stringify_keys!
  keys.each do |key|
    val = delete(key)
    self[key.to_s] = val.is_a?(Hash) ? val.deep_stringify_keys! : val
  end
  self
end

It faster then separate iteration

@lucashungaro

lucashungaro May 1, 2012

Contributor

Nice @gazay! It seems Ruby doesn't mind if you change the hash while iterating over the keys only. Good to know. :)

I made a commit using this code for deep_stringify_keys! and deep_symbolize_keys!.

Owner

rafaelfranca commented Apr 29, 2012

Related with #5587

Owner

rafaelfranca commented Apr 29, 2012

Related with #5724

@gazay gazay and 2 others commented on an outdated diff Apr 29, 2012

activesupport/lib/active_support/core_ext/hash/keys.rb
@@ -51,4 +51,56 @@ def assert_valid_keys(*valid_keys)
raise ArgumentError.new("Unknown key: #{k}") unless valid_keys.include?(k)
end
end
+
+ # Return a new hash with all keys converted to strings.
+ # This includes the keys from the root hash and from all
+ # nested hashes.
+ def deep_stringify_keys
+ result = {}
+ self.each do |key, value|
@gazay

gazay Apr 29, 2012

Contributor

Self is unnecessary here

@lucashungaro

lucashungaro Apr 29, 2012

Contributor

Sure. It's more a style thing. Should I remove it?

@rafaelfranca

rafaelfranca Apr 29, 2012

Owner

Yes please. We had a pull requesting from @gazay merged yesterday only changing these style things on Active Support

@lucashungaro

lucashungaro Apr 29, 2012

Contributor

Ok, no problem. :)

@gazay gazay and 1 other commented on an outdated diff Apr 29, 2012

activesupport/lib/active_support/core_ext/hash/keys.rb
@@ -51,4 +51,48 @@ def assert_valid_keys(*valid_keys)
raise ArgumentError.new("Unknown key: #{k}") unless valid_keys.include?(k)
end
end
+
+ # Return a new hash with all keys converted to strings.
+ # This includes the keys from the root hash and from all
+ # nested hashes.
+ def deep_stringify_keys
+ result = {}
+ each do |key, value|
+ result[key.to_s] = value.is_a?(Hash) ? value.deep_stringify_keys : self[key]
@gazay

gazay Apr 29, 2012

Contributor

Instead of calling [] on self will be better to pass just value in the false part of ternary operator

@lucashungaro

lucashungaro Apr 29, 2012

Contributor

Yeah, way better. Missed that somehow. Thank you.

Najaf commented May 1, 2012

Quick heads up, may or not be relevant but i18n appears to implement deep_symbolize_keys in their core_ext/hash.rb too.

Also, minor annoyance (to which I'd be willing to put together a pull-request for). It would be nice if all hashes in arrays were deep symbolized too.

Contributor

lucashungaro commented May 1, 2012

Najaf, I've saw i18n's code and it only defines the method if it isn't already defined. I tried to implement it in a way that doesn't break i18n.

My initial implementation took into account hashes inside arrays but then I thought that could be a bad surprise for some people and ignored that possibility. Should we have it?

Najaf commented May 1, 2012

+1 from me, I found it to be a bad surprise when it didn't symbolize hashes inside arrays.

Contributor

masterkain commented May 2, 2012

+1 with backport

Hey @lucashungaro, thanks for your pull request. Perhaps you could add an entry to the changelog talking about the new methods, and some quick docs to AS Core Extensions guide before it goes in? Thanks!

Contributor

lucashungaro commented May 15, 2012

Will do! :)

Contributor

lucashungaro commented May 16, 2012

Ok, done. I added simple notes to the Guides mentioning the existence of the deep versions. Do you guys think this is enough or should I add "sections" dedicated to those?

Also, should I change the code to also convert keys from hashes inside arrays?

@carlosantoniodasilva carlosantoniodasilva and 1 other commented on an outdated diff May 16, 2012

activesupport/CHANGELOG.md
@@ -1,5 +1,7 @@
## Rails 4.0.0 (unreleased) ##
+* Adds `Hash#deep_symbolize_keys` and `Hash#deep_symbolize_keys!` to convert all keys from a +Hash+ instance into symbols *Lucas Húngaro*
@carlosantoniodasilva

carlosantoniodasilva May 16, 2012

Owner

I think you can comment about both deep_symbolize_keys and deep_stringify_keys :)

@lucashungaro

lucashungaro May 16, 2012

Contributor

Gee, I need to rest. :P

@lucashungaro not sure it's necessary to add a specific section for them... is it, @vijaydev? Thanks.

Member

vijaydev commented May 16, 2012

@lucashungaro @carlosantoniodasilva No need for a specific section. I think what's in this PR is good enough. Add an example may be?

Contributor

lucashungaro commented May 16, 2012

Ok, one more commit adding the changelog note and simple examples to the Guides. :)

Member

vijaydev commented May 16, 2012

Squash the commits into one please.

Contributor

lucashungaro commented May 16, 2012

Done.

@lucashungaro I believe it's fine to merge, but somehow it does not apply cleanly anymore (probably just the changelog conflicting). Can you rebase from current master? Thanks.

Contributor

lucashungaro commented May 23, 2012

Thanks! But I guess I'll have to bother you again :P

Apparently this merge c1487f6 has messed up with your pull request, and it can't be merged again :D

Owner

rafaelfranca commented May 23, 2012

ooops! Sorry @lucashungaro.

Contributor

lucashungaro commented May 23, 2012

@carlosantoniodasilva @rafaelfranca Shit happens ;). Did the rebase again.

@lucashungaro hehe, all the time :D.. Merging, thank you.

@carlosantoniodasilva carlosantoniodasilva added a commit that referenced this pull request May 23, 2012

@carlosantoniodasilva carlosantoniodasilva Merge pull request #6060 from lucashungaro/master
Adding deep versions of stringify_keys and symbolize_keys (plain and bang) for nested hashes
541429f

@carlosantoniodasilva carlosantoniodasilva merged commit 541429f into rails:master May 23, 2012

@rilian rilian commented on the diff Aug 30, 2012

activesupport/lib/active_support/core_ext/hash/keys.rb
+ # nested hashes.
+ def deep_stringify_keys!
+ keys.each do |key|
+ val = delete(key)
+ self[key.to_s] = val.is_a?(Hash) ? val.deep_stringify_keys! : val
+ end
+ self
+ end
+
+ # Destructively convert all keys to symbols, as long as they respond
+ # to +to_sym+. This includes the keys from the root hash and from all
+ # nested hashes.
+ def deep_symbolize_keys!
+ keys.each do |key|
+ val = delete(key)
+ self[(key.to_sym rescue key)] = val.is_a?(Hash) ? val.deep_stringify_keys! : val
@rilian

rilian Aug 30, 2012

why stringify here ?

@lucashungaro

lucashungaro Aug 30, 2012

Contributor

Silly mistake. It's correct on master, which uses a much improved code through transform_keys :)

@rilian

rilian Aug 31, 2012

huh, thanks :)

should I backport to activesupport the deep_transform_keys method that goes each Array value and transforms it for such hashes

{ :a => { :b => { :c => 3, :d => [ { :e => 4 }, { :f => 5 } ] } } }

useful for validation of hashes of model with has_many related models parsed from JSON

@lucashungaro

lucashungaro Aug 31, 2012

Contributor

I think that would be nice. :)

rmsy commented Oct 21, 2013

Sorry for the bump, but I just want to profess my sincere thanks for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment