Support constants defined within arrays or hashes #63

owst · 2017-05-11T23:14:32Z

Our code base includes examples such as:

STATUSES = [
  OPEN = 'open',
  ...
]

DISPLAY_MAPPING = {
  CANCELLED = 'cancelled' => 'Cancelled by user',
  ...
}

currently, OPEN and CANCELLED are not picked up by ripper-tags.

This PR aims to support such nested definitions.

~~I'm not 100% certain of the implications of making on_args now process its args when it was ignoring them before, but none of the existing tests fail.~~ Please let me know what you think!

mislav

Thanks for your contribution and sorry for the late review!

mislav · 2017-09-04T16:43:35Z

test/test_ripper_tags.rb

+
+      D = {
+        E = 3 => F = 4,
+        x: G = 5


Do we need to support F and G here, i.e. parsing assignment on the right side of the => hash syntax? Does anyone define constants that way?

I don't think so, but it falls out "for free" from the use of on_assoc_new (and it would actually be harder to support nested hashes if we tried to only handle definitions in keys)

but it falls out "for free"

Even though we get is basically for "free", that is not reason enough to test for and support a feature in the library. If we don't need a specific feature, let's not test for it. Keep in mind that every test that we add here, I have to maintain for months/years over future releases of Ruby and ripper-tags itself. Let's keep the scope of the feature focused on just what was the initial goal.

mislav · 2017-09-04T16:43:35Z

lib/ripper-tags/parser.rb

@@ -320,6 +324,8 @@ def on_assign(name, rhs, line, *junk)
        namespace = namespace + parts
      end

+      process(rhs)


Why was this process added?

If we don't add this process, the Visitor won't discover potential tags on the RHS of a constant assignment. If I remove this call my added test case fails with:

Failure: test_nested_constant_definitions(TagRipperTest) /Users/owenstephens/code/ripper-tags/test/test_ripper_tags.rb:92:in `test_nested_constant_definitions' 89: } 90: EOC 91: => 92: assert_equal ('A'..'G').to_a, tags.map { |t| t[:name] }.sort 93: tags.each do |t| 94: assert_equal t[:name], t[:full_name] 95: end <["A", "B", "C", "D", "E", "F", "G"]> expected but was <["A", "D"]>

as the visitor is no longer looking "inside" the expressions in the (top-level) assignments

Thanks for the explanation. Are we interested in processing RHS at all?

I'm not sure I understand the question.

We are interested in processing the RHS since we wouldn't otherwise discover the constant definitions on the RHS of the top-level assignment, which was the aim of this PR. Indeed, if we don't process the RHS, in my original example we only get tags for STATUSES and DISPLAY_MAPPING; with the process(rhs) call we also get tags for OPEN and CANCELLED as desired.

mislav · 2017-09-04T16:43:35Z

lib/ripper-tags/parser.rb

@@ -382,12 +388,15 @@ def on_class_eval(name, body)
      @namespace.pop
    end

+    def on_args(*args)
+      process(args)


I'm worried about the potential negative implications of this. Have you tried running ripper-tags from this branch against a large Ruby codebase such as Rails?

Yes, great idea. After my subsequent changes, I've confirmed that the tags files have no changes between master and this branch.

To avoid a crash when running against the rails codebase

owst · 2017-09-05T00:06:32Z

lib/ripper-tags/parser.rb

@@ -65,6 +65,9 @@ def on_const_path_ref(a, b)
  def on_binary(*args)
  end

+  def on_unary(*args)
+  end


This was needed to avoid trying to process a unary method (-@) in a hash in the rails codebase (triggered since we now expose the keys/values of a hash assignment in the sexp)

owst · 2017-09-05T00:07:25Z

lib/ripper-tags/parser.rb

+    if args && args[0] == :args
+      args[1..-1]
+    end
+  end


This is the better way of handling arrays, rather than the on_args processing I first suggested

owst · 2017-09-05T00:09:25Z

Thanks for the review @mislav and good suggestion of testing against the rails codebase! Let me know what you think of the changes and I can squash the commits if you're happy.

mislav

Thanks for working on this and on your patience.

mislav · 2017-09-05T03:03:38Z

test/test_ripper_tags.rb

+
+      L = {
+        'some key' => {
+          M = 2 => 3


This test is getting too evolved. It represents features that I'm not keen on supporting, even if it's technically trivial at the moment.

Could we simply make a test that verifies only features that we're interested in here? That would be your initial use case:

STATUSES = [ OPEN = 'open', ] DISPLAY_MAPPING = { CANCELLED = 'cancelled' => 'Cancelled by user', }

mislav · 2017-09-05T03:03:38Z

lib/ripper-tags/parser.rb

@@ -320,6 +324,8 @@ def on_assign(name, rhs, line, *junk)
        namespace = namespace + parts
      end

+      process(rhs)


Thanks for the explanation. Are we interested in processing RHS at all?

mislav · 2017-09-05T03:03:38Z

test/test_ripper_tags.rb

+
+      D = {
+        E = 3 => F = 4,
+        x: G = 5


but it falls out "for free"

Even though we get is basically for "free", that is not reason enough to test for and support a feature in the library. If we don't need a specific feature, let's not test for it. Keep in mind that every test that we add here, I have to maintain for months/years over future releases of Ruby and ripper-tags itself. Let's keep the scope of the feature focused on just what was the initial goal.

owst · 2017-09-05T09:29:21Z

I'm slightly stumped by the build failures, it seems that we're getting an additional assoc node in the sexp, but I can't reproduce it locally on Linux or Mac, on a variety of Ruby versions.

In travis the sexp of my test is:

[[:assign, "STATUSES", [[:assign, "OPEN", nil, 2]], 1], [:assign, "DISPLAY_MAPPING", [[:assoc, [:assign, "CANCELLED", nil, 6], nil]], 5]]

but locally it is

[[:assign, "STATUSES", [[:assign, "OPEN", nil, 2]], 1], [:assign, "DISPLAY_MAPPING", [[:assign, "CANCELLED", nil, 6]], 5]]

(There's an additional [:assoc ... ] wrapping the CANCELLED node.)

I could probably add a Visitor#on_assoc method that does the right thing, but I'm hesitant to do so without being to reproduce/explain the issue locally.

@mislav any thoughts or advice? :-)

mislav · 2017-09-05T10:07:46Z

I could probably add a Visitor#on_assoc method that does the right thing, but I'm hesitant to do so without being to reproduce/explain the issue locally.

I can't reproduce locally either. I'm stumped. I guess we just have to work around this.

owst · 2017-09-05T14:20:29Z

Ok, I understand what's happened - the Travis build runs off the git fetch origin +refs/pull/63/merge ref (which is the implicit merge commit of my branch into master), which therefore has picked up this commit which was merged to master after my branch.

That commit adds the on_assoc_new method which adds the troubling :assoc sexp entry. I'll update my branch to handle it.

owst · 2017-09-11T19:27:52Z

Hi @mislav is there anything else for me to do in order for this to be merged, or can I squash it down to 1 commit ready for a merge?

mislav · 2017-09-18T12:24:27Z

Just got back from vacation. This PR looks great; thanks for your patience! There's nothing I need you to do right now.

owst · 2017-09-18T14:13:42Z

Great, thanks @mislav - shall I squash this into one commit? 16 (including debug) commits looks a bit messy for a +40,-1 PR, but I don't know the project's stance on rewriting... I suppose you could also squash it when you merge the PR.

mislav · 2017-09-18T14:23:18Z

Yes it's easy for me to squash when I merge.

owst · 2017-09-18T14:41:44Z

Great, thanks! 👍

Support constants defined within arrays or hashes

Loading status checks…

d81fe69

mislav reviewed Sep 4, 2017

View changes

owst added 3 commits Sep 4, 2017

Better handle nested-array constants

fbee381

Do not process unary nodes

a22539c

To avoid a crash when running against the rails codebase

Expand test and properly handle nested hashes

Loading status checks…

2d5de6c

owst reviewed Sep 5, 2017

View changes

lib/ripper-tags/parser.rb Outdated

if args && args[0] == :args

args[1..-1]

end

end

This comment has been minimized.

Sign in to view

owst Sep 5, 2017
Author Contributor

This is the better way of handling arrays, rather than the on_args processing I first suggested

Undo on_args method change

Loading status checks…

45feb18

mislav requested changes Sep 5, 2017

View changes

owst added 7 commits Sep 5, 2017

Simplify test

Loading status checks…

4868717

Handle hashes directly not via assoc_new

Loading status checks…

4600b0a

remove unnecessary on_unary method now we don't inspect hash keys

Loading status checks…

23aaa21

Add debug method to debug travis-only test failure

Loading status checks…

296850f

Add more debug

Loading status checks…

71f181d

more debug

Loading status checks…

13ab0d4

Remove debug method to show problem

Loading status checks…

3fb2ace

owst force-pushed the owst:constants_inside_arrays_and_hashes branch from 89ef46f to 8b254a7 Sep 5, 2017

different debug

Loading status checks…

181cf52

owst force-pushed the owst:constants_inside_arrays_and_hashes branch from 8b254a7 to 181cf52 Sep 5, 2017

owst added 2 commits Sep 5, 2017

fix heredoc

Loading status checks…

fdead33

Merge branch 'master' into constants_inside_arrays_and_hashes

Loading status checks…

8962617

Undo debug and handle :assoc nodes

Loading status checks…

57baeed

mislav approved these changes Sep 18, 2017

View changes

mislav closed this in c9df1b7 Sep 27, 2017

owst mentioned this pull request Feb 2, 2018

Process inside calls to handle frozen (nested) constants #79

Open

tmm1 / ripper-tags

Support constants defined within arrays or hashes #63

Support constants defined within arrays or hashes #63

owst commented May 11, 2017 •

edited

mislav left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

owst commented Sep 5, 2017

mislav left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

owst commented Sep 5, 2017 •

edited

mislav commented Sep 5, 2017

owst commented Sep 5, 2017

owst commented Sep 11, 2017

mislav commented Sep 18, 2017

owst commented Sep 18, 2017

mislav commented Sep 18, 2017

owst commented Sep 18, 2017

tmm1 / ripper-tags

Support constants defined within arrays or hashes #63

Support constants defined within arrays or hashes #63

Conversation

owst commented May 11, 2017 • edited

mislav left a comment

This comment has been minimized.

mislav Sep 4, 2017 Collaborator

This comment has been minimized.

owst Sep 5, 2017 Author Contributor

This comment has been minimized.

mislav Sep 5, 2017 Collaborator

This comment has been minimized.

mislav Sep 4, 2017 Collaborator

This comment has been minimized.

owst Sep 5, 2017 Author Contributor

This comment has been minimized.

mislav Sep 5, 2017 Collaborator

This comment has been minimized.

owst Sep 5, 2017 Author Contributor

This comment has been minimized.

mislav Sep 4, 2017 Collaborator

This comment has been minimized.

owst Sep 5, 2017 Author Contributor

This comment has been minimized.

owst Sep 5, 2017 • edited Author Contributor

This comment has been minimized.

owst Sep 5, 2017 Author Contributor

owst commented Sep 5, 2017

mislav left a comment

This comment has been minimized.

mislav Sep 5, 2017 Collaborator

This comment has been minimized.

mislav Sep 5, 2017 Collaborator

This comment has been minimized.

mislav Sep 5, 2017 Collaborator

owst commented Sep 5, 2017 • edited

mislav commented Sep 5, 2017

owst commented Sep 5, 2017

owst commented Sep 11, 2017

mislav commented Sep 18, 2017

owst commented Sep 18, 2017

mislav commented Sep 18, 2017

owst commented Sep 18, 2017

owst commented May 11, 2017 •

edited

mislav Sep 4, 2017
Collaborator

owst Sep 5, 2017
Author Contributor

mislav Sep 5, 2017
Collaborator

mislav Sep 4, 2017
Collaborator

owst Sep 5, 2017
Author Contributor

mislav Sep 5, 2017
Collaborator

owst Sep 5, 2017
Author Contributor

mislav Sep 4, 2017
Collaborator

owst Sep 5, 2017
Author Contributor

owst Sep 5, 2017 •

edited

Author Contributor

owst Sep 5, 2017
Author Contributor

mislav Sep 5, 2017
Collaborator

mislav Sep 5, 2017
Collaborator

mislav Sep 5, 2017
Collaborator

owst commented Sep 5, 2017 •

edited