Support constants defined within arrays or hashes #63

owst · 2017-05-11T23:14:32Z

Our code base includes examples such as:

STATUSES = [
  OPEN = 'open',
  ...
]

DISPLAY_MAPPING = {
  CANCELLED = 'cancelled' => 'Cancelled by user',
  ...
}

currently, OPEN and CANCELLED are not picked up by ripper-tags.

This PR aims to support such nested definitions.

~~I'm not 100% certain of the implications of making on_args now process its args when it was ignoring them before, but none of the existing tests fail.~~ Please let me know what you think!

mislav

Thanks for your contribution and sorry for the late review!

mislav · 2017-09-04T16:41:44Z

test/test_ripper_tags.rb

+
+      D = {
+        E = 3 => F = 4,
+        x: G = 5


Do we need to support F and G here, i.e. parsing assignment on the right side of the => hash syntax? Does anyone define constants that way?

I don't think so, but it falls out "for free" from the use of on_assoc_new (and it would actually be harder to support nested hashes if we tried to only handle definitions in keys)

but it falls out "for free"

Even though we get is basically for "free", that is not reason enough to test for and support a feature in the library. If we don't need a specific feature, let's not test for it. Keep in mind that every test that we add here, I have to maintain for months/years over future releases of Ruby and ripper-tags itself. Let's keep the scope of the feature focused on just what was the initial goal.

mislav · 2017-09-04T16:42:29Z

lib/ripper-tags/parser.rb

@@ -320,6 +324,8 @@ def on_assign(name, rhs, line, *junk)
        namespace = namespace + parts
      end

+      process(rhs)


Why was this process added?

If we don't add this process, the Visitor won't discover potential tags on the RHS of a constant assignment. If I remove this call my added test case fails with:

Failure: test_nested_constant_definitions(TagRipperTest) /Users/owenstephens/code/ripper-tags/test/test_ripper_tags.rb:92:in `test_nested_constant_definitions' 89: } 90: EOC 91: => 92: assert_equal ('A'..'G').to_a, tags.map { |t| t[:name] }.sort 93: tags.each do |t| 94: assert_equal t[:name], t[:full_name] 95: end <["A", "B", "C", "D", "E", "F", "G"]> expected but was <["A", "D"]>

as the visitor is no longer looking "inside" the expressions in the (top-level) assignments

Thanks for the explanation. Are we interested in processing RHS at all?

I'm not sure I understand the question.

We are interested in processing the RHS since we wouldn't otherwise discover the constant definitions on the RHS of the top-level assignment, which was the aim of this PR. Indeed, if we don't process the RHS, in my original example we only get tags for STATUSES and DISPLAY_MAPPING; with the process(rhs) call we also get tags for OPEN and CANCELLED as desired.

mislav · 2017-09-04T16:43:17Z

lib/ripper-tags/parser.rb

@@ -382,12 +388,15 @@ def on_class_eval(name, body)
      @namespace.pop
    end

+    def on_args(*args)
+      process(args)


I'm worried about the potential negative implications of this. Have you tried running ripper-tags from this branch against a large Ruby codebase such as Rails?

Yes, great idea. After my subsequent changes, I've confirmed that the tags files have no changes between master and this branch.

To avoid a crash when running against the rails codebase

owst · 2017-09-05T00:06:32Z

lib/ripper-tags/parser.rb

@@ -65,6 +65,9 @@ def on_const_path_ref(a, b)
  def on_binary(*args)
  end

+  def on_unary(*args)
+  end


This was needed to avoid trying to process a unary method (-@) in a hash in the rails codebase (triggered since we now expose the keys/values of a hash assignment in the sexp)

owst · 2017-09-05T00:07:25Z

lib/ripper-tags/parser.rb

+    if args && args[0] == :args
+      args[1..-1]
+    end
+  end


This is the better way of handling arrays, rather than the on_args processing I first suggested

owst · 2017-09-05T00:09:25Z

Thanks for the review @mislav and good suggestion of testing against the rails codebase! Let me know what you think of the changes and I can squash the commits if you're happy.

mislav

Thanks for working on this and on your patience.

mislav · 2017-09-05T02:59:00Z

test/test_ripper_tags.rb

+
+      L = {
+        'some key' => {
+          M = 2 => 3


This test is getting too evolved. It represents features that I'm not keen on supporting, even if it's technically trivial at the moment.

Could we simply make a test that verifies only features that we're interested in here? That would be your initial use case:

STATUSES = [ OPEN = 'open', ] DISPLAY_MAPPING = { CANCELLED = 'cancelled' => 'Cancelled by user', }

mislav · 2017-09-05T03:00:19Z

lib/ripper-tags/parser.rb

@@ -320,6 +324,8 @@ def on_assign(name, rhs, line, *junk)
        namespace = namespace + parts
      end

+      process(rhs)


Thanks for the explanation. Are we interested in processing RHS at all?

mislav · 2017-09-05T03:03:24Z

test/test_ripper_tags.rb

+
+      D = {
+        E = 3 => F = 4,
+        x: G = 5


but it falls out "for free"

Even though we get is basically for "free", that is not reason enough to test for and support a feature in the library. If we don't need a specific feature, let's not test for it. Keep in mind that every test that we add here, I have to maintain for months/years over future releases of Ruby and ripper-tags itself. Let's keep the scope of the feature focused on just what was the initial goal.

owst · 2017-09-05T09:29:21Z

I'm slightly stumped by the build failures, it seems that we're getting an additional assoc node in the sexp, but I can't reproduce it locally on Linux or Mac, on a variety of Ruby versions.

In travis the sexp of my test is:

[[:assign, "STATUSES", [[:assign, "OPEN", nil, 2]], 1], [:assign, "DISPLAY_MAPPING", [[:assoc, [:assign, "CANCELLED", nil, 6], nil]], 5]]

but locally it is

[[:assign, "STATUSES", [[:assign, "OPEN", nil, 2]], 1], [:assign, "DISPLAY_MAPPING", [[:assign, "CANCELLED", nil, 6]], 5]]

(There's an additional [:assoc ... ] wrapping the CANCELLED node.)

I could probably add a Visitor#on_assoc method that does the right thing, but I'm hesitant to do so without being to reproduce/explain the issue locally.

@mislav any thoughts or advice? :-)

mislav · 2017-09-05T10:07:46Z

I could probably add a Visitor#on_assoc method that does the right thing, but I'm hesitant to do so without being to reproduce/explain the issue locally.

I can't reproduce locally either. I'm stumped. I guess we just have to work around this.

owst · 2017-09-05T14:20:29Z

Ok, I understand what's happened - the Travis build runs off the git fetch origin +refs/pull/63/merge ref (which is the implicit merge commit of my branch into master), which therefore has picked up this commit which was merged to master after my branch.

That commit adds the on_assoc_new method which adds the troubling :assoc sexp entry. I'll update my branch to handle it.

owst · 2017-09-11T19:27:52Z

Hi @mislav is there anything else for me to do in order for this to be merged, or can I squash it down to 1 commit ready for a merge?

mislav · 2017-09-18T12:24:27Z

Just got back from vacation. This PR looks great; thanks for your patience! There's nothing I need you to do right now.

owst · 2017-09-18T14:13:42Z

Great, thanks @mislav - shall I squash this into one commit? 16 (including debug) commits looks a bit messy for a +40,-1 PR, but I don't know the project's stance on rewriting... I suppose you could also squash it when you merge the PR.

mislav · 2017-09-18T14:23:18Z

Yes it's easy for me to squash when I merge.

owst · 2017-09-18T14:41:44Z

Great, thanks! 👍

Support constants defined within arrays or hashes

d81fe69

mislav reviewed Sep 4, 2017

View reviewed changes

owst added 3 commits September 5, 2017 00:38

Better handle nested-array constants

fbee381

Do not process unary nodes

a22539c

To avoid a crash when running against the rails codebase

Expand test and properly handle nested hashes

2d5de6c

owst commented Sep 5, 2017

View reviewed changes

Undo on_args method change

45feb18

mislav requested changes Sep 5, 2017

View reviewed changes

owst and others added 7 commits September 5, 2017 08:27

Simplify test

4868717

Handle hashes directly not via assoc_new

4600b0a

remove unnecessary on_unary method now we don't inspect hash keys

23aaa21

Add debug method to debug travis-only test failure

296850f

Add more debug

71f181d

more debug

13ab0d4

Remove debug method to show problem

3fb2ace

owst force-pushed the constants_inside_arrays_and_hashes branch from 89ef46f to 8b254a7 Compare September 5, 2017 13:26

different debug

181cf52

owst force-pushed the constants_inside_arrays_and_hashes branch from 8b254a7 to 181cf52 Compare September 5, 2017 13:28

Owen Stephens added 2 commits September 5, 2017 14:31

fix heredoc

fdead33

Merge branch 'master' into constants_inside_arrays_and_hashes

8962617

Undo debug and handle :assoc nodes

57baeed

mislav approved these changes Sep 18, 2017

View reviewed changes

mislav closed this in c9df1b7 Sep 27, 2017

owst mentioned this pull request Feb 2, 2018

Process inside calls to handle frozen (nested) constants #79

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support constants defined within arrays or hashes #63

Support constants defined within arrays or hashes #63

owst commented May 11, 2017 •

edited

Loading

mislav left a comment

mislav Sep 4, 2017

owst Sep 5, 2017

mislav Sep 5, 2017

mislav Sep 4, 2017

owst Sep 5, 2017

mislav Sep 5, 2017

owst Sep 5, 2017

mislav Sep 4, 2017

owst Sep 5, 2017

owst Sep 5, 2017 •

edited

Loading

owst Sep 5, 2017

owst commented Sep 5, 2017

mislav left a comment

mislav Sep 5, 2017

mislav Sep 5, 2017

mislav Sep 5, 2017

owst commented Sep 5, 2017 •

edited

Loading

mislav commented Sep 5, 2017

owst commented Sep 5, 2017

owst commented Sep 11, 2017

mislav commented Sep 18, 2017

owst commented Sep 18, 2017

mislav commented Sep 18, 2017

owst commented Sep 18, 2017

Support constants defined within arrays or hashes #63

Support constants defined within arrays or hashes #63

Conversation

owst commented May 11, 2017 • edited Loading

mislav left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

owst Sep 5, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

owst commented Sep 5, 2017

mislav left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

owst commented Sep 5, 2017 • edited Loading

mislav commented Sep 5, 2017

owst commented Sep 5, 2017

owst commented Sep 11, 2017

mislav commented Sep 18, 2017

owst commented Sep 18, 2017

mislav commented Sep 18, 2017

owst commented Sep 18, 2017

owst commented May 11, 2017 •

edited

Loading

owst Sep 5, 2017 •

edited

Loading

owst commented Sep 5, 2017 •

edited

Loading