Type based completion using Prism and RBS #708

tompng · 2023-09-08T19:09:36Z

Implement type based autocompletion #734 using Prism and RBS

Default completor is still :regexp. To enable :type completor, write IRB.conf[:COMPLETOR] = :type to irbrc.

When Prism is available

When Prism and RBS is available

When Prism is not available, it fallbacks to RegexpCompletor

This pull request will port gem katakata_irb's completion(https://github.com/tompng/katakata_irb/) to IRB.

Morriar · 2023-09-13T13:30:54Z

Using static analysis to provide auto-completion is a surprising direction for IRB which already has access to a lot of dynamic information.

It seems like you are reimplementing features that belong more to a static type checker such as Steep or Sorbet or IDE features already covered by the Ruby LSP. I wonder if IRB is the right place for this?

tompng · 2023-09-18T16:35:21Z

It seems like you are reimplementing features that belong more to a static type checker

Yes, it's a reimplemention but has some difference from other static type checker.

Few static information and many runtime information
In IRB, most of the thing is dynamically defined. It doesn't analyze the whole project code but only analyze code in current input buffer.
It is tightly coupled with binding by design to use many runtime information.
Focuses on loose completion, not strict type check

So I think it is worth making a static + dynamic type analyzer specialized for REPL.

st0012 · 2023-09-18T16:44:14Z

So I think it is worth making a static + dynamic type analyzer specialized for REPL.

Since we're going to need a full analyzer with dependency on rbs, could it be a standalone gem instead, and IRB just has an completion adapter for it? Similar to how Rails define adapters for background job libraries like resque, sidekiq....etc. but not directly including them.

lib/irb/type_completion/completor.rb

lib/irb/type_completion/type_analyzer.rb

kddnewton

Honestly this looks great. It's super impressive how much work you've put in.

Please let me know if there's anything in the prism API that could be added to help you.

lib/irb/type_completion/completor.rb

lib/irb/context.rb

lib/irb/input-method.rb

lib/irb/type_completion/completor.rb

lib/irb/type_completion/types.rb

lib/irb/type_completion/scope.rb

test/irb/type_completion/test_scope.rb

lib/irb/type_completion/types.rb

st0012 · 2023-11-04T12:31:48Z

lib/irb/type_completion/types.rb

+        end
+      end
+
+      def self.type_from_object_recursive(object, max_level:)


Yeah let's just do 1 level for now + limit it to 50 samples maybe. In Rails console it's not rare to call users = User.some_scope.to_a, which could have thousands of objects.

test/irb/type_completion/test_type_completor.rb

…w, calculation depth is 1

…hread object

lib/irb/type_completion/completor.rb

…definition is not supported)

st0012

Amazing work 👍

hasumikin

I'd agree with shipping this feature with Ruby 3.3

Functions I would like to add gradually:

Freely change the completer within the IRB session
To display a kind of a "backtrace" of how the completion candidates are created with one simple command

ima1zumi

✨

(ruby/irb#708) * Add completor using prism and rbs * Add TypeCompletion test * Switchable completors: RegexpCompletor and TypeCompletion::Completor * Add completion info to irb_info * Complete reserved words * Fix [*] (*) {**} and prism's change of KeywordParameterNode * Fix require, frozen_string_literal * Drop prism<=0.16.0 support * Add Completor.last_completion_error for debug report * Retrieve `self` and `Module.nesting` in more safe way * Support BasicObject * Handle lvar and ivar get exception correctly * Skip ivar reference test of non-self object in ruby < 3.2 * BaseScope to RootScope, move method objects constant under Methods * Remove unused Splat struct * Drop deeply nested array/hash type calculation from actual object. Now, calculation depth is 1 * Refactor loading rbs in test, change preload_in_thread not to cache Thread object * Use new option added in prism 0.17.1 to parse code with localvars * Add Prism version check and warn when :type completor cannot be enabled * build_type_completor should skip truffleruby (because endless method definition is not supported) ruby/irb@1048c7ed7a

tompng force-pushed the yarp_rbs_completor branch 2 times, most recently from a8d1547 to 46ae44b Compare September 9, 2023 21:53

tompng force-pushed the yarp_rbs_completor branch 11 times, most recently from 14ca0bb to 92d2639 Compare October 18, 2023 18:41

tompng mentioned this pull request Oct 18, 2023

Better completion using types #734

Closed

tompng force-pushed the yarp_rbs_completor branch 2 times, most recently from 7fe5db3 to 121ae29 Compare October 25, 2023 16:45

tompng marked this pull request as ready for review October 25, 2023 17:03

st0012 reviewed Oct 25, 2023

View reviewed changes

lib/irb/type_completion/completor.rb Outdated Show resolved Hide resolved

lib/irb/type_completion/completor.rb Outdated Show resolved Hide resolved

st0012 reviewed Oct 25, 2023

View reviewed changes

lib/irb/type_completion/completor.rb Outdated Show resolved Hide resolved

lib/irb/type_completion/type_analyzer.rb Show resolved Hide resolved

tompng force-pushed the yarp_rbs_completor branch from 121ae29 to 43fbbe2 Compare October 26, 2023 11:47

tompng mentioned this pull request Oct 26, 2023

Use Ripper.sexp instead of Regexp for completion #616

Closed

tompng force-pushed the yarp_rbs_completor branch from 43fbbe2 to dadd45b Compare October 28, 2023 18:44

kddnewton approved these changes Oct 30, 2023

View reviewed changes

lib/irb/type_completion/completor.rb Outdated Show resolved Hide resolved

Add completor using prism and rbs

93ca339

tompng force-pushed the yarp_rbs_completor branch from dadd45b to b8137d4 Compare October 30, 2023 18:02

tompng added 2 commits October 31, 2023 21:58

Add TypeCompletion test

3662af2

Switchable completors: RegexpCompletor and TypeCompletion::Completor

7ea69a9

Fix [*] (*) {**} and prism's change of KeywordParameterNode

6c59ff4

st0012 reviewed Nov 4, 2023

View reviewed changes

tompng added 5 commits November 4, 2023 12:25

Fix require, frozen_string_literal

9dede7b

Drop prism<=0.16.0 support

977b13f

Add Completor.last_completion_error for debug report

2960a14

Retrieve self and Module.nesting in more safe way

71dd503

Support BasicObject

63693d3

st0012 reviewed Nov 4, 2023

View reviewed changes

tompng added 5 commits November 4, 2023 22:44

Handle lvar and ivar get exception correctly

8a0fb2e

Skip ivar reference test of non-self object in ruby < 3.2

635ad5b

BaseScope to RootScope, move method objects constant under Methods

95f5031

Remove unused Splat struct

09b8346

Drop deeply nested array/hash type calculation from actual object. No…

12efb40

…w, calculation depth is 1

tompng force-pushed the yarp_rbs_completor branch from 976487d to 12efb40 Compare November 4, 2023 16:11

Refactor loading rbs in test, change preload_in_thread not to cache T…

a41c769

…hread object

kddnewton reviewed Nov 5, 2023

View reviewed changes

lib/irb/type_completion/completor.rb Outdated Show resolved Hide resolved

tompng mentioned this pull request Nov 5, 2023

README section for IRB::TypeCompletion #746

Merged

tompng added 2 commits November 5, 2023 23:33

Use new option added in prism 0.17.1 to parse code with localvars

e9d649b

Add Prism version check and warn when :type completor cannot be enabled

124f80f

tompng force-pushed the yarp_rbs_completor branch from 386d0ca to 124f80f Compare November 5, 2023 14:39

tompng added 2 commits November 6, 2023 19:14

build_type_completor should skip truffleruby (because endless method …

0aa23f1

…definition is not supported)

Merge branch 'master' into yarp_rbs_completor

dffc27c

st0012 added the enhancement New feature or request label Nov 6, 2023

st0012 approved these changes Nov 6, 2023

View reviewed changes

st0012 changed the title ~~Type based completion using YARP and RBS~~ Type based completion using Prism and RBS Nov 6, 2023

hasumikin approved these changes Nov 8, 2023

View reviewed changes

ima1zumi approved these changes Nov 8, 2023

View reviewed changes

tompng merged commit 1048c7e into ruby:master Nov 8, 2023
24 checks passed

tompng deleted the yarp_rbs_completor branch November 8, 2023 02:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Type based completion using Prism and RBS #708

Type based completion using Prism and RBS #708

tompng commented Sep 8, 2023 •

edited

Loading

Morriar commented Sep 13, 2023

tompng commented Sep 18, 2023

st0012 commented Sep 18, 2023

kddnewton left a comment

st0012 Nov 4, 2023

st0012 left a comment

hasumikin left a comment

ima1zumi left a comment

Type based completion using Prism and RBS #708

Type based completion using Prism and RBS #708

Conversation

tompng commented Sep 8, 2023 • edited Loading

Morriar commented Sep 13, 2023

tompng commented Sep 18, 2023

st0012 commented Sep 18, 2023

kddnewton left a comment

Choose a reason for hiding this comment

st0012 Nov 4, 2023

Choose a reason for hiding this comment

st0012 left a comment

Choose a reason for hiding this comment

hasumikin left a comment

Choose a reason for hiding this comment

ima1zumi left a comment

Choose a reason for hiding this comment

tompng commented Sep 8, 2023 •

edited

Loading