Skip to content

Add YARD macro support for DSL methods#1187

Open
lekemula wants to merge 8 commits into
castwide:masterfrom
lekemula:lm-yard-macros
Open

Add YARD macro support for DSL methods#1187
lekemula wants to merge 8 commits into
castwide:masterfrom
lekemula:lm-yard-macros

Conversation

@lekemula
Copy link
Copy Markdown
Contributor

@lekemula lekemula commented May 13, 2026

Note

AI usage disclaimer. The original implementation on my fork was hand-written by me starting in November 2024 (~18 months ago, branch lm-named-macros). The diff in this PR is a port of those 11 commits onto current master, which I produced with AI assistance because the codebase had drifted significantly in the meantime — most notably the rewrite of the YardMap class into DocMap/GemPins (#781). The original work is unchanged in intent; only the integration points were rewritten to fit the new architecture. Original 11-commit diff for reference: 524c94e9...389def6e.

Note

PR Size is not as big as it looks. Most of the lines come from the gem-with-yard-macros fixture gem for testing purpose and extracting the yard mapping directives onto their own modules/classes.

Summary

Adds support for YARD @!macro directives attached to DSL-style class methods, expanding to method/attribute/parse pins at the call sites. Works for macros declared in workspace source and in third-party gems.

Solution

The pipeline has three stages:

  1. Parser stage — collect DSL call sites.
    Solargraph::Parser::ParserGem::NodeProcessors::SendNode now detects "DSL-shaped" calls (no receiver, in instance/class scope, at least one argument) and emits a Pin::Ephemeral::ClassMethodSend. The pin captures the call's name, code, comments, and arguments — arguments are pre-converted to plain Ruby values via a new simple_convert helper so macros can interpolate them as $1, $2, etc.

  2. Pin stage — wrap macros as first-class objects.
    Pin::Base#collect_macros now wraps each @!macro directive in a new YardMap::Macro, which knows its method pin and exposes generate_pins_from(send_pin, source_map) to expand the macro at a call site. The existing inline directive handling in SourceMap::Mapper was extracted into per-directive modules under YardMap::Directives::* so both source-map processing and macro expansion can share the same logic.

  3. ApiMap stage — match calls to macros, expand, store.
    ApiMap#process_macros runs after the bench is gathered but before the store update. It pairs every Pin::Ephemeral::ClassMethodSend with the method pin whose name it matches, expands each attached macro, and adds the generated pins to the store. Ephemeral pins are then filtered out so they don't pollute typechecking or completion.

Why ephemeral pins instead of resolving inline

The parser doesn't know whether a class-method-shaped call corresponds to a macro-decorated method — that match only happens after all pins (workspace + gems) are gathered, because macros can be defined in other files and/or gems. Emitting an ephemeral pin during parsing keeps the parser stateless, defers resolution to ApiMap#catalog, and gives gem-defined macros the same treatment as user-defined ones (the consuming workspace's parser sees the call; the gem's pin.macros are already loaded via YardMap::Mapper).

Other design points

  • Gem support comes for free from YardMap::Mapper's new attached_macros index — gem yardocs that contain MacroObjects become pin.macros on the corresponding method pins, and the same process_macros loop handles them.
  • Supported sub-directives inside a macro: @!method, @!attribute, @!parse. Listed in Macro::PROCESSABLE_DIRECTIVES.
  • Argument-value substitution handles literals (strings, numbers, symbols, true/false), constants (resolved to their name), arrays, and hashes. Non-literal arguments fall through to Solargraph::Parser.chain so chains stay introspectable.
  • Inline tags above the DSL call (e.g. # @return [Integer] written above multi_property :a, :b) are appended to each generated @!method directive's text, so per-call-site documentation overrides macro defaults.

TODOs

  • Fix CI
  • Test performance

🤖 Generated with Claude Code

lekemula and others added 2 commits May 13, 2026 23:47
Original 11-commit diff:
524c94e...389def6

Squashes the original branch and ports it onto current upstream/master,
where the YardMap class was gutted and replaced with DocMap/GemPins
(upstream 94006fb).

Differences from the original implementation:

- Parser layer: original work added `simple_convert` and `process_dsl_method`
  to `parser/rubyvm/{node_methods,node_processors/send_node}`. Upstream
  removed the rubyvm parser entirely. Rewrote both for the parser_gem AST
  shape: lowercase node types (`:send`, `:hash`, `:const`, `:array`),
  `:send` children indexed as `[receiver, method_name, *args]`, literals
  split into `:int`/`:float`/`:sym`/`:str` instead of `:LIT`.

- ApiMap integration: original `process_macros(pins)` hooked into a `pins`
  parameter that no longer exists. Adapted to the new `catalog(bench)`
  flow — consumes `iced_pins + live_pins + doc_map.pins`, filters
  `Pin::Ephemeral::ClassMethodSend` from iced and live separately before
  the store update. Kept the original logging.

- MethodDirective: original `Parser.process_node(...).first.last`
  regressed `spec/source_map/mapper_spec.rb:89`. Upstream had since added
  a `Pin::Method` filter inline; backported that into the extracted
  directive module.

- Spec relocation: `spec/yard_map_spec.rb` was deleted upstream. The
  `loads macros from gems` test moved to `spec/yard_map/mapper_spec.rb`
  and uses the new `pins_with(name)` (DocMap-based) helper. Assertion
  tightened from `macros.count > 0` to checking that the
  `MyStruct.my_attribute` method pin exists and exposes the macro by
  name.

- All other new files (Macro, Directives::*, Pin::Ephemeral::*,
  gem-with-yard-macros fixture, api_map_spec/clip_spec additions) landed
  unchanged from the squashed branch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The skeleton gemspec from `bundle gem` left TODO placeholders in
summary, description, homepage, and metadata fields, which Bundler
rejects in CI. Replaced with real values describing the fixture's
purpose and trimmed the file list to `lib/**/*.rb` so it doesn't depend
on `git ls-files` working in the CI checkout.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
# @param directive [YARD::Tags::Directive]
# @return [void]
def process_directive source_position, comment_position, directive
# @sg-ignore Need to add nil check here
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are mainly refactored and extracted into their own "directive processors", see yard_map/directives.

lekemula and others added 4 commits May 14, 2026 00:13
- Autocorrected style issues across the new/ported files (string quoting,
  empty-method one-liners, redundant cop disables, def-without-parens, etc).
- Excluded the gem-with-yard-macros fixture from rubocop entirely; it's a
  `bundle gem` skeleton that exists to be loaded as a gem, not as project
  source.
- Bumped Metrics/ModuleLength.Max in the todo file from 167 to 195 to
  accommodate the simple_convert helpers added to ParserGem::NodeMethods.
- Cleaned up YARD `@param` mismatches in Macro and ClassMethodSend, and
  rewrote one multi-line block chain in Macro#generate_yardoc_from.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Removed the `bundle gem` skeleton boilerplate (LICENSE, README, CHANGELOG,
CODE_OF_CONDUCT, Rakefile, bin/, the gem's own Gemfile/Gemfile.lock, RBS
sig, .gitignore). None are needed: the fixture exists only to be resolved
as a path gem and have its YARD macro loaded. What remains is the
gemspec, the macro definition, and version.rb.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`Pin::Base#assert_source_provided` raises (under SOLARGRAPH_ASSERTS=on,
as the overcommit CI job runs) when a pin is created without a `source:`.
The extracted attribute/override directive modules built `Pin::Method`,
`Pin::Parameter`, and `Pin::Reference::Override` pins without one.
Tagged them `:yard_map` since they originate from YARD `@!` directives.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The `.rubocop_todo.yml` CI job runs `rubocop -c .rubocop.yml` across the
whole repo and was failing on 8 offenses unrelated to this PR. Fixed them
in place rather than suppressing:

- Style/ArgumentsForwarding: anonymous block forwarding (`&`) in
  Solargraph.with_clean_env, UniqueType#each, Host#show_message_request.
- Style/ArrayIntersect: `(a & b).any?` -> `a.intersect?(b)` in
  TypeChecker#parameterized_arity_problems_for.
- Lint/UnreachableCode: the body of Pin::Method#combine_same_type_arity_
  signatures is intentionally preserved behind a debug stub `return`
  (upstream 6d8ce95); wrapped it in a scoped rubocop:disable with a
  comment explaining why, instead of deleting the kept code.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`ArgumentValue = Struct.new(:value)` was constructed with a keyword
argument (`ArgumentValue.new(value: ...)`). On Ruby 3.1 a plain Struct
treats that as a positional Hash, so `#value` returned `{ value: x }`
instead of `x`. That garbled `ClassMethodSend#argument_values`, which
shifted every macro placeholder (`$1`, `$2`, ...) — producing method
pins like `value` and dropping real ones. Added `keyword_init: true`.

Fixes the 6 macro specs failing on the Ruby 3.1 CI matrix job.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ruby/setup-ruby@v1 currently 404s on `head` for ubuntu-24.04
("Unavailable version head for ruby"). Removed it from the matrix so CI
isn't blocked; left a @todo to restore once setup-ruby publishes it.

See: https://github.com/castwide/solargraph/actions/runs/25863741955/job/76000137015?pr=1187

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant