Resolve non-imported symbols in comments [#1153] #1298

astashov · 2016-12-17T21:50:25Z

@Hixie noticed, that sometimes, a library, like the Flutter rendering
library, wants to refer to another library, like the Flutter widgets
library, in the documentation, but doesn't want to introduce a
dependency in the code. Currently, there’s no mechanisms in dartdoc,
which allow that.

This commit adds that. You can use either a short name (e.g. [icon]) or
a fully qualified name (like, [material.Icon.icon]), in the HTML docs,
it always will be shown as a short name though (is that a desired
behavior?). Dartdoc will go over all the libraries, classes, methods,
properties, constants of the package that is being documented, and will
try to resolve the symbol. If it’s successful, it will add the link to
it, same way as when the symbol is in scope.

Some outstanding questions:

What to do in case of ambiguity here? Show a warning, crash with an
error message, ignore?
Do we want to show short name in case a fully qualified name is
provided? I saw in the comments in the ticket
(handle dartdoc references to non-imported symbols (was: dartdoc-specific imports) #1153 (comment)) that @Hixie would want that.
Anything else?

Testing: Unit tests, of course, also tried to generate Flutter docs with
that, and observed that the links for previously non-resolved symbols
work.

(fixes #1153)

@Hixie

@Hixie noticed, that sometimes, a library, like the Flutter rendering library, wants to refer to another library, like the Flutter widgets library, in the documentation, but doesn't want to introduce a dependency in the code. Currently, there’s no mechanisms in dartdoc, which allows that. This commit adds that. You can use either a short name (e.g. [icon]) or a fully qualified name (like, [material.Icon.icon]), in the HTML docs, it always will be shown as a short name though (is that a desired behavior?). Dartdoc will go over all the libraries, classes, methods, properties, constants of the package that is being documented, and will try to resolve the symbol. If it’s successful, it will add the link to it, same way as when the symbol is in scope. Some outstanding questions: * What to do in case of ambiguity here? Show a warning, crash with an error message, ignore? * Do we want to show short name in case a fully qualified name is provided? I saw in the comments in the ticket (dart-lang#1153 (comment)) that @Hixie would want that. * Anything else? Testing: Unit tests, of course, also tried to generate Flutter docs with that, and observed that the links for previously non-resolved symbols work.

devoncarew · 2016-12-19T15:50:58Z

Reviewing now -

What to do in case of ambiguity here? Show a warning, crash with an error message, ignore?

I believe @Hixie mentioned fast-fail. We should definitely at least warn. Dartdoc does generate docs unless there are significant parse errors in the code - it's lenient towards somewhat bad source, so you can always generate docs.

The approach the analyzer takes is to allow clients to opt-into failing on warnings (--fatal-warnings). This is probably what we should do here as well. It's outside the scope of this PR however, so I would say warn, and open an issue to add a --fatal-warnings flag and associated behavior.

Do we want to show short name in case a fully qualified name is provided? I saw in the comments in the ticket

Sounds like (from @Hixie) yes :)

devoncarew · 2016-12-19T15:52:14Z

Re: material.Icon.icon as the fully qualified name - there are cases where this format is not unique (specifically, in the flutter package, there are two libraries with the name material). While that's a nice, clean fully specified format, we may need something more unique.

devoncarew · 2016-12-19T15:57:23Z

lib/src/markdown_processor.dart

  }
+  final linkedElement = result.element;


Please add the type here (and below).

devoncarew · 2016-12-19T15:59:24Z

lib/src/markdown_processor.dart

+    return new MatchingLinkResult(result.values.first, result.values.first.name);
+  } else {
+    if (_emitWarning) {
+      print("Abiguous reference '${codeRef}' in '${element.fullyQualifiedName}', " +


sp: ambiguous

Also, can you get the file and line number here in order to print them out?

Something like: Ambiguous reference to [foo], lib/my_lib.dart, line 100.

Although, I see that we're not doing this (locating by file and line number) in the original code, so while this would be great, it's not necessary for this PR.

devoncarew · 2016-12-19T16:05:46Z

lib/src/markdown_processor.dart

+}
+
+MatchingLinkResult _findRefElementInLibrary(String codeRef, ModelElement element, List<CommentReference> commentRefs) {
+  final package = element.library.package;


please add the type here

devoncarew · 2016-12-19T16:07:35Z

lib/src/markdown_processor.dart

+MatchingLinkResult _findRefElementInLibrary(String codeRef, ModelElement element, List<CommentReference> commentRefs) {
+  final package = element.library.package;
+  final Map<String, ModelElement> result = {};
+  package.libraries.forEach((library) {


This lookup should probably live somewhere more general, like as a method on Package. That would probably also help with performance issues, as the set of all symbols (and their fully-qualified names) could be calculated once.

Oh, good point!

devoncarew · 2016-12-19T16:12:32Z

OK, looks like String get fullyQualifiedName already exists on model elements. Let's try the existing libName[.enclosingElement].symbolName and see how that works.

* Add types to declarations * Add filename/line number to the warnings * Move the code for gathering all the model elements of a package to the package's accessor and cache it.

astashov · 2016-12-20T02:11:22Z

@devoncarew I addessed your feedback, thanks! PTAL

devoncarew · 2016-12-21T00:13:11Z

lib/src/model.dart

@@ -1425,6 +1425,26 @@ abstract class ModelElement implements Comparable, Nameable, Documentable {
    return (_fullyQualifiedName ??= _buildFullyQualifiedName());
  }

+  String get sourceFileName {


nit: this can use the function shorthand (=>)

devoncarew · 2016-12-21T00:17:00Z

lib/src/markdown_processor.dart

-      return (modelElement.element.computeNode() as AnnotatedNode)
-          .documentationComment
-          .references;
+    if ((modelElement.element.computeNode() as AnnotatedNode).documentationComment != null) {


We're doing this cast twice inside this if statement (modelElement.element.computeNode() as AnnotatedNode) - perhaps create a new local variable to host the AnnotatedNode?

devoncarew · 2016-12-21T00:23:30Z

lib/src/markdown_processor.dart

-        result[modelElement.fullyQualifiedName] = modelElement;
-      }
-    });
+  package.allModelElements.forEach((modelElement) {


Last comment, I promise :)

If we use a Map here, in the event we have multiple matches, the entry in the map could be overridden. So that would hide the fact that the reference is not unique. Perhaps use a List instead?

And we'll be iterating over a lot of elements - I tend to avoid using a closure for that; a for loop will be more performant.

You might do something thuswise:

List<ModelElement> results = []; for (ModelElement element in package.allModelElements) { if (codeRef.contains('.')) { // It's a fully qualified reference. if (element.fullyQualifiedName == codeRef) { results.add(element); break; } } else { if (element.name == codeRef) { results.add(element); } } }

devoncarew · 2016-12-21T00:25:03Z

@astashov, I think there were one or two issues w/ the lookup code - I added a proposal code snippet; everything else looks good.

astashov · 2016-12-21T03:13:43Z

Hmm, it seems like there's one drawback with using fullyQualifiedName - it's the same for inherited methods/properties. E.g. tons of classes inherit from Widget in Flutter, so they all have key property, and all their fully qualified name are material.Widget.key. So, I see a lot of warnings in Flutter docs like this:

Ambiguous reference to [key] in 'material.State.didUpdateConfig' (/Users/anton/projects/flutter/packages/flutter/lib/src/widgets/framework.dart:785). We found matches to the following elements: 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.DataRow.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.MergeableMaterialItem.key', 'material.Widget.key', 'material.MergeableMaterialItem.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.MergeableMaterialItem.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.TableRow.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key', 'material.Widget.key'

Hixie · 2016-12-21T03:42:01Z

Isn't that fine? They can all just point to Widget.key.

Hixie · 2016-12-21T03:58:16Z

Having said that, I don't think we'd want [key] to match material.Widget.key. I think we just want things in the global scope of their respective libraries to be accessible without qualification.

astashov · 2016-12-21T04:28:43Z

Isn't that fine? They can all just point to Widget.key.

Oh right, that's actually fine. Sorry :)

Having said that, I don't think we'd want [key] to match material.Widget.key. I think we just want things in the global scope of their respective libraries to be accessible without qualification.

Umm, now I'm lost :) Could you please elaborate? Why you don't want key to match material.Widget.key? Because it's a different library?

Hixie · 2016-12-21T06:54:57Z

Because it's not in scope. Conceptually, the way I imagine this working is that you import all the libraries. But even if you did that, in code, key wouldn't resolve to Widget.key, right? You can only get to Widget.key if you qualify it via an instance of the class (or for statics via the class itself).

More practically, if we consider every member of every class to be in scope everywhere, we're going to have conflicts up the wazoo. :-)

astashov · 2016-12-22T06:13:55Z

So, you don't want [key] to match material.Widget.key, you only want key to match material.key or something like that? I.e. the top level stuff only? And Widget.key to match material.Widget.key. Something like that?

Hixie · 2016-12-22T06:16:58Z

That would be my recommendation, yeah.

devoncarew · 2016-12-22T23:20:28Z

Cycling back to this, what @Hixie says makes sense. It sounds like material.Widget.key would be the fully qualified way to specify key, and Widget.key would be legal shorthand for it as long as there wasn't a Widget.key in another library you're generating docs for.

Similarly with material.Widget and Widget.

From now on, if the reference was, for example, `[key]`, we won't match `material.Widget.key`, we will only match `material.key` (i.e. top-level `key` symbol). If you still want to match `Widget.key`, you should use either `[Widget.key]` or `[material.Widget.key]`.

astashov · 2016-12-23T04:42:51Z

Okay, I've updated the PR. So, from now on, if the reference was, for example, [key], we won't match material.Widget.key, we will only match material.key (i.e. top-level key symbol). If you still want to match Widget.key, you should use either [Widget.key] or [material.Widget.key].

That's how you want it to work, right?

Hixie · 2016-12-23T07:37:00Z

How do I know what value to use for the library name? Is it (exclusively) the identifiers used with library keywords?

astashov · 2016-12-24T02:36:10Z

Yeah, I think so. E.g. Widget is in lib/src/widgets/framework.dart, which is exported in lib/widgets.dart, and that file has library widgets; on top, so it will be widgets.Widget. The confusing thing that e.g. lib/material.dart also exports lib/widgets.dart, so material.Widget is also a valid reference. I believe there's an issue for that though (#1158)

Hixie · 2016-12-24T05:53:36Z

SGTM

devoncarew · 2016-12-27T17:18:57Z

lgtm! Landing...

googlebot added the cla: yes Google CLA check succeeded. label Dec 17, 2016

devoncarew reviewed Dec 19, 2016

View reviewed changes

lib/src/markdown_processor.dart

}

final linkedElement = result.element;

Copy link

Member

devoncarew Dec 19, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add the type here (and below).

devoncarew reviewed Dec 19, 2016

View reviewed changes

Address @devoncarew feedback

d00d2ea

* Add types to declarations * Add filename/line number to the warnings * Move the code for gathering all the model elements of a package to the package's accessor and cache it.

devoncarew reviewed Dec 21, 2016

View reviewed changes

devoncarew merged commit 3c653b4 into dart-lang:master Dec 27, 2016

alexander-doroshko mentioned this pull request Jan 11, 2021

Analysis Server doesn't provide navigation information for non-imported dartdoc symbols in comments dart-lang/sdk#44637

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Resolve non-imported symbols in comments [#1153] #1298

Resolve non-imported symbols in comments [#1153] #1298

astashov commented Dec 17, 2016 •

edited by devoncarew

devoncarew commented Dec 19, 2016

devoncarew commented Dec 19, 2016

devoncarew Dec 19, 2016

devoncarew Dec 19, 2016

devoncarew Dec 19, 2016

devoncarew Dec 19, 2016

devoncarew Dec 19, 2016

devoncarew Dec 19, 2016

astashov Dec 20, 2016

devoncarew commented Dec 19, 2016

astashov commented Dec 20, 2016

devoncarew Dec 21, 2016

devoncarew Dec 21, 2016

devoncarew Dec 21, 2016

devoncarew commented Dec 21, 2016

astashov commented Dec 21, 2016

Hixie commented Dec 21, 2016

Hixie commented Dec 21, 2016

astashov commented Dec 21, 2016

Hixie commented Dec 21, 2016

astashov commented Dec 22, 2016

Hixie commented Dec 22, 2016

devoncarew commented Dec 22, 2016

astashov commented Dec 23, 2016

Hixie commented Dec 23, 2016

astashov commented Dec 24, 2016

Hixie commented Dec 24, 2016

devoncarew commented Dec 27, 2016

Resolve non-imported symbols in comments [#1153] #1298

Resolve non-imported symbols in comments [#1153] #1298

Conversation

astashov commented Dec 17, 2016 • edited by devoncarew

devoncarew commented Dec 19, 2016

devoncarew commented Dec 19, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

devoncarew commented Dec 19, 2016

astashov commented Dec 20, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

devoncarew commented Dec 21, 2016

astashov commented Dec 21, 2016

Hixie commented Dec 21, 2016

Hixie commented Dec 21, 2016

astashov commented Dec 21, 2016

Hixie commented Dec 21, 2016

astashov commented Dec 22, 2016

Hixie commented Dec 22, 2016

devoncarew commented Dec 22, 2016

astashov commented Dec 23, 2016

Hixie commented Dec 23, 2016

astashov commented Dec 24, 2016

Hixie commented Dec 24, 2016

devoncarew commented Dec 27, 2016

astashov commented Dec 17, 2016 •

edited by devoncarew