Cannot generate docs with a local version of the snippets tool #144408
Assignees
Labels
d: api docs
Issues with https://api.flutter.dev/
dependency: dart
Dart team may need to help us
found in release: 3.20
Found to occur in 3.20
framework
flutter/packages/flutter repository. See also f: labels.
has reproducible steps
The issue has been confirmed reproducible and is ready to work on
P2
Important issues not at the top of the work list
team-framework
Owned by Framework team
triaged-framework
Triaged by Framework team
Comments
|
@srawlins @parlough Apologies, this is a bit of a shot in the dark. I am not exactly sure where the issue here is, but we are able to reproduce it on three different machines. Does the Dart CLI tool mess something up when |
|
@gspencergoog figured out a work around: you can run a local version of the snapshot tool by building an |
|
The failure seems to be coming from the pub tool: @sigurdm @szakarias Do you maybe have any insights? |
|
I also see the same issue on the master channel Output log (just the head of the output log, it has more)dartdoc:stdout: Documenting Flutter...
dartdoc:stdout: Discovering libraries...
dartdoc:stdout: Linking elements...
dartdoc:stdout: Precaching local docs for 800333 elements...
dartdoc:stderr: error: tool execution failed: Tool "snippet" returned non-zero exit code (1) when run as "/Users/huynq/Documents/GitHub/flutter_master/bin/cache/dart-sdk/bin/dart pub global run snippets --output-directory=doc/snippets --type=snippet" from /private/tmp/dartdoc.CkGj0
dartdoc:stderr: Input to snippet was:
dartdoc:stderr:
dartdoc:stderr: A card with `Hello World!` embedded inline within a TextSpan tree.
dartdoc:stderr:
dartdoc:stderr: ```dart
dartdoc:stderr: const Text.rich(
dartdoc:stderr: TextSpan(
dartdoc:stderr: children: <InlineSpan>[
dartdoc:stderr: TextSpan(text: 'Flutter is'),
dartdoc:stderr: WidgetSpan(
dartdoc:stderr: child: SizedBox(
dartdoc:stderr: width: 120,
dartdoc:stderr: height: 50,
dartdoc:stderr: child: Card(
dartdoc:stderr: child: Center(
dartdoc:stderr: child: Text('Hello World!')
dartdoc:stderr: )
dartdoc:stderr: ),
dartdoc:stderr: )
dartdoc:stderr: ),
dartdoc:stderr: TextSpan(text: 'the best!'),
dartdoc:stderr: ],
dartdoc:stderr: )
dartdoc:stderr: )
dartdoc:stderr: ```
dartdoc:stderr: Stderr output was:
dartdoc:stderr: Could not make "/Users/huynq/.pub-cache/bin/snippets" executable (exit code 1):
dartdoc:stderr: chmod: /Users/huynq/.pub-cache/bin/snippets: No such file or directory
dartdoc:stderr:
dartdoc:stderr:
dartdoc:stderr: from material.WidgetSpan: (file:///Users/huynq/Documents/GitHub/flutter_master/packages/flutter/lib/src/widgets/widget_span.dart:67:7)
dartdoc:stderr: error: tool execution failed: Tool "snippet" returned non-zero exit code (1) when run as "/Users/huynq/Documents/GitHub/flutter_master/bin/cache/dart-sdk/bin/dart pub global run snippets --output-directory=doc/snippets --type=snippet" from /private/tmp/dartdoc.CkGj0
dartdoc:stderr: Input to snippet was:
dartdoc:stderr:
dartdoc:stderr: This example shows some [Text] when the `_visible` member field is true, and
dartdoc:stderr: hides it when it is false:
dartdoc:stderr:
dartdoc:stderr: ```dart
dartdoc:stderr: Opacity(
dartdoc:stderr: opacity: _visible ? 1.0 : 0.0,
dartdoc:stderr: child: const Text("Now you see me, now you don't!"),
dartdoc:stderr: )
dartdoc:stderr: ```
dartdoc:stderr: Stderr output was:
dartdoc:stderr: Could not make "/Users/huynq/.pub-cache/bin/snippets" executable (exit code 1):
dartdoc:stderr: chmod: /Users/huynq/.pub-cache/bin/snippets: No such file or directory
.....flutter doctor -v (master)[!] Flutter (Channel master, 3.20.0-16.0.pre.40, on macOS 14.1 23B74 darwin-x64, locale en-VN)
• Flutter version 3.20.0-16.0.pre.40 on channel master at /Users/huynq/Documents/GitHub/flutter_master
! Warning: `flutter` on your path resolves to /Users/huynq/Documents/GitHub/flutter/bin/flutter, which is not inside your current Flutter SDK checkout at /Users/huynq/Documents/GitHub/flutter_master. Consider adding /Users/huynq/Documents/GitHub/flutter_master/bin to the front of your path.
! Warning: `dart` on your path resolves to /Users/huynq/Documents/GitHub/flutter/bin/dart, which is not inside your current Flutter SDK checkout at /Users/huynq/Documents/GitHub/flutter_master. Consider adding /Users/huynq/Documents/GitHub/flutter_master/bin to the front of your path.
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision a78a0d5934 (48 minutes ago), 2024-02-29 21:22:19 -0500
• Engine revision 34cf95c7bd
• Dart version 3.4.0 (build 3.4.0-190.0.dev)
• DevTools version 2.33.1
• If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly to perform update checks and upgrades.
[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
• Android SDK at /Users/huynq/Library/Android/sdk
• Platform android-34, build-tools 34.0.0
• ANDROID_HOME = /Users/huynq/Library/Android/sdk
• Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 15.0.1)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Build 15A507
• CocoaPods version 1.15.2
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2023.1)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• android-studio-dir = /Applications/Android Studio.app/
• Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)
[✓] VS Code (version 1.87.0)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.82.0
[✓] Connected device (4 available)
• RMX2001 (mobile) • EUYTFEUSQSRGDA6D • android-arm64 • Android 11 (API 30)
• iPhone (mobile) • d9a94afe2b649fef56ba0bfeb052f0f2a7dae95e • ios • iOS 15.8 19H370
• macOS (desktop) • macos • darwin-x64 • macOS 14.1 23B74 darwin-x64
• Chrome (web) • chrome • web-javascript • Google Chrome 122.0.6261.94
[✓] Network resources
• All expected network resources are available.
! Doctor found issues in 1 category. |
huycozy
added
framework
|
Yeah something is wrong. It seems the "binstub" (small script that can be used to run the activated binary) doesn't exist while pub expects it to. I cannot reproduce I tried outputI get some other errors, but not the pub one. Is this something that used to work, but is now failing? Or a new task that is supposed to work? |
|
@sigurdm Thanks for taking a look.
This used to work about 8 month ago when I last worked on the docs.
The error you're seeing is dart-lang/sdk#55056, which is fixed, but hasn't rolled into flutter yet. You can work around it by disabling analytics for dart with |
goderbauer
added
P2
|
can this be closed as the issue is fixed on the dart side |
|
Where/how was this problem fixed on the dart side? |
|
I was looking at your comment above about dart-lang/sdk#55056 and assumed that was the fix, if that is not the case let me know and I will direct this issue to the appropriate team |
|
@a-siva - that was a fix for another problem, that I saw while trying to reproduce. And that other problem has indeed been fixed, but is not the one this issue is about. For now this looks like a pub issue - I will try to find time to look at it. Is p2 the correct priority, since this has a workaround? |
|
Are you running things in parallel? |
I don't think there's anything running in parallel. Although, I don't know for sure if dartdoc is parallelizing anything with tool invocations (maybe @srawlins can speak to that with more certainty).
The only lower P-level we have would be P3, which is basically for something that'll never get fixed. Despite the work around, I am hoping that this can eventually get fixed since it makes working on our documentation somewhat painful. It's not super time-critical since this is not a workflow we execute every day, though. |
That's entirely possible actually. |
There was a bug in the analytics package that likely caused this error based on the outputted logs above, the analytics version should be getting bumped in the sdk to fix this: https://dart-review.googlesource.com/c/sdk/+/358220 |
|
@eliasyishak That analytics error was a separate problem that we saw while reproducing the other problem the original post is about. As far as I can tell, that original problem is not fixed yet (and likely unrelated to analytics). |
|
@sigurdm are you the right owner for this issue ? |
|
Tried reproducing again Not sure how using a local version is supposed to work, as the My only guess as to what happened is that we global run Probably we don't need to delete the binstub before rewriting it in: And probably we could write the binstub to a temp-file, chmod it there, and do an atomic move. Filed dart-lang/pub#4216 I would highly suggest you don't rely on Why is |
|
Reading the code again, we already create the binstub atomically since dart-lang/pub#3285 Are you using a dart sdk from before that? Otherwise the error message should also not be |
|
Thanks for looking into this!
As mentioned in my original post: You have to modify the script here to activate a local version.
Agreed, that this could be improved. I don't know how to set this up with the dartdocs generation (see below).
Dartdoc is configured here to invoke it.
The problem still reproduces for me today with tip of tree flutter using Dart SDK version: 3.5.0-47.0.dev. |
Oh sorry - completely misread that - I need more coffee.
Yes! - I found the bug! https://github.com/dart-lang/pub/pull/3285/files#diff-609b203653065d15282bde9c47cbf5c38885fa92957ce5b89adbf9be5db0b1c8R793 joins an absolute path to the temp-path, effectively using just the absolute path. Made it look like the temp-path wasn't used at all Will make a fix! |
|
Fix is taking a bit longer than expected. There are a few more races involved. |
|
Awesome. Thanks for debugging this! |
|
Wow, thanks @sigurdm!! Great find!! |
(Thanks - Not sure how proud I am - I created this bug in the first place :) ) |
1 task
|
This now appears to be causing the build to break. Not sure what changed so that the build machines now fail as well as running it locally, but maybe it's just some race condition in Linux that changed when it was updated. |
gspencergoog
added
P0
|
Returning this to P2 priority now that dart-lang/pub#4205 has been reverted in dart-lang/pub#4255. |
gspencergoog
added
P2
This was referenced May 2, 2024
auto-submit bot
pushed a commit
that referenced
this issue
May 3, 2024
## Description This moves the snippets package back into the Flutter repo so that API documentation generation can happen without the use of `dart pub global run` because `pub run` doesn't handle concurrency well. The change modifies the dartdoc building process to include building an executable from the snippets tool and installing that in the cache directory for use during docs generation. The snippets tool will reside in dev/snippets, where it originally resided before being moved to https://github.com/flutter/assets-for-api-docs. The snippets code itself is unchanged from the code that is in https://github.com/flutter/assets-for-api-docs/packages/snippets. ## Related Issues - #144408 - #147609 - #147645 ## Tests - Added snippets tests to the overall testing build.
auto-submit bot
pushed a commit
that referenced
this issue
May 7, 2024
…l. (#147893) ## Description This fixes the API doc generation that I broke when I moved the snippets tool into the framework. It removes the last of the template support (properly this time), and makes sure all of the tests pass. ## Related Issues - #144408 - #147609 - #147645 ## Tests - Fixed tests, including smoke test of doc generation.
itsjustkevin
pushed a commit
to itsjustkevin/flutter
that referenced
this issue
May 8, 2024
This moves the snippets package back into the Flutter repo so that API documentation generation can happen without the use of `dart pub global run` because `pub run` doesn't handle concurrency well. The change modifies the dartdoc building process to include building an executable from the snippets tool and installing that in the cache directory for use during docs generation. The snippets tool will reside in dev/snippets, where it originally resided before being moved to https://github.com/flutter/assets-for-api-docs. The snippets code itself is unchanged from the code that is in https://github.com/flutter/assets-for-api-docs/packages/snippets. - flutter#144408 - flutter#147609 - flutter#147645 - Added snippets tests to the overall testing build.
itsjustkevin
pushed a commit
to itsjustkevin/flutter
that referenced
this issue
May 8, 2024
…l. (flutter#147893) ## Description This fixes the API doc generation that I broke when I moved the snippets tool into the framework. It removes the last of the template support (properly this time), and makes sure all of the tests pass. ## Related Issues - flutter#144408 - flutter#147609 - flutter#147645 ## Tests - Fixed tests, including smoke test of doc generation.
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Steps to reproduce:
dev/bots/docs.shto generate the docsRunning the command from the error directly works without issues. The file it is complaining about not existing definitely exists.
The text was updated successfully, but these errors were encountered: