Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Analysis server sometimes fails to resolve flutter_gen package after running "pub get" #3438

Closed
orestesgaolin opened this issue Jun 29, 2021 · 27 comments
Labels
in commands Relates to commands (usually invoked from the command Palette) is bug
Milestone

Comments

@orestesgaolin
Copy link

orestesgaolin commented Jun 29, 2021

Describe the bug
Almost every hour there's a situation when after getting packages or updating the .arb files the analyzer hangs for a second and then looses information about the Flutter-generated localization files.

This is partially related to #3270 as sometimes this issue occurs after Flutter extension tries to format the file. In this particular case however, it occurred after adding new package.

To Reproduce
Steps to reproduce the behavior:

  1. Open project and add new package in pubspec.yaml
    • it may be significant that the package is a local flutter package stored in ./packages/package_name subdirectory
  2. Run Pub: Get packages in command palette
  3. Try to reference the new package somewhere in the widget tree
  4. Notice it doesn't get imported
  5. Run Pub: Get packages again
  6. Analyzer restarts and after that it looses information about AppLocalizations, running Pub: Get packages doesn't help and only way out of that is either restarting analyzer or VS Code entirely

I was able to reproduce it in my project just by removing package from pubspec.yaml, calling Pub Get Packages several times, and then adding the package again.

Expected behavior
The package should be available immediately after getting packages for the first time.
The AppLocalizations should not be lost, as they are still present in the .dart-tool directory.

Screenshots
CleanShot 2021-06-29 at 11 23 33@2x

Logs

I sent the Dart instrumentation and analysis logs to logs@dartcode.org

Versions (please complete the following information):

[✓] Flutter (Channel stable, 2.2.2, on macOS 11.4 20F71 darwin-x64, locale pl-PL)
    • Flutter version 2.2.2 at /Users/dominik/fvm/versions/stable
    • Framework revision d79295af24 (3 weeks ago), 2021-06-11 08:56:01 -0700
    • Engine revision 91c9fc8fe0
    • Dart version 2.13.3

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /Users/dominik/Library/Android/sdk
    • Platform android-30, build-tools 30.0.3
    • ANDROID_HOME = /Users/dominik/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.5.1, Build version 12E507
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)

[✓] VS Code (version 1.57.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.23.0

[✓] Connected device (3 available)
    • SM T500 (mobile) • R9TR105TMWJ • android-arm64  • Android 11 (API 30)
    • macOS (desktop)  • macos       • darwin-x64     • macOS 11.4 20F71 darwin-x64
    • Chrome (web)     • chrome      • web-javascript • Google Chrome 91.0.4472.114

• No issues found!
@DanTup
Copy link
Member

DanTup commented Jun 29, 2021

Can you reliably reproduce this? I tried, but couldn't - though the package appeared in the import completion immediately after the first Pub: Get Packages run.

I'd be interested to know:

  • Is flutter_gen listed in .dart_tool/package_config.json when this happens?
  • Could you capture a log using Dart: Capture Logs (leave everything ticked) starting just before you first run Pub: Get Packages and stopping after you get the analysis error for AppLocalizations not appearing).
  • Can you reproduce this in a simple newly-created project (Flutter: New Application Project) or is it specific to this project?
    Thanks!

@DanTup DanTup added the awaiting info Requires more information from the customer to progress label Jun 29, 2021
@orestesgaolin
Copy link
Author

  • Is flutter_gen listed in .dart_tool/package_config.json when this happens?

Yes, it's there

CleanShot 2021-06-29 at 12 43 02@2x

  • Could you capture a log using Dart: Capture Logs (leave everything ticked) starting just before you first run Pub: Get Packages and stopping after you get the analysis error for AppLocalizations not appearing).

Yes, I sent the logs to logs@dartcode.org

  1. Open VS Code
  2. Wait for it to settle
  3. Enable logging
  4. Remove local package from pubspec.yaml, save, automatic flutter get packages runs
  5. Call Pub: Get packages
  6. Call Pub: Get packages
  7. Call Pub: Get packages
  8. Observe problems in the workspace like missing AppLocalizations

CleanShot 2021-06-29 at 12 47 48@2x

  • Can you reproduce this in a simple newly-created project (Flutter: New Application Project) or is it specific to this project?

I tried to reproduce in newly created project and after setting the l10n.yaml the same way as in my project, but it couldn't fail on its own even I had it open for many hours. I will try now with the package removing and adding.

@orestesgaolin
Copy link
Author

Success. I was able to reproduce it in an example project.

Steps to reproduce

  1. Launch the l10n_troubleshooting
  2. Let it settle and get packages
  3. Open main.dart
  4. Open pubspec.yaml
  5. Remove example package
  6. Save and let it get packages
  7. Run Pub: Get Packages once or twice
  8. Observe the error showing up in the Problems

CleanShot 2021-06-29 at 12 57 55@2x

Repository:

https://github.com/orestesgaolin/l10n_troubleshooting

Logs

Dart-Code-Log-2021-05-02 12-54-10.txt

@DanTup
Copy link
Member

DanTup commented Jun 29, 2021

@orestesgaolin thanks for the steps - although unfortunately I'm still not able to repro. I tried with the same Flutter version:

[✓] Flutter (Channel stable, 2.2.2, on macOS 11.4 20F71 darwin-x64, locale en-GB)
    • Flutter version 2.2.2 at /Users/danny/Dev/Google/flutter
    • Framework revision d79295af24 (3 weeks ago), 2021-06-11 08:56:01 -0700
    • Engine revision 91c9fc8fe0
    • Dart version 2.13.3

You mentioned flutter_gen was in your .dart_tool/package_config.json - although there should be an entry specifically for "flutter_gen" which doesn't seem to be the case from your screenshot. If that's really missing, could you try:

  • Getting the project back to a working state, then verify "flutter_gen" is explicitly in package_config.json
  • If so, run the Get Packages command a few times
  • Check whether "flutter_gen" has disappeared from the package_config.json file

@larssn
Copy link

larssn commented Jun 29, 2021

I can reproduce it too, but on an old Mac. And I see it daily anyway.

Maybe there's a race condition, or a simple timer that expects pub get to be done before it actually is?

@orestesgaolin
Copy link
Author

You mentioned flutter_gen was in your .dart_tool/package_config.json - although there should be an entry specifically for "flutter_gen" which doesn't seem to be the case from your screenshot. If that's really missing, could you try:

  • Getting the project back to a working state, then verify "flutter_gen" is explicitly in package_config.json

flutter_gen, flutter_gen_core and flutter_gen_runner are present both in failed and working state, the file seems not to change while the problem happens, I can see it being regenerated but to the exact same state as previously

CleanShot 2021-06-30 at 11 20 40@2x

  • If so, run the Get Packages command a few times
  • Check whether "flutter_gen" has disappeared from the package_config.json file

The flutter_gen entry seems to remain in package_config.json no matter how many times I run pub get.

@DanTup
Copy link
Member

DanTup commented Jun 30, 2021

Thanks for confirming. Could you try the following (after you're in the state where the invalid errors are showing):

  • Run the Dart: Open Analyzer Diagnostics command
  • Click Contexts on the left
  • Scroll down to PackageMapUriResolver
  • See if flutter_gen is listed in that section (note: the package list is its own scrollable window so you may need to scroll down within that page list to see them all)

Is it possible you could also send me both your .packages file and .dart_tool/package_config.json files? (you can send them to logs@dartcode.org) when you're in this state?

Thanks!

@orestesgaolin
Copy link
Author

Seems like flutter_gen itself is not present for the app in PackageMapUriResolver:

{
_fe_analyzer_shared = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/_fe_analyzer_shared-20.0.0/lib] ,
analyzer = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/analyzer-1.4.0/lib] ,
[redacted]
[redacted]
archive = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/archive-3.1.2/lib] ,
args = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/args-2.0.0/lib] ,
async = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/async-2.6.1/lib] ,
[redacted]
bloc = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/bloc-7.0.0/lib] ,
bloc_test = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/bloc_test-8.0.0/lib] ,
boolean_selector = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/boolean_selector-2.1.0/lib] ,
build = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/build-2.0.1/lib] ,
build_config = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/build_config-1.0.0/lib] ,
build_daemon = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/build_daemon-3.0.0/lib] ,
build_resolvers = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/build_resolvers-2.0.1/lib] ,
build_runner = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/build_runner-2.0.3/lib] ,
build_runner_core = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/build_runner_core-7.0.0/lib] ,
build_verify = [/Users/dominik/.pub-cache/git/build_verify-fb4c4447d49728ecfa7cc240f253f05f0b393c85/lib] ,
build_version = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/build_version-2.0.3/lib] ,
built_collection = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/built_collection-5.0.0/lib] ,
built_value = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/built_value-8.0.0/lib] ,
cached_network_image = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/cached_network_image-3.0.0/lib] ,
characters = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/characters-1.1.0/lib] ,
charcode = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/charcode-1.2.0/lib] ,
[redacted]
checked_yaml = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/checked_yaml-2.0.1/lib] ,
cli_util = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/cli_util-0.3.0/lib] ,
[redacted]
[redacted]
clock = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/clock-1.1.0/lib] ,
cloud_firestore = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/cloud_firestore-2.3.0/lib] ,
cloud_firestore_platform_interface = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/cloud_firestore_platform_interface-5.2.0/lib] ,
cloud_firestore_web = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/cloud_firestore_web-2.2.0/lib] ,
code_builder = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/code_builder-4.0.0/lib] ,
collection = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/collection-1.15.0/lib] ,
color = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/color-3.0.0/lib] ,
convert = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/convert-3.0.0/lib] ,
coverage = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/coverage-1.0.2/lib] ,
crypto = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/crypto-3.0.0/lib] ,
dart_style = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/dart_style-2.0.1/lib] ,
dartx = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/dartx-0.7.1/lib] ,
[redacted]
[redacted]
equatable = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/equatable-2.0.2/lib] ,
fake_async = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/fake_async-1.2.0/lib] ,
ffi = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/ffi-1.0.0/lib] ,
file = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/file-6.1.0/lib] ,
firebase_auth = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/firebase_auth-2.0.0/lib] ,
firebase_auth_platform_interface = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/firebase_auth_platform_interface-5.0.0/lib] ,
firebase_auth_web = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/firebase_auth_web-2.0.0/lib] ,
firebase_core = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/firebase_core-1.3.0/lib] ,
firebase_core_platform_interface = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/firebase_core_platform_interface-4.0.1/lib] ,
firebase_core_web = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/firebase_core_web-1.1.0/lib] ,
firebase_crashlytics = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/firebase_crashlytics-2.0.7/lib] ,
firebase_crashlytics_platform_interface = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/firebase_crashlytics_platform_interface-3.0.6/lib] ,
firebase_storage = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/firebase_storage-8.1.3/lib] ,
firebase_storage_platform_interface = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/firebase_storage_platform_interface-2.1.2/lib] ,
firebase_storage_web = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/firebase_storage_web-1.1.2/lib] ,
[redacted]
fixnum = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/fixnum-1.0.0/lib] ,
fl_chart = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/fl_chart-0.36.2/lib] ,
flow_builder = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/flow_builder-0.0.3/lib] ,
flutter = [/Users/dominik/fvm/versions/stable/packages/flutter/lib] ,
flutter_bloc = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/flutter_bloc-7.0.1/lib] ,
flutter_blurhash = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/flutter_blurhash-0.6.0/lib] ,
flutter_cache_manager = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/flutter_cache_manager-3.1.1/lib] ,
flutter_cache_manager_firebase = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/flutter_cache_manager_firebase-2.0.0/lib] ,
flutter_gen_core = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/flutter_gen_core-3.1.1/lib] ,
flutter_gen_runner = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/flutter_gen_runner-3.1.1/lib] ,
flutter_localizations = [/Users/dominik/fvm/versions/stable/packages/flutter_localizations/lib] ,
flutter_plugin_android_lifecycle = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/flutter_plugin_android_lifecycle-2.0.2/lib] ,
flutter_secure_storage = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/flutter_secure_storage-4.0.0/lib] ,
flutter_svg = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/flutter_svg-0.22.0/lib] ,
flutter_test = [/Users/dominik/fvm/versions/stable/packages/flutter_test/lib] ,
flutter_web_plugins = [/Users/dominik/fvm/versions/stable/packages/flutter_web_plugins/lib] ,
font_awesome_flutter = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/font_awesome_flutter-9.0.0/lib] ,
[redacted]
formz = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/formz-0.4.0/lib] ,
frontend_server_client = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/frontend_server_client-2.1.0/lib] ,
gap = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/gap-2.0.0/lib] ,
glob = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/glob-2.0.0/lib] ,
google_maps_flutter = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/google_maps_flutter-2.0.6/lib] ,
google_maps_flutter_platform_interface = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/google_maps_flutter_platform_interface-2.1.0/lib] ,
google_sign_in = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/google_sign_in-5.0.4/lib] ,
google_sign_in_platform_interface = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/google_sign_in_platform_interface-2.0.1/lib] ,
google_sign_in_web = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/google_sign_in_web-0.10.0/lib] ,
graphs = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/graphs-2.0.0/lib] ,
hive = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/hive-2.0.0/lib] ,
http = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/http-0.13.3/lib] ,
http_multi_server = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/http_multi_server-3.0.1/lib] ,
http_parser = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/http_parser-4.0.0/lib] ,
hydrated_bloc = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/hydrated_bloc-7.0.1/lib] ,
image = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/image-3.0.2/lib] ,
in_app_review = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/in_app_review-2.0.2/lib] ,
in_app_review_platform_interface = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/in_app_review_platform_interface-2.0.2/lib] ,
intl = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/intl-0.17.0/lib] ,
io = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/io-1.0.0/lib] ,
js = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/js-0.6.3/lib] ,
json_annotation = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/json_annotation-4.0.1/lib] ,
logging = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/logging-1.0.1/lib] ,
[redacted]
matcher = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/matcher-0.12.10/lib] ,
meta = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/meta-1.3.0/lib] ,
mime = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/mime-1.0.0/lib] ,
mocktail = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/mocktail-0.1.1/lib] ,
nested = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/nested-1.0.0/lib] ,
node_preamble = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/node_preamble-2.0.0/lib] ,
octo_image = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/octo_image-1.0.0+1/lib] ,
package_config = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/package_config-2.0.0/lib] ,
path = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/path-1.8.0/lib] ,
path_drawing = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/path_drawing-0.5.1/lib] ,
path_parsing = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/path_parsing-0.2.1/lib] ,
path_provider = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/path_provider-2.0.1/lib] ,
path_provider_linux = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/path_provider_linux-2.0.0/lib] ,
path_provider_macos = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/path_provider_macos-2.0.0/lib] ,
path_provider_platform_interface = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/path_provider_platform_interface-2.0.1/lib] ,
path_provider_windows = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/path_provider_windows-2.0.0/lib] ,
pedantic = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/pedantic-1.11.0/lib] ,
petitparser = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/petitparser-4.0.2/lib] ,
platform = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/platform-3.0.0/lib] ,
plugin_platform_interface = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/plugin_platform_interface-2.0.0/lib] ,
pool = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/pool-1.5.0/lib] ,
process = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/process-4.1.0/lib] ,
provider = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/provider-5.0.0/lib] ,
pub_semver = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/pub_semver-2.0.0/lib] ,
pubspec_parse = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/pubspec_parse-1.0.0/lib] ,
quiver = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/quiver-3.0.0/lib] ,
rxdart = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/rxdart-0.27.1/lib] ,
[redacted]
shelf = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/shelf-1.0.0/lib] ,
shelf_packages_handler = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/shelf_packages_handler-3.0.0/lib] ,
shelf_static = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/shelf_static-1.0.0/lib] ,
shelf_web_socket = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/shelf_web_socket-1.0.0/lib] ,
sign_in_with_apple = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/sign_in_with_apple-3.0.0/lib] ,
sky_engine = [/Users/dominik/fvm/versions/stable/bin/cache/pkg/sky_engine/lib] ,
source_map_stack_trace = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/source_map_stack_trace-2.1.0/lib] ,
source_maps = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/source_maps-0.10.10/lib] ,
source_span = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/source_span-1.8.1/lib] ,
sqflite = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/sqflite-2.0.0+3/lib] ,
sqflite_common = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/sqflite_common-2.0.0+2/lib] ,
stack_trace = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/stack_trace-1.10.0/lib] ,
[redacted]
stream_channel = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/stream_channel-2.1.0/lib] ,
stream_transform = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/stream_transform-2.0.0/lib] ,
string_scanner = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/string_scanner-1.1.0/lib] ,
synchronized = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/synchronized-3.0.0/lib] ,
[redacted]
term_glyph = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/term_glyph-1.2.0/lib] ,
test = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/test-1.16.8/lib] ,
test_api = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/test_api-0.3.0/lib] ,
test_core = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/test_core-0.3.19/lib] ,
time = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/time-2.0.0/lib] ,
timing = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/timing-1.0.0/lib] ,
typed_data = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/typed_data-1.3.0/lib] ,
url_launcher = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/url_launcher-6.0.4/lib] ,
url_launcher_linux = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/url_launcher_linux-2.0.0/lib] ,
url_launcher_macos = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/url_launcher_macos-2.0.0/lib] ,
url_launcher_platform_interface = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/url_launcher_platform_interface-2.0.2/lib] ,
url_launcher_web = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/url_launcher_web-2.0.0/lib] ,
url_launcher_windows = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/url_launcher_windows-2.0.0/lib] ,
uuid = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/uuid-3.0.4/lib] ,
vector_math = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/vector_math-2.1.0/lib] ,
very_good_analysis = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/very_good_analysis-2.1.1/lib] ,
vm_service = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/vm_service-6.2.0/lib] ,
watcher = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/watcher-1.0.0/lib] ,
web_socket_channel = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/web_socket_channel-2.1.0/lib] ,
webkit_inspection_protocol = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/webkit_inspection_protocol-1.0.0/lib] ,
win32 = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/win32-2.0.0/lib] ,
xdg_directories = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/xdg_directories-0.2.0/lib] ,
xml = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/xml-5.0.2/lib] ,
yaml = [/Users/dominik/.pub-cache/hosted/pub.dartlang.org/yaml-3.1.0/lib] ,
}

As requested I sent the files to logs@dartcode.org

@DanTup
Copy link
Member

DanTup commented Jul 8, 2021

Sorry for the delay!

@bwilkerson do you have any thoughts on what might trigger this? There's a generated project at .dart_tool/flutter_gen which is listed in .dart_tool/package_config.json (but not .packages - I don't know if that's correct, but it doesn't generally cause issues since the former is used).

Sometimes for @orestesgaolin running pub get causes the flutter_gen package to disappear from the PackageMapUriResolver list in the servers diagnostic page and it starts reporting errors for package:flutter_gen URIs.

I had a thought that maybe pub get was modifying .packages and maybe (based on timing) the server would use that (which doesn't contain flutter_gen). However, if I manually modify just .packages myself, that doesn't seem to trigger the issue for me (that is, flutter_gen remains in the analyzers PackageMapUriResolver list).

@DanTup
Copy link
Member

DanTup commented Jul 8, 2021

Actually, I have managed to reproduce this locally (although I do still see flutter_gen in package_config.json- it's not listed in the diagnostics page and has generated diagnostics though). I'll add some logging into the server and see if I can track down what's happening.

@DanTup
Copy link
Member

DanTup commented Jul 8, 2021

So, this is weird.. I added some logging in the server to dump the package_json.config file as it reads it. The log contains the whole file but it is missing flutter_gen. It ends like this:

    {
      "name":: "vector_math",
      "rootUri":: "file::///Users/danny/.pub-cache/hosted/pub.dartlang.org/vector_math-2.1.0",
      "packageUri":: "lib/",
      "languageVersion":: "2.12"
    },
    {
      "name":: "flutter_app_3112",
      "rootUri":: "../",
      "packageUri":: "lib/",
      "languageVersion":: "2.12"
    }
  ],
  "generated":: "2021-07-08T14::37::08.084465Z",
  "generator":: "pub",
  "generatorVersion":: "2.14.0-289.0.dev"
}

But if I open the file I now have on disk, it ends like this:

		{
			"name": "flutter_gen",
			"rootUri": "flutter_gen",
			"languageVersion": "2.12"
		}
	],
	"generated": "2021-07-08T14:37:08.084465Z",
	"generator": "pub",
	"generatorVersion": "2.14.0-289.0.dev"
}

The timestamps are identical, but the one the server read was missing the package (it was also formatted differently). My suspicion is that this file is being written twice, once containing only the explicit packages, and then again with flutter_gen added. The server picked up the first one, and the second one was written while/before it set up its file watcher, so it didn't reload on the second one.

If it's pub writing this file twice, maybe it's not so difficult to fix. If pub is writing it and then calling flutter and its re-writing it, then it might be a little trickier.

@DanTup
Copy link
Member

DanTup commented Jul 8, 2021

It does appear to be Flutter adding this in:

https://github.com/flutter/flutter/blob/88f38110553e5c407649ff59f43ac476c5a64aeb/packages/flutter_tools/lib/src/dart/pub.dart#L463

  /// if [generateSyntheticPackage] is true then insert flutter_gen synthetic
  /// package into the package configuration. This is used by the l10n localization
  /// tooling to insert a new reference into the package_config file, allowing the import
  /// of a package URI that is not specified in the pubspec.yaml

@bwilkerson I think what's happening here is:

  • pub get writes the package_config.json file with the pubspec dependencies
  • server sees the modification event, destroys old contexts and starts creating new ones
  • flutter injects flutter_gen into the file
  • server creates its file system watcher

I wonder if the server could either read the package file after creating the watcher, or check it wasn't modified in the meantime after creating the watcher. If I've understood correctly, I think the package file is being read in the call from line 415 here, and then line 436 sets up the watcher:

https://github.com/dart-lang/sdk/blob/ccb3943e48524a993b6ca6e91cc39dabd360797f/pkg/analysis_server/lib/src/context_manager.dart#L415-L437

@bwilkerson @scheglov I'm interested in your thoughts (if it might not be a trivial change and I should move this to the SDK repo for tracking, let me know).

@bwilkerson
Copy link

I think that @scheglov did some work on this fairly recently. Is this reproducible on master?

If so, then it seems reasonable to me to validate that nothing changes after the files were read the first time, but it will have a performance impact as a result of reading files twice. It's probably not too concerning given that this only happens when we're re-building contexts, which is presumably a low frequency event.

I don't know whether it's significant, but I noticed that the file on disk doesn't contain a packageUri key for flutter_gen. Is it possible that server is just discarding that element as a result of thinking that it's malformed (and that it isn't a timing issue)?

@DanTup
Copy link
Member

DanTup commented Jul 8, 2021

I think that @scheglov did some work on this fairly recently. Is this reproducible on master?

Yep, I was using Flutter master from today, and when running the server from source it was change dart-lang/sdk@7748735 from 23hrs ago.

I don't know whether it's significant, but I noticed that the file on disk doesn't contain a packageUri key for flutter_gen. Is it possible that server is just discarding that element as a result of thinking that it's malformed (and that it isn't a timing issue)?

Unfortunately not - the second log I posted was from me opening the file myself (after Flutter had added it), but the first one was the raw contents of the file printed directly before being parsed:

Screenshot 2021-07-08 at 16 30 39

It also only occurred about 1 in 10-15 attempts for me today (I have to keep running flutter pub get until it triggers), which seems to add up - if Flutter usually writes the file fast enough, there's no issue.

(Edit: I checked the server code and it does seem to also handle missing packageUris - https://github.com/dart-lang/sdk/blob/82ea8416ed2a22e40c5c67252cb37bebec3a57e8/pkg/analyzer/lib/src/context/package_config_json.dart#L183 during parsing correctly, using rootUri if it's not present)

@scheglov
Copy link

scheglov commented Jul 8, 2021

Yes, there are holes in the way watch for files. We first read configuration files, such as package_config.json, while creating analysis contexts, and only then start watching for files. Not sure what to do with this though. An obvious solution is to start watching before creating analysis contexts, but I had bad experience with this last time. There were either no events, or at wrong time.

I will give it a try one more time.

@scheglov
Copy link

scheglov commented Jul 8, 2021

I still see that package:watcher does not work.

If I do this, and change pubspec.yaml quickly (note that I disabled a previous work around), I get watch events with id 0 (created some time ago), but not for id 1, created just a few (1625785677678 - 1625785676247 = 1431) milliseconds ago.

This is on Mac, so I guess we should read DirectoryWatcher and MacOSDirectoryWatcher.

1625785654169 [_createAnalysisContexts][creatingAnalysisContextId: 0]
1625785655903 +++ Analyzing.
1625785668128 --- Analyzing in 12224 ms.
1625785676124 +++ Analyzing.
1625785676139 --- Analyzing in 13 ms.
1625785676247 [_createAnalysisContexts][creatingAnalysisContextId: 1]
1625785677669 +++ Analyzing.
1625785677678 	[_createAnalysisContexts][creatingAnalysisContextId: 0][event: modify /Users/scheglov/Source/Dart/sdk.git/sdk/pkg/analyzer/pubspec.yaml]
1625785677678 	[_createAnalysisContexts][creatingAnalysisContextId: 0][event: modify /Users/scheglov/Source/Dart/sdk.git/sdk/pkg/analyzer/.idea/workspace.xml]
1625785683455 --- Analyzing in 5785 ms.

If I add some hack to package:watcher and see when it discards events, we can see that it actually happens.

1625786764845 +++ Analyzing.
1625786764854 --- Analyzing in 9 ms.
1625786764999 [_createAnalysisContexts][creatingAnalysisContextId: 1]
1625786764999 [_MacOSDirectoryWatcher][new]
<cut>
1625786765966 [_MacOSDirectoryWatcher][new]
1625786766287 +++ Analyzing.
1625786766296 	[_createAnalysisContexts][creatingAnalysisContextId: 0][event: modify /Users/scheglov/Source/Dart/sdk.git/sdk/pkg/analyzer/pubspec.yaml]
1625786766296 	[_createAnalysisContexts][creatingAnalysisContextId: 0][event: modify /Users/scheglov/Source/Dart/sdk.git/sdk/pkg/analyzer/.idea/workspace.xml]
1625786766301 	[_createAnalysisContexts][creatingAnalysisContextId: 0][event: modify /Users/scheglov/Source/Dart/sdk.git/sdk/pkg/analyzer/pubspec.yaml]
1625786766301 	[_createAnalysisContexts][creatingAnalysisContextId: 0][event: modify /Users/scheglov/Source/Dart/sdk.git/sdk/pkg/analyzer/.idea/workspace.xml]
1625786766301 	[_MacOSDirectoryWatcher][!isReady]
1625786766312 	[_MacOSDirectoryWatcher][listDir.done][ready]
<cut>
1625786766575 	[_MacOSDirectoryWatcher][listDir.done][ready]
1625786772415 --- Analyzing in 6127 ms.

@scheglov
Copy link

scheglov commented Jul 8, 2021

I opened dart-lang/watcher#115, but don't have much hope that it will be fixed.

@DanTup
Copy link
Member

DanTup commented Jul 9, 2021

Ah, thanks :(

I noticed when looking at your change the existing _scheduleCollectionConsistencyCheck code that looks it was it was intended to workaround issues like this. It seems to check the pubspec hasn't changed during that period:

https://github.com/dart-lang/sdk/blob/ccb3943e48524a993b6ca6e91cc39dabd360797f/pkg/analyzer/lib/src/workspace/pub.dart#L44

Could extending that to include the package file too handle this? (In this case the pubspec has not changed, but .dart_tool/package_config.json did).

@scheglov
Copy link

scheglov commented Jul 9, 2021

Well, it is just a general matter - if package:watcher cannot be trusted, if it can fail to deliver events, then when how much can we do? And when to check for the inconsistencies? After 1 second? 2 seconds? 5 seconds? 1 minute? What if the CPU is very busy?

And it is not only configuration files, even a Dart file could change after we read them, and then if package:watcher does not notify us, we will fail to re-analyze this file and files that depend on it.

@DanTup
Copy link
Member

DanTup commented Jul 12, 2021

And when to check for the inconsistencies? After 1 second? 2 seconds? 5 seconds? 1 minute? What if the CPU is very busy?

Good point - I was thinking about the case where the file had been modified before we created the watcher (so re-reading it from disk immediately in _scheduleCollectionConsistencyCheck would pick up the issue). But I realise now the modification could actually be happening after that, and the event not emitted.

Based on dart-lang/watcher#115 (comment)
it seems like it may be related to this 200ms delay, so maybe there's hope for a fix :-)

@DanTup DanTup added this to the On Deck milestone Jul 12, 2021
@DanTup DanTup added blocked on dart / flutter Requires a change in Dart or Flutter to progress and removed awaiting info Requires more information from the customer to progress labels Jul 12, 2021
@DanTup DanTup changed the title Analyzer hangs, restarts and looses information about localization files Analysis server sometimes fails to resolve flutter_gen package after running "pub get" Jul 12, 2021
@maks
Copy link

maks commented Jul 26, 2021

@DanTup I've read through comments and not sure if I am seeing the same thing, but for me even doing a Restart Analysis Server doesn't get it to recognise the contents of .dart_tool\flutter_gen\gen_l10n which is where the generated app_localizations.dart lives for me, so I keep needing to add import 'package:flutter_gen/gen_l10n/app_localizations.dart'; by hand instead of the usual CTRL+. and then getting the usual popup with option to "Add library package:...` in every file I want to add usage of the localisation generated code.

@DanTup
Copy link
Member

DanTup commented Jul 26, 2021

@maks if adding the import manually works then I don't think it's this issue (this issue prevents the analysis server from being able to resolve package:flutter_gen to a location on disk so would report uri_not_found errors on the import).

I just tried with flutter_gallery and I also don't get quick-fixes to import package:flutter_gen/gen_l10n/gallery_localizations.dart. IIRC, completion/quick-fix usually only suggest from packages that are explicitly listed in pubspec.yaml (eg. not transitive dependencies) so I wonder if that's why this doesn't work here. I'm not sure what the fix is, but it's probably worth filing an SDK issue about that.

@orestesgaolin
Copy link
Author

Just wanted to update that after recent update of VS Code extension v3.26.0 and bumping Flutter to 2.5.0-5.2.pre it still happens the same way as before :/

@felix-ht
Copy link

A hack that works for me - i pinned .dart_tool/package_config.json in vscode. Whenever the error happens i just save the file and the errors disappear. No need to reload the windows any longer.

Would still be nice if a proper fix would land soon.

@orestesgaolin
Copy link
Author

A hack that works for me - i pinned .dart_tool/package_config.json in vscode. Whenever the error happens i just save the file and the errors disappear. No need to reload the windows any longer.

Thanks, seems to do the job! The issue still persists on Flutter 2.8.0 and Flutter extension version 3.29.0

@felix-ht
Copy link

felix-ht commented Dec 10, 2021

Somehow this got way worse for me, so i wrote a simple script that watches '.dart_tool/package_config.json' and resets the modified date after the file has becomes stable.

This has no external dependencies and you can simply copy and paste it into a .dart file and run it with dart.
e.g.: dart refresher.dart

It expects to be run from within the root folder of your flutter project. (tested only on mac)

import 'dart:io';
import 'package:path/path.dart' as path;

void main() async {
  final file = File(path.join('.dart_tool', 'package_config.json'));
  final events = file.watch();
  var lastModifiedByThis = DateTime.fromMicrosecondsSinceEpoch(0);
  events.timeout(const Duration(seconds: 2), onTimeout: (s) {
    if (lastModifiedByThis.isBefore(file.lastModifiedSync())) {
      lastModifiedByThis = DateTime.now();

      // ignore: avoid_print
      print('refreshing package_config');
      file.setLastModified(lastModifiedByThis);
    }
  }).listen((_) {});
}

As a file for convenience:
refresher.dart.zip

@DanTup DanTup modified the milestones: v3.32.0, v3.34.0 Dec 29, 2021
copybara-service bot pushed a commit to dart-lang/sdk that referenced this issue Jan 15, 2022
…creation and watchers being ready

See Dart-Code/Dart-Code#3438.

Change-Id: I3ca038d8c878b118a465b6f230c62912d63990eb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/222763
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Konstantin Shcheglov <scheglov@google.com>
@DanTup DanTup added in commands Relates to commands (usually invoked from the command Palette) and removed blocked on dart / flutter Requires a change in Dart or Flutter to progress labels Jan 17, 2022
@DanTup
Copy link
Member

DanTup commented Jan 17, 2022

Sorry for the slow progress on this, it was a little involved. I believe the issue to be fixed by dart-lang/sdk@650b962, so once that gets into an SDK release you should no longer see this.

(If you do, please let me know!).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in commands Relates to commands (usually invoked from the command Palette) is bug
Projects
None yet
Development

No branches or pull requests

7 participants