Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DartAnalyzer stops working from time to time / autocompletion in IntelliJ fails #24812

Closed
MikeMitterer opened this issue Nov 3, 2015 · 60 comments
Assignees
Labels
area-analyzer Use area-analyzer for Dart analyzer issues, including the analysis server and code completion. P1 A high priority bug; for example, a single project is unusable or has many test failures
Milestone

Comments

@MikeMitterer
Copy link

I know the subject sound weird but thats how it is. The IDEA Team is really helpful but I'm almost sure this is not their fault but it's a "Analyzer Server problem" https://youtrack.jetbrains.com/issue/WEB-18020

THIS REALLY DRIVES ME CRAZY
I know my project is big and has quite a few modules but Dart is almost 4 years old, I'm working with one of the best IDEs and at the moment it's like working with Notepad+syntax highlighting.

It costs me about one hour per day restarting the IDE, waiting for DartAnalyzer, restarting again, writing ISSUE-reports and so on. It's like being in a Beta-Program...

@MikeMitterer
Copy link
Author

As you can see - no problem for cmdline dartanalyzer but the IDE is not even able to resolve the path!
It would be a workaround if Dart-Analysis-Server would start to rescan the current file but it does nothing. And there is no way to trigger the analysis!!!

screenshot-1894

@MikeMitterer
Copy link
Author

Another point is that Analysis-Server doesn't cache anything - it always starts from zero. In my case it takes about 30 to 40mins until it has scanned my project.

@MikeMitterer
Copy link
Author

I took the "turn it off / turn it on" approach and restarted the IDE. After a while autocompletion works again. Maybe I ran into this problem: https://youtrack.jetbrains.com/issue/WEB-18833 - don't know but as said - it's very frustrating at the moment to work with Dart/IntelliJ

@alexander-doroshko
Copy link

According to the screenshot semantic highlighting works in the file on the right side. It means that you didn't have WEB-18833 problem, otherwise you'd have nothing at all working for the whole project (and no 'Dart Analysis' progress on startup).

@kevmoo kevmoo added the area-analyzer Use area-analyzer for Dart analyzer issues, including the analysis server and code completion. label Nov 3, 2015
@kevmoo
Copy link
Member

kevmoo commented Nov 3, 2015

@MikeMitterer What version of Dart are you on? 1.13 dev, right? Would you please include the full output of dart --version.

We've made some BIG fixes in the last couple of dev releases – 1.13.0-dev.7.5 and 1.13.0-dev.7.6

@MikeMitterer
Copy link
Author

Dart VM version: 1.13.0-dev.7.6 (Fri Oct 30 15:14:38 2015) on "macos_x64"

@bwilkerson bwilkerson added Type-Defect P1 A high priority bug; for example, a single project is unusable or has many test failures labels Nov 3, 2015
@scheglov
Copy link
Contributor

scheglov commented Nov 3, 2015

What specifically does not work?
Code completion?
Can you name one more feature that suddenly stops working?

We need to have more information.
Dart Analysis Server has so named "instrumentation".
Let's see if it can provide any hints for us.

You can enable it in IDEA by following these steps:

  1. image
  2. image

image

  1. Add --instrumentation-log-file=/Users/scheglov/tmp/das_log.txt to the end.

  2. Restart and start working.

  3. When you see a problem, try to use the feature that does not work several times, archive the log file and attach / send it.

    NB the file is really huge.
    NB the file will include complete contents of all the files you edit. If you edit any source you don't want to make public do not attach the file to the issue. Send it directly to scheglov@google.com (me).

@MikeMitterer
Copy link
Author

As I am aware of it's "just" code completion. Here it's almost 11pm. I'll send the log tomorrow. Thanks so far!

@MikeMitterer
Copy link
Author

Mail is on the way to you - please keep me informed what you found out.

@scheglov
Copy link
Contributor

scheglov commented Nov 4, 2015

Unfortunately the logs are all after restart on IDEA, so we don't see the reason WHY it was started.
There must be an exception in DAS.
Could you please check the IDEA's Event Log?

@scheglov
Copy link
Contributor

scheglov commented Nov 4, 2015

I also noticed that DAS uses a lot of heap.

Unfortunately until the VM team (@johnmccutchan @iposva-google @rmacnak-google) fixes #24749 we're basically blind and cannot analyze heap problems we don't see on our computers.

@MikeMitterer
Copy link
Author

OK, I've added #com.jetbrains.lang.dart.analyzer to Help | Configure Debug Log Settings and restarted the IDE...

@alexander-doroshko
Copy link

According to Mike's screenshot code completion is not the only thing that doesn't work for the file on the left side. Navigation to declaration doesn't work as well. I guess for each file either all or none DAS-driven features work.

@Pacane
Copy link

Pacane commented Nov 7, 2015

@MikeMitterer Hey, I've got this issue as well. It used to drive me crazy as well. But now instead of restarting intellij, if you have the latest version of the plugin, you can click on the "Reanalyze sources" button in the analyzer tab. This will get your autocompletion + analyzing stuff to work again.

Hope this helps you

image

Although, this is just a band-aid on a real problem. The analyzer keeps crashing for some reason, and it has been happening for several releases to me.

For documentation's sake, I'm using Arch Linux x64, with IntelliJ 15 (used to happen on 14 as well)
My Dart SDK version has always been on dev, but it also did it on stable.
Current version

Dart VM version: 1.13.0-dev.7.10 (Fri Nov 6 02:03:14 2015) on "linux_x64"

@mit-mit mit-mit added this to the 1.13 milestone Nov 9, 2015
@mit-mit
Copy link
Member

mit-mit commented Nov 9, 2015

Tagging as an 1.13 candidate.

@scheglov
Copy link
Contributor

scheglov commented Nov 9, 2015

@MikeMitterer sent the log that shows the following exceptions.
DAS stops sending navigation and completion information after these exceptions.

Exceptions
Unexpected exception while performing ComputeConstantValueTask for element dynamic _BYTES_PER_WORD in
source /Users/mikemitterer/.pub-cache/hosted/pub.dartlang.org/crypto-0.9.0/lib/src/hash_utils.dart
#0 AnalysisTask._safelyPerform (package:analyzer/task/model.dart:320)
#1 AnalysisTask.perform (package:analyzer/task/model.dart:210)
#2 AnalysisDriver.performWorkItem (package:analyzer/src/task/driver.dart:271)
#3 AnalysisDriver.performAnalysisTask (package:analyzer/src/task/driver.dart:246)
#4 AnalysisContextImpl.performAnalysisTask. (package:analyzer/src/context/context.dart:1098)
#5 _PerformanceTagImpl.makeCurrentWhile (package:analyzer/src/generated/utilities_general.dart:170)
#6 AnalysisContextImpl.performAnalysisTask (package:analyzer/src/context/context.dart:1096)
#7 PerformAnalysisOperation.perform (package:analysis_server/src/operation/operation_analysis.dart:366)
#8 AnalysisServer.performOperation (package:analysis_server/src/analysis_server.dart:793)
#9 Future.Future. (dart:async/future.dart:118)
#10 _rootRun (dart:async/zone.dart:903)
#11 _CustomZone.run (dart:async/zone.dart:802)
#12 _CustomZone.runGuarded (dart:async/zone.dart:708)
#13 _CustomZone.bindCallback. (dart:async/zone.dart:733)
#14 _rootRun (dart:async/zone.dart:907)
#15 _CustomZone.run (dart:async/zone.dart:802)
#16 _CustomZone.runGuarded (dart:async/zone.dart:708)
#17 _CustomZone.bindCallback. (dart:async/zone.dart:733)
#18 Timer._createTimer. (dart:async-patch/timer_patch.dart:16)
#19 _Timer._runTimers (dart:isolate-patch/timer_impl.dart:385)
#20 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:150)
Caused by The null object does not have a getter 'declaredVariables'. NoSuchMethodError: method not found:
'declaredVariables' Receiver: null Arguments: []
#0 Object._noSuchMethod (dart:core-patch/object_patch.dart:42)
#1 Object.noSuchMethod (dart:core-patch/object_patch.dart:45)
#2 ComputeConstantValueTask.internalPerform (package:analyzer/src/task/dart.dart:1705)
#3 AnalysisTask._safelyPerform (package:analyzer/task/model.dart:310)
#4 AnalysisTask.perform (package:analyzer/task/model.dart:210)
#5 AnalysisDriver.performWorkItem (package:analyzer/src/task/driver.dart:271)
#6 AnalysisDriver.performAnalysisTask (package:analyzer/src/task/driver.dart:246)
#7 AnalysisContextImpl.performAnalysisTask. (package:analyzer/src/context/context.dart:1098)
#8 _PerformanceTagImpl.makeCurrentWhile (package:analyzer/src/generated/utilities_general.dart:170)
#9 AnalysisContextImpl.performAnalysisTask (package:analyzer/src/context/context.dart:1096)
#10 PerformAnalysisOperation.perform (package:analysis_server/src/operation/operation_analysis.dart:366)
#11 AnalysisServer.performOperation (package:analysis_server/src/analysis_server.dart:793)
#12 Future.Future. (dart:async/future.dart:118)
#13 _rootRun (dart:async/zone.dart:903)
#14 _CustomZone.run (dart:async/zone.dart:802)
#15 _CustomZone.runGuarded (dart:async/zone.dart:708)
#16 _CustomZone.bindCallback. (dart:async/zone.dart:733)
#17 _rootRun (dart:async/zone.dart:907)
#18 _CustomZone.run (dart:async/zone.dart:802)
#19 _CustomZone.runGuarded (dart:async/zone.dart:708)
#20 _CustomZone.bindCallback. (dart:async/zone.dart:733)
#21 Timer._createTimer. (dart:async-patch/timer_patch.dart:16)
#22 _Timer._runTimers (dart:isolate-patch/timer_impl.dart:385)
#23 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:150)


Unexpected exception while performing VerifyUnitTask for source /Users/mikemitterer/.pub-
cache/hosted/pub.dartlang.org/intl-0.11.12/lib/src/intl/bidi_formatter.dart
#0 AnalysisTask._safelyPerform (package:analyzer/task/model.dart:320)
#1 AnalysisTask.perform (package:analyzer/task/model.dart:210)
#2 AnalysisDriver.performWorkItem (package:analyzer/src/task/driver.dart:271)
#3 AnalysisDriver.performAnalysisTask (package:analyzer/src/task/driver.dart:246)
#4 AnalysisContextImpl.performAnalysisTask. (package:analyzer/src/context/context.dart:1098)
#5 _PerformanceTagImpl.makeCurrentWhile (package:analyzer/src/generated/utilities_general.dart:170)
#6 AnalysisContextImpl.performAnalysisTask (package:analyzer/src/context/context.dart:1096)
#7 PerformAnalysisOperation.perform (package:analysis_server/src/operation/operation_analysis.dart:366)
#8 AnalysisServer.performOperation (package:analysis_server/src/analysis_server.dart:793)
#9 Future.Future. (dart:async/future.dart:118)
#10 _rootRun (dart:async/zone.dart:903)
#11 _CustomZone.run (dart:async/zone.dart:802)
#12 _CustomZone.runGuarded (dart:async/zone.dart:708)
#13 _CustomZone.bindCallback. (dart:async/zone.dart:733)
#14 _rootRun (dart:async/zone.dart:907)
#15 _CustomZone.run (dart:async/zone.dart:802)
#16 _CustomZone.runGuarded (dart:async/zone.dart:708)
#17 _CustomZone.bindCallback. (dart:async/zone.dart:733)
#18 Timer._createTimer. (dart:async-patch/timer_patch.dart:16)
#19 _Timer._runTimers (dart:isolate-patch/timer_impl.dart:385)
#20 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:150)
Caused by The null object does not have a getter 'context'. NoSuchMethodError: method not found: 'context'
Receiver: null Arguments: []
#0 Object._noSuchMethod (dart:core-patch/object_patch.dart:42)
#1 Object.noSuchMethod (dart:core-patch/object_patch.dart:45)
#2 ConstantVerifier.ConstantVerifier (package:analyzer/src/generated/resolver.dart:896)
#3 VerifyUnitTask.internalPerform (package:analyzer/src/task/dart.dart:4603)
#4 AnalysisTask._safelyPerform (package:analyzer/task/model.dart:310)
#5 AnalysisTask.perform (package:analyzer/task/model.dart:210)
#6 AnalysisDriver.performWorkItem (package:analyzer/src/task/driver.dart:271)
#7 AnalysisDriver.performAnalysisTask (package:analyzer/src/task/driver.dart:246)
#8 AnalysisContextImpl.performAnalysisTask. (package:analyzer/src/context/context.dart:1098)
#9 _PerformanceTagImpl.makeCurrentWhile (package:analyzer/src/generated/utilities_general.dart:170)
#10 AnalysisContextImpl.performAnalysisTask (package:analyzer/src/context/context.dart:1096)
#11 PerformAnalysisOperation.perform (package:analysis_server/src/operation/operation_analysis.dart:366)
#12 AnalysisServer.performOperation (package:analysis_server/src/analysis_server.dart:793)
#13 Future.Future. (dart:async/future.dart:118)
#14 _rootRun (dart:async/zone.dart:903)
#15 _CustomZone.run (dart:async/zone.dart:802)
#16 _CustomZone.runGuarded (dart:async/zone.dart:708)
#17 _CustomZone.bindCallback. (dart:async/zone.dart:733)
#18 _rootRun (dart:async/zone.dart:907)
#19 _CustomZone.run (dart:async/zone.dart:802)
#20 _CustomZone.runGuarded (dart:async/zone.dart:708)
#21 _CustomZone.bindCallback. (dart:async/zone.dart:733)
#22 Timer._createTimer. (dart:async-patch/timer_patch.dart:16)
#23 _Timer._runTimers (dart:isolate-patch/timer_impl.dart:385)
#24 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:150)

@scheglov
Copy link
Contributor

scheglov commented Nov 9, 2015

It seems that it both cases we have a CompilationUnitElement which is not attached to its LibraryElement.

We attempt to create a new instance of ConstantVerifier at

ConstantVerifier constantVerifier = new ConstantVerifier(
and pass in null as libraryElement. So unitElement.library is null. So, this UnitElement does not have an enclosing LibraryElement.

@whesse
Copy link
Member

whesse commented Nov 11, 2015

Is this a blocking issue for releasing 1.13 stable? Do we have an estimate on a fix for this? Until we hear otherwise, we can't release 1.13.

@dgrove
Copy link
Contributor

dgrove commented Nov 12, 2015

Any updates on this issue?

@bwilkerson
Copy link
Member

We have not been able to reproduce the exceptions, but we did find one problem that might account for part of the slowness being reported by our users.

@MikeMitterer a couple of questions. Have you re-run pub so that there are .packages files and/or is server still running pub list-package-dirs? Also, do you have large numbers of HTML files in the doc directory (such as are produced by running dartdoc)?

@MikeMitterer
Copy link
Author

Almost all of my projects have packages AND .packages. Most of my doc-folders are empty.

@scheglov
Copy link
Contributor

scheglov added a commit that referenced this issue Nov 12, 2015
@scheglov
Copy link
Contributor

We have fixed the problem that caused @MikeMitterer exceptions in a21e3a5.
This may or may not be the only problem causing losing completion and navigation.

This requires more testing with this fix applied.

@keertip
Copy link
Contributor

keertip commented Nov 16, 2015

@MikeMitterer , could you try this build http://gsdview.appspot.com/dart-archive/channels/dev/raw/1.13.0-dev.7.12/sdk/. It should fix the issues you were seeing with the -dev.7.11 sdk.

@bwilkerson
Copy link
Member

@MikeMitterer Did you try deleting the packages directories? If so, was there any change in performance?

@MikeMitterer
Copy link
Author

@keertip Thanks I'll give it a try tomorrow!

@bwilkerson I can't delete all the packages. I thought you gonna come up with a final solution for the package problem - removing all of them is not the best one... what, and in my case that's the case, if one package referenz assets from another package? Then you need at least one package-folder in web or in project-root...

To be more precise:
Something like this:

sample.scss in dart-application X:

@import "packages/mdl/assets/styles/animation/animation";

@bwilkerson
Copy link
Member

Sorry, I guess I was unclear. The pub command currently produces a packages directory unless you explicitly ask it not to (using the --no-package-symlinks argument). It also creates a .packages file. We are moving toward getting rid of the packages directory altogether and using the file in its place. So, if you delete the directory we will still be able to resolve your package: URI's. The concept of packages is not going away, we're just changing how we implement it.

@bwilkerson
Copy link
Member

Oh, I just re-read you question and I think I understand better what you were asking. The VM also understands .package files, and I would guess that it will be able to load assets by using the files rather than the directories.

@munificent Can you confirm this?

@zoechi
Copy link
Contributor

zoechi commented Nov 16, 2015

The sass transformer calls out to the sass command line tool which doesn't know about Dart and the .packages file. If the files aren't actually there it fails.

@bwilkerson
Copy link
Member

The sass transformer ...

That's a problem. My understanding is that both the directory and the file are produced by default in 1.13. In 1.14 and later only the file will be produced by default. In 1.14 you'll still be able to explicitly ask pub to produce the directory, but as of 1.15 even that won't be possible.

I think the transformer will need to be updated to follow the package spec DEP (https://github.com/lrhn/dep-pkgspec).

@bwilkerson
Copy link
Member

I created an issue for the transformer (dart-league/dart-sass#1) in case they were not aware of the changes that are coming.

@zoechi
Copy link
Contributor

zoechi commented Nov 16, 2015

You can update the transformers, but that won't happen for non-Dart related 3rd party tools like the sass tool called by the transformer. That's a known problem. I haven't seen any discussions how to deal with such tools yet though.

@munificent
Copy link
Member

It is possible for the SASS transformer to loop back to barback when it wants to load and resolve @import directives using a custom importer. Ideally, that's what the SASS transformer should be doing.

Otherwise, yes, when the "packages" directory goes away, tools that presume it's there will not be in a happy place. We don't anticipate getting rid of that directory for a while, though, so there should be time to move tools off of it.

@MikeMitterer
Copy link
Author

I tried the new version with latest IJ Dart plugin and it's really much faster now - Cool! But there are still a few issues:

  • It takes about 4 minutes or so until DA starts to analyze (check my mp4)
  • I crashed the Analyzer by enabling a module while it was analyzing ~ 07:46 in mp4 or 14:50 (error message lower right corner!) It shows the server-version 12 - is this correct?

Here you can see the whole process:
http://www.mikemitterer.at/fileadmin/_temp_/dart_analyzer.mp4

@scheglov I've sent you the log via mail...

[Update]
DA was correct by showing an error message for the "Awesome" class (07:27)

@bwilkerson
Copy link
Member

@MikeMitterer Excellent! I'm glad we're starting to make progress. We'll take a look at the log files and see what we can learn.

@scheglov
Copy link
Contributor

There are two reasons why in your video it too 4 minutes:

  1. Dart Plugin for IntelliJ (DPI) does not start DAS until you open a Dart file.
  2. DAS spends 2.5 minutes doing pub list-package-dirs.

Usually DAS uses .packages files, so avoids calling pub list-package-dirs, so works much faster.
But I suspect that many of pubspec.yaml files are in templates, so they have invalid name: and in this case .packages file cannot be generated. Unfortunately unless we validate pubspec.yaml files in DAS, we don't know whether the file is valid or not. And Pub generates pubspec.lock anyway, so we cannot use its absence as an indicator, that the package should not be analyzed.

The easy solution would be to exclude these templates from analysis.
We will talk to the Pub authors to see if something can be done to optimize.

analyzer:
  exclude:
    - 'samples/**'
    - 'lib/_templates/**'

The crash (actually not a crash - just a non-fatal exception) must be easy to fix.

@scheglov
Copy link
Contributor

The fix for the exception.
https://codereview.chromium.org/1445283003/

@MikeMitterer
Copy link
Author

Sorry - but I have another one.
I few days ago I've moved all the MDL/Dart samples to https://goo.gl/UM9hsN
As you can see in the SS DA ist not able to resolve an Object defined in it's own lib LoginDialog
And again - it's DA-Server. The cmdline-version works.

[Update]
I forgot to mention that I have a .analysis_options file for this module. I wasn't sure if this is already implemented or if this was just a proposal - so I tried it out

analyzer:
  exclude:
    - 'samples/**'

screenshot-1934

@bwilkerson
Copy link
Member

Sorry - but I have another one.

We're not. You're being extremely helpful, and we're grateful.

What was the error message associated with those two uses of the class name?
Which library defines LoginDialog?

@scheglov
Copy link
Contributor

@MikeMitterer, you see the error because your samples folder is excluded from analysis by the .analysis_options file in the root of the dart-material-design-lite-site project.

analyzer:
  exclude:
    - 'samples/**'

This mean that when you open a Dart file in a sample, it is analyzed as an independent file, not a part of the template_spa package, just because there are no such analysis context from the DAS point of view. Note, that if you open the template_spa separately, it also works in DAS, just like it works with dartanalyzer.

@zoechi
Copy link
Contributor

zoechi commented Nov 18, 2015

Why is it analyzed at all, when it is excluded?

@scheglov
Copy link
Contributor

Good point, @zoechi.
We will fix it.

@mit-mit
Copy link
Member

mit-mit commented Nov 18, 2015

@zoechi, @scheglov, I would like to move the tracking of that .analysis_options problem to a new github issue. We are getting ready to ship 1.13, and it does not sound like that new issue is a blocker.

Let me know if that is OK. Thanks!

@scheglov
Copy link
Contributor

Yes, a separate issue for exclude handling in .analysis_options would be nice.

@zoechi
Copy link
Contributor

zoechi commented Nov 18, 2015

I worked quite some time and can confirm that without the packages folders it works quite well. I would see #24910 as blocker. Otherwise I'm fine.

@mit-mit
Copy link
Member

mit-mit commented Nov 19, 2015

Closing this bug as 1.13 was promoted to stable.

@zoechi, we will follow up on #24910 in that issue.

@mit-mit mit-mit closed this as completed Nov 19, 2015
@MikeMitterer
Copy link
Author

@scheglov It's to bad but there is still somewhere a bug in how IJ or DA handles analysis / reanalyzing.
I've added to files mdlanimation.dart and animation.dart and as you can see here www.mikemitterer.at/fileadmin/_temp_/da_bug.mp4 I had no chance to reanimate DA. I clicked around tried several thing - nothing happened! The weird this is that when I stopped the recording of the screencast, maybe because the app focus changed, DA started to analyze...

I've sent you the log via mail
@mit-mit I think this is still open

Here the SS after I stopped recording - (DA is still analyzing...)
screenshot-1936

@mit-mit
Copy link
Member

mit-mit commented Nov 19, 2015

@MikeMitterer, can you please open a new issue(s) to track any remaining problems? I'd like to close this one out so that we can easier tell what was resolved, and what is still not working.

@scheglov
Copy link
Contributor

@MikeMitterer It might be the same problem as before - we spend about 150 seconds doing pub list-package-dirs. Until it's done, DAS cannot respond to the client.

This is something we can optimize in DAS after Pub fixes its issue.
dart-lang/pub#1361

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-analyzer Use area-analyzer for Dart analyzer issues, including the analysis server and code completion. P1 A high priority bug; for example, a single project is unusable or has many test failures
Projects
None yet
Development

No branches or pull requests