Skip to content

Conversation

geoffw0
Copy link
Contributor

@geoffw0 geoffw0 commented Sep 13, 2024

Add Missing Elements diagnostic query (rust/diagnostics/missing-elements) that lists Unimplemented, MissingExpr and MissingPat elements. At the moment there's rather a lot, even on the simple test - probably mostly coming from the standard libraries. Future extractor changes should greatly reduce this.

@geoffw0 geoffw0 added no-change-note-required This PR does not need a change note Rust Pull requests that update Rust code labels Sep 13, 2024
@geoffw0
Copy link
Contributor Author

geoffw0 commented Sep 13, 2024

Hmm, the locations need cleaning up here...

@geoffw0
Copy link
Contributor Author

geoffw0 commented Sep 13, 2024

OK, we get different results on CI to my run locally. I think this will be a continuous source of problems unless we either (1) restrict results to inside the source location or (2) remove or constrain the test. (1) seems reasonable but it does mean we won't have visibility on problems extracting the standard library.

I'll have another think about it on Monday. Opinions welcome.

@aibaars
Copy link
Contributor

aibaars commented Sep 14, 2024

OK, we get different results on CI to my run locally. I think this will be a continuous source of problems unless we either (1) restrict results to inside the source location or (2) remove or constrain the test. (1) seems reasonable but it does mean we won't have visibility on problems extracting the standard library.

I'll have another think about it on Monday. Opinions welcome.

My other PR no longer extracts files in the standard library or dependencies so that problem should be solved. I added a flag to enable the extraction of libraries but by default we no longer do. Extracting the standard library is too brittle because results depend on whichever version of rust is installed and most likely the platform on which it runs also causes differences.

@aibaars
Copy link
Contributor

aibaars commented Sep 16, 2024

@geoffw0 @redsun82 This PR has a compilation error because class Unimplemented(Element) does not have a getLocation method. We could either change the schema to have class Unimplemented(Locatable) or add a predicate like

Location getLocation(Unimplemented node) {
   result = node.(Locatable).getLocation() or 
   result instanceof UnknownLocation and not node instanceof Locatable}
}

to this PR. Not sure what is best, I'm fine with either approach.

@geoffw0
Copy link
Contributor Author

geoffw0 commented Sep 16, 2024

My other PR no longer extracts files in the standard library or dependencies so that problem should be solved. I added a flag to enable the extraction of libraries but by default we no longer do. Extracting the standard library is too brittle because results depend on whichever version of rust is installed and most likely the platform on which it runs also causes differences.

Does that mean we'll have difficulty assigning targets for calls to standard library functions? Or are we extracting some stuff in the standard libraries still?

c = strictcount(Unimplemented n | cleanLocationString(getUnimplementedLocation(n)) = location) and
c =
strictcount(Unimplemented n |
exists(getUnimplementedLocation(n).getFile().getRelativePath()) and
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This excludes elements from user code for which the extractor did not know a location.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But if we show things with an EmptyLocation, that will include things from outside user code and we'll get inconsistent results. It's also of dubious value if we don't know where to look for the problem.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The MissingElement query is a bit special. It's purpose is to measure the quality of the database/extractor. If we throw out all MissingElements without location then we might think a database is in perfect health while in reality it is not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... and we're committed to not extracting non-user code now as I understand it (still now 100% sure how that works). I'll try including them again.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've just reverted the restriction to user code in this PR as well. I'm fairly sure I was seeing inconsistencies with it though, we will see...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That reduces me to "Not yet implemented (x17)" locally.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you comment out theprintln! in my_macro.rs as well as main.rs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... oops ...

Yep, that gets me down to 14, which is presumably in line with CI now. I've pushed the fix. There is still an issue in the extractor presumably, but this PR should be mergeable now I think.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, yes there must be some issue, or perhaps the standard library for rust on the CI runners has a different implementation for println!.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, that is unfortunate. But maybe we just need to do a https://github.com/actions-rust-lang/setup-rust-toolchain step for running the QL tests?

redsun82
redsun82 previously approved these changes Sep 16, 2024
Copy link
Contributor

@redsun82 redsun82 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with this going in as it is, it will be valuable for us!

Can you open an issue to track the problem with println! in tests leading to differences in extraction?

@redsun82
Copy link
Contributor

@geoffw0 maybe just a naming nit: if we go with #17479, then even though the query code itself can remain as it is (after all it's useful to get the breakdown into the separate cases), the name of the query might be better using unextracted rather than missing for consistency.

@geoffw0
Copy link
Contributor Author

geoffw0 commented Sep 16, 2024

Can you open an issue to track the problem with println! in tests leading to differences in extraction?

Done (provisionally assigned to you).

the name of the query might be better using unextracted rather than missing for consistency.

Will do...

@geoffw0
Copy link
Contributor Author

geoffw0 commented Sep 16, 2024

the name of the query might be better using unextracted rather than missing for consistency.

Done.

@geoffw0 geoffw0 merged commit 27dca74 into github:main Sep 16, 2024
14 checks passed
@geoffw0 geoffw0 deleted the missing branch October 15, 2024 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no-change-note-required This PR does not need a change note Rust Pull requests that update Rust code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants