Skip to content

Conversation

@auduchinok
Copy link
Member

@auduchinok auduchinok commented Jan 4, 2022

Fixes remaining known repro of #11291.

We've debugged it with @TIHan and discovered that stale results could be cached in the service and returned without checking timestamps of prior files in the project. When some prior file is changed and other files in the project are checked afterwards, a file with previously cached results may return stale results. This PR forces checking prior timestamps up to the file that has cached results.

@auduchinok auduchinok force-pushed the service-cachedResults-logicalTimestamp branch 3 times, most recently from 8c65ab7 to 02779ce Compare January 4, 2022 21:08
@auduchinok auduchinok force-pushed the service-cachedResults-logicalTimestamp branch from 02779ce to abd6a87 Compare January 5, 2022 11:01
@dsyme
Copy link
Contributor

dsyme commented Jan 13, 2022

@TIHan Would you have a chance to review this please

@auduchinok Is there an automated test we can be adding for this?

@auduchinok
Copy link
Member Author

@auduchinok Is there an automated test we can be adding for this?

@dsyme Nope, unfortunately. We could only reliably reproduce it in a bigger project, using the file system shim and accessing results concurrently.

}

let MaxTimeStampInDependencies stamps =
let MaxTimeStampInDependencies stamps fileSlot =
Copy link
Contributor

@TIHan TIHan Jan 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passing -1 is a little weird since we have operated on 0 and above for file slots. But I understand why, as -1 means look at all the stamps and anything greater than that you want to only look at that many file stamps. So maybe fileSlot could be renamed to fileCount and add a comment explaining what happens when -1 is passed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, will do.

Copy link
Contributor

@TIHan TIHan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is more or less what @auduchinok and I talked about last time, except for the changes in MaxTimeStampInDependencies.

Creating a test for this is challenging as we cannot seem to create a simple repo, it only happens in more complex projects.

I'm confident this will fix the issue, though it would mean that we will be computing timestamps for files on every call since we must compute the logical timestamp every time. A call to computeStampedFileNames looks at all files for their last write time. This might affect performance, so I advise a little bit of caution regarding this change.

@vzarytovskii
Copy link
Member

vzarytovskii commented Jan 14, 2022

As @TIHan said, it may affect performance.

Will it be possible to measure latency - now vs with this change?

@dsyme
Copy link
Contributor

dsyme commented Jan 21, 2022

This might affect performance, so I advise a little bit of caution regarding this change.

Given this comment, we need some kind of measure on scenarios that may be of concern

@TIHan @auduchinok Could you iterate on how we can resolve this? Either through an adhoc manual test, or measured manual test?

@TIHan
Copy link
Contributor

TIHan commented Feb 1, 2022

@dsyme We could do a measured test in our benchmarkdotnet tests. It could look something like this (pseudo):

Setup() =
    options <- .. // FSharpProjectOptions could have as an example, 1000 files.
    checker.ParseAndCheckProject(options) // check entire project so it builds up a cache
    
Execute() =
    // Cause all files to have their timestamps re-evaluated
    let updatedFirstFileContents = .. // updated contents of the first source file, content should be different with each execution
    checker.CheckFileInProject("first_file.fs", updatedFirstFileContents, options) // check first file in the project with the new contents
    checker.CheckFileInProject("last_file.fs", sameLastFileContents, options) // check last file in the project

@KevinRansom
Copy link
Contributor

@auduchinok can you offer some insight about how we can verify performance impact to this change.

@dsyme
Copy link
Contributor

dsyme commented May 10, 2022

I talked with @vzarytovskii - we will take this change

@vzarytovskii vzarytovskii enabled auto-merge (squash) May 10, 2022 11:38
@vzarytovskii vzarytovskii merged commit b9556cc into dotnet:main May 11, 2022
charlesroddie pushed a commit to charlesroddie/fsharp that referenced this pull request May 2, 2023
…otnet#12570)

* Service: check project logical timestamp when getting cached results

* Calculate logical timestamp up to a file only

* don't check source files timestamps before first file in project

* format

* format

* Fix after-build conflicts

Co-authored-by: Don Syme <donsyme@fastmail.fm>
Co-authored-by: Vlad Zarytovskii <vzaritovsky@hotmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants