Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a version header to Object-Streams. #15794

Merged
merged 9 commits into from Dec 10, 2016

Conversation

CyrusNajmabadi
Copy link
Member

@CyrusNajmabadi CyrusNajmabadi commented Dec 9, 2016

This way we can tell, up front if the persistence version has changed, and if we can read the data properly or not.

Fixes #15780

Customer scenario

Customer upgrades from previous version of VS (including Dev14 or an RC build) to RTM. Customer opens an existing project in Dev15. VS crashes. Problem persists until they delete their .vs folder (which they have no idea is the cause).

Bugs this fixes:

Fixes #15780

Workarounds, if any

Customer deletes their .vs folder for every project they're opening in Dev15. This is pretty bad as there is no way for them to know to do this.

Risk

Extremely low. This is just adding a check that our on-disk format is what we expect. If it isn't we know the previous cache isn't safe to use and we build a new one.

Performance impact

Low/None. We're just checking an additional 2-bytes to validate that the disk format is one we expect.

Is this a regression from a previous update?

Yes.

Root cause analysis:

A recent PR changed our on-disk format for persisting data (like Find-References indices). This chnage happened without any augmentation to the format indicating what version it was. So the new code had no way to say "this persisted stream of bytes has the wrong format, we can't read this", and would instead crash.

How was the bug found?

This hit most members of teh roslyn team immediately.

To help avoid this, we're investing more in adding actual unit tests to test upgrade scenarios.

…t if the persistence version has changed, and if we can read the data properly or not.
@CyrusNajmabadi
Copy link
Member Author

CyrusNajmabadi commented Dec 9, 2016

Tagging @jasonmalinowski @tmat @heejaechang @mattwar

This supersedes: #15787

Here we actually store a two-byte version at teh start of the object-stream. That way the reader can tell up front if the persistence format has changed, and can immediately return null.

In cases where we we are only serializing and not persisting across sessions, we assert that this should always succeed. In cases where we persist across sessions, we are now resilient to getting null back.

I've also narrowed our exception checking to only IOException. @heejaechang Does your esent storage layer ever throw any sort of esent specific exceptions? Or does is swallow/translate them?

@heejaechang
Copy link
Contributor

esent doesn't leak out any exception. it catch all exception and return false.

/// this version, just change VersionByte2.
/// </summary>
internal const byte VersionByte1 = 0b10101010;
internal const byte VersionByte2 = 0b00000001;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Binary literals FTW!

return null;
}

return new StreamObjectReader(stream, knownObjects, binder, cancellationToken);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could put the read of the format (recursive, etc) byte here too and have it also return null if it doesn't match one of the known types.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could. But that indicates a true bug. That means the format changed and the version wasn't updated. That's actually a bug, and i want an exceptoin in that case.

{
Debug.Assert(reader != null,
@"We only ge a reader for data transmitted between live processes.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get?

@mattwar
Copy link
Contributor

mattwar commented Dec 9, 2016

👍

@CyrusNajmabadi
Copy link
Member Author

test this please

@CyrusNajmabadi
Copy link
Member Author

esent doesn't leak out any exception. it catch all exception and return false.

Great. That simplifies things enormously.

@CyrusNajmabadi
Copy link
Member Author

@MattGertz For approval. Yesterday i checked in a mitigation for #15780. But this is an actual fix.

Filling out template now.

@CyrusNajmabadi CyrusNajmabadi merged commit 927031b into dotnet:master Dec 10, 2016
@CyrusNajmabadi CyrusNajmabadi deleted the objectStreamVersion branch December 10, 2016 01:08
CyrusNajmabadi added a commit that referenced this pull request Jan 25, 2017
…sion"

This reverts commit 927031b, reversing
changes made to 55ab603.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Crash in serialization.
5 participants