Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove System.IO.Hashing dependency from Nrbf #103700

Merged
merged 3 commits into from
Jun 19, 2024
Merged

Conversation

stephentoub
Copy link
Member

@dotnet-issue-labeler dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Jun 19, 2024
Copy link
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not acceptable, please close the PR.

#endif
return (int)XxHash32.HashToUInt32(MemoryMarshal.AsBytes(integers));
}
public override int GetHashCode() => _id;
Copy link
Member

@adamsitnik adamsitnik Jun 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is done on purpose, to prevent from hashcode collision attacks (the attacker can create a payload with such Ids that would cause OOM in the dictionary)

cc @GrabYourPitchforks

Copy link
Member Author

@stephentoub stephentoub Jun 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no randomization in xxhash32. this is just a deterministic mapping from one integer to another, with the only possibility of creating more collisions. to mitigate such attacks, this should use a random seed, eg System.HashCode.

Copy link
Member

@adamsitnik adamsitnik Jun 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no randomization in xxhash32. this is just a deterministic mapping from one integer to another, with the only possibility of creating more collisions. to mitigate such attacks, this should use a random seed, eg System.HashCode

Thanks, I was not aware of that. I've assumed that it's the right thing to do since it got approved by @bartonjs.

This library needs to target Full Framework and .NET Standard, so we can not use System.HashCode for all TFMs. But... w can use it for #if NET. Would that solve the problem for windowsdesktop repo?

Copy link
Member Author

@stephentoub stephentoub Jun 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so we can not use System.HashCode for all TFMs

We can:
https://www.nuget.org/packages/Microsoft.Bcl.HashCode/

But if the goal here is just to include some randomness with a single integer, I'd assume we can just do something like:

private static readonly int s_seed = new Random().Next(-int.MaxValue, int.MaxValue);

public int GetHashCode() => _id ^ s_seed;

or some other form of incorporation that Jeremy and Levi bless.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can: https://www.nuget.org/packages/Microsoft.Bcl.HashCode/

Nice, I was not aware of the fact that such package exists. It's already used by Cbor, so we should be fine using it?

<PackageReference Include="Microsoft.Bcl.HashCode" Version="$(MicrosoftBclHashCodeVersion)" />

But if the goal here is just to include some randomness with a single integer
or some other form of incorporation that Jeremy and Levi bless.

I would prefer to wait for @bartonjs or @GrabYourPitchforks feedback.

@stephentoub would you mind updating your PR to switch to HashCode type or would you prefer me to do it?

@stephentoub
Copy link
Member Author

@ViktorHofer, does this need any addition to NetCoreAppLibrary.props? Presumably no because the Microsoft.Bcl.HashCode dependency is only downlevel?

Copy link
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you for raising the question, sharing the knowledge and providing the fix!

Copy link
Member

@ViktorHofer ViktorHofer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. This again needs a reaction in windowsdesktop.

Presumably no because the Microsoft.Bcl.HashCode dependency is only downlevel?

Exactly. This change is a net positive for net9.0 (and only that flows to windowsdesktop) as it removes the IO.Hashing dependency without introducing a new one.

@stephentoub
Copy link
Member Author

This again needs a reaction in windowsdesktop.

Is this a typo? I'm confused as based on the second part of your response I thought I didn't need to do anything else in windowsdesktop.

@ViktorHofer
Copy link
Member

ViktorHofer commented Jun 19, 2024

Is this a typo? I'm confused as based on the second part of your response I thought I didn't need to do anything else in windowsdesktop.

Nope, no typo. This PR needs to be updated again to remove the System.IO.Hashing entry: dotnet/windowsdesktop#4459

Reason is, the WindowsDesktop shared framework needs to be closure complete (so it must not have any dangling references) which requires listing all of the assemblies + its references in the sfxprojs.

Now that we remove that dependency again, we need to remove that line from the sfxprojs.

@stephentoub
Copy link
Member Author

Nope, no typo. This PR needs to be updated again to remove the System.IO.Hashing entry: dotnet/windowsdesktop#4459

Oh, I thought you were saying this PR (as in 103700) needs further updates. But you're not saying that; you're saying a subsequent change needs to be made in dotnet/windowsdesktop, right?

@ViktorHofer
Copy link
Member

ViktorHofer commented Jun 19, 2024

Exactly. This PR is ready to go. Sorry for the confusion.

@stephentoub
Copy link
Member Author

Cool, thanks.

@stephentoub
Copy link
Member Author

/ba-g failures are known and unrelated

@stephentoub stephentoub merged commit 063255a into dotnet:main Jun 19, 2024
80 of 83 checks passed
@stephentoub stephentoub deleted the nohash branch June 19, 2024 17:36
@adamsitnik adamsitnik added area-System.Formats.Nrbf binary-serialization and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants