Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

YamlDotNot is not thread-safe #1536

Conversation

nathanwoctopusdeploy
Copy link
Contributor

@nathanwoctopusdeploy nathanwoctopusdeploy commented Mar 5, 2024

This PR demonstrates that KubernetesClientConfiguration.LoadKubeConfig is not thread-safe due to YamlDotNot Deserializers.

   Exception during deserialization
   
   at YamlDotNet.Serialization.ValueDeserializers.NodeValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)
   at YamlDotNet.Serialization.ValueDeserializers.AliasValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)
   at YamlDotNet.Serialization.Deserializer.Deserialize(IParser parser, Type type)
   at YamlDotNet.Serialization.Deserializer.Deserialize[T](IParser parser)
   at k8s.KubernetesYaml.Deserialize[TValue](String yaml, Boolean strict) in F:\Code\kubernetes-client\src\KubernetesClient\KubernetesYaml.cs:line 207
   at k8s.KubernetesYaml.<LoadFromStreamAsync>d__11`1.MoveNext() in F:\Code\kubernetes-client\src\KubernetesClient\KubernetesYaml.cs:line 181
   at k8s.KubernetesClientConfiguration.<LoadKubeConfigAsync>d__31.MoveNext() in F:\Code\kubernetes-client\src\KubernetesClient\KubernetesClientConfiguration.ConfigFile.cs:line 639
   at k8s.KubernetesClientConfiguration.<LoadKubeConfigAsync>d__36.MoveNext() in F:\Code\kubernetes-client\src\KubernetesClient\KubernetesClientConfiguration.ConfigFile.cs:line 713
   at k8s.KubernetesClientConfiguration.LoadKubeConfig(FileInfo[] kubeConfigs, Boolean useRelativePaths) in F:\Code\kubernetes-client\src\KubernetesClient\KubernetesClientConfiguration.ConfigFile.cs:line 695
   at k8s.Tests.KubernetesClientConfigurationTests.<>c__DisplayClass48_0.<LoadKubeConfigShouldBeThreadSafe>g__LoadKubeConfig|3() in F:\Code\kubernetes-client\tests\KubernetesClient.Tests\KubernetesClientConfigurationTests.cs:line 691

   # INNER EXCEPTION

   Operations that change non-concurrent collections must have exclusive access. A concurrent update was performed on this collection and corrupted its state. The collection's state is no longer correct.

      at System.ThrowHelper.ThrowInvalidOperationException_ConcurrentOperationsNotSupported()
   at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
   at System.Collections.Generic.Dictionary`2.set_Item(TKey key, TValue value)
   at YamlDotNet.Serialization.ObjectFactories.DefaultObjectFactory.GetStateMethods(Type attributeType, Type valueType)
   at YamlDotNet.Serialization.ObjectFactories.DefaultObjectFactory.ExecuteState(Type attributeType, Object value)
   at YamlDotNet.Serialization.ObjectFactories.DefaultObjectFactory.ExecuteOnDeserializing(Object value)
   at YamlDotNet.Serialization.NodeDeserializers.ObjectNodeDeserializer.Deserialize(IParser parser, Type expectedType, Func`3 nestedObjectDeserializer, Object& value)
   at YamlDotNet.Serialization.ValueDeserializers.NodeValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Mar 5, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: nathanwoctopusdeploy
Once this PR has been reviewed and has the lgtm label, please assign tg123 for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Mar 5, 2024
@@ -12,6 +12,27 @@ namespace k8s
/// </summary>
public static class KubernetesYaml
{
//private static DeserializerBuilder GetCommonDeserializerBuilder() =>
Copy link
Contributor Author

@nathanwoctopusdeploy nathanwoctopusdeploy Mar 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing YamlDotNet Deserializer to no longer be a static makes the test pass

@nathanwoctopusdeploy
Copy link
Contributor Author

#1537

@nathanwoctopusdeploy nathanwoctopusdeploy changed the title YamlDotNot no longer appears to be thread-safe YamlDotNot is not thread-safe Mar 5, 2024
@tg123
Copy link
Member

tg123 commented Mar 5, 2024

oh seems some state inside
what about a cheap solution by adding a lock?

@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

❗ No coverage uploaded for pull request base (master@f78a516). Click here to learn what that means.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff            @@
##             master    #1536   +/-   ##
=========================================
  Coverage          ?   62.07%           
=========================================
  Files             ?      102           
  Lines             ?     3014           
  Branches          ?      636           
=========================================
  Hits              ?     1871           
  Misses            ?     1143           
  Partials          ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@nathanwoctopusdeploy
Copy link
Contributor Author

oh seems some state inside what about a cheap solution by adding a lock?

Hi @tg123

I'm not familiar enough with the code base and YamlDotNet library to know what the best direction would be to fix this. The PR was created to provide an easy reproduction of the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants