Catastrophic error deserializing item #1: System.ArgumentException: An item with the same key has already been added in Test runs triggered from VsTest Test run task in the release pipeline

[mythilimuthyala]

Problem:
After execution of xUnit tests in Azure release pipeline triggered from Devops Test plan, test runner reports error:
##[error]RunMessage : [xUnit.net 00:00:01.00] Tests.Integration: Catastrophic error deserializing item #1: System.ArgumentException: An item with the same key has already been added. Key: ff59767d14fb96ac4d731140b9960ea8e31f1c8e

Environment:
.Net version: netcoreapp2.2
xunit version: 2.4.1
xunit.runner.visualstudio version: 2.4.1

Steps to reproduce:

1. Create test method with attributes [Theory] [MemberData(nameof(ArgumentsGenerator))]
   where ArgumentsGenerator is some method which generates arguments.

2. Serialized MemberData arguments using IXunitSerializable so that all the tests show up for each theory instead of one in the Test Explorer

3. Associated tests with test case from Devops Test Plan
   

4. Created a release pipeline with VsTest run task and a stage

5. From Testplan, chose run with options, and chose appropriate build, pipeline and the stage and triggered the test run

6. Only 1 test is being executed and all the remaining tests throws an exception "Catastrophic error deserializing item Support Visual Studio 2014 (CTP) #1: System.ArgumentException: An item with the same key has already been added"
   

The issue is similar to this - https://developercommunity.visualstudio.com/content/problem/426873/azure-devops-test-hub-auto-run-catastrophic-error.html and was resolved as XUnit issue.

Any advises here please?

[bradwilson]

Based on the error you're seeing, it sounds like it's going to come down to the serialization implementation. Can you provide that, please?

[mythilimuthyala]

Sure! thanks for attending this.

The MemberDataSerializer.cs look like this

namespace Product.NAS.Tests.Common.Helpers
{
    public class MemberDataSerializer<T> : IXunitSerializable
    {
        public T Object { get; private set; }
        // required for deserializer
        public MemberDataSerializer()
        {
        }

        public MemberDataSerializer(T objSerialize)
        {
            Object = objSerialize;
        }

        public void Deserialize(IXunitSerializationInfo info)
        {
            Object = JsonConvert.DeserializeObject<T>(info.GetValue<string>("objValue"));
        }

        public void Serialize(IXunitSerializationInfo info)
        {
            var json = JsonConvert.SerializeObject(Object);
            info.AddValue("objValue", json);
        } 
    }
}

My theory tests look like this -

`  [Theory, MemberData(nameof(EventsToTest.DupDataEvent), MemberType = typeof(EventsToTest))]
        public void TestDeDupEvent(string productId, string testName, MemberDataSerializer<EventMessage> evt)
        {
            //Test methods go here
        }`

EventsToTest used in the Theory look like this -

public static class EventsToTest
    {
public static IEnumerable<object[]> DupDataEvent = GetDupDataValuesEvent(ValidEvents);
       

public static IEnumerable<object[]> GetDupDataValuesEvent(Dictionary<string, Dictionary<string, EventMessage>> events)
        {
            foreach (var product in events)
            {
                foreach (var elt in events[product.Key].Where(x => x.Value.Body.eventType.ToLower().Equals("wkflw")))
                {
                        yield return new object[] { product.Key, elt.Key, new MemberDataSerializer<EventMessage>(elt.Value) };
                }
            }
        }
}

[mythilimuthyala]

Hi BradWilson - Did you get a chance to look at this? Is this the way serialization is implemented as in the script pasted above? please advise. Thank you.

[mythilimuthyala]

Hi, can I get some help here please?

[mythilimuthyala]

Updating for the benefit of information for others who encounters this issue -

The data set that I am trying to run using Theroy Memberdata is random data which seems is not handled currently by XUnit in preenumeration as well as test association. Hence, migrated to NUnit. It supports random data as well as serialization of parameterized tests out of the box.

[dev-fJ-del]

Any feedback regarding this issue?

Any feedback regarding this issue?

I ran into this myself, "solved" it like this:

	public virtual void Serialize(IXunitSerializationInfo info)
	{
		info.AddValue("uniquifier", Guid.NewGuid().ToString());
		/* ... */
	}

[bradwilson]

Making an educated guess here based on the fix: the issue is that test case serialization requires that we give each test case a unique ID:

https://github.com/xunit/xunit/blob/161cf974e98ab52f9daf36388b3695564afafa74/src/xunit.execution/Sdk/Frameworks/TestMethodTestCase.cs#L211-L248

Part of that unique ID calculation uses the test method arguments (for data-driven tests) as part of the unique ID calculation:

if (TestMethodArguments != null)
    Write(stream, SerializationHelper.Serialize(TestMethodArguments));

If you have two tests in the test case with what ends up to be identical test method arguments, from a serialization standpoint, they end up with the same unique ID. And then when deserialized, we place them into a dictionary based on their unique ID, which is where the failure is likely coming from. Adding the random value into your serialization now causes you to end up with a unique set of argument values, bypassing the issue.

This is, unfortunately, working as designed and not something that will be fixed.