Skip to content

Utf8JsonReader.BytesConsumed incorrect for input "2e2" in multi-segment mode #30462

@GSPP

Description

@GSPP

When parsing "2e2", BytesConsumed seems to be smaller by 1 than it should be. The number itself is being parsed correctly. "[2e2]" also is being parsed correctly and again, BytesConsumed is incorrect.

Single-segment mode is correct.

Another interesting case to test is 2e+2.

static class ReproProgram
{
    public static void Run()
    {
        IEnumerable<Memory<byte>> GetMemories(byte[] array)
        {
            for (int i = 0; i < array.Length; i++)
                yield return array.AsMemory(i, 1);
        }

        var array = Encoding.UTF8.GetBytes("2e2");
        var jsonReader = new Utf8JsonReader(ReadOnlyBufferSegment.Create(GetMemories(array)));

        while (jsonReader.Read())
        {
            Console.WriteLine($"{jsonReader.TokenType} {jsonReader.BytesConsumed} {jsonReader.GetDouble()}");
        }

        Console.WriteLine($"Input length: {array.Length}, BytesConsumed: {jsonReader.BytesConsumed}");
    }

    class ReadOnlyBufferSegment : ReadOnlySequenceSegment<byte>
    {
        public static ReadOnlySequence<byte> Create(IEnumerable<Memory<byte>> buffers)
        {
            ReadOnlyBufferSegment segment = null;
            ReadOnlyBufferSegment first = null;
            foreach (Memory<byte> buffer in buffers)
            {
                var newSegment = new ReadOnlyBufferSegment()
                {
                    Memory = buffer,
                };

                if (segment != null)
                {
                    segment.Next = newSegment;
                    newSegment.RunningIndex = segment.RunningIndex + segment.Memory.Length;
                }
                else
                {
                    first = newSegment;
                }

                segment = newSegment;
            }

            if (first == null)
            {
                first = segment = new ReadOnlyBufferSegment();
            }

            return new ReadOnlySequence<byte>(first, 0, segment, segment.Memory.Length);
        }
    }
}

Number 2 200
Input length: 3, BytesConsumed: 2

This is 3.0.0-preview7-27912-14.

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions