Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix empty byte[] bug in EventSource #52602

Merged
merged 6 commits into from
May 13, 2021
Merged

Fix empty byte[] bug in EventSource #52602

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1790ee3
Fix empty byte[] bug in EventSource
MihaZupan May 11, 2021
277d48f
Add some non-zero data to byte[] tests
MihaZupan May 11, 2021
c61141c
Remove leftover usings from testing
MihaZupan May 11, 2021
cb9cce3
Merge main
MihaZupan May 12, 2021
da53d41
Return Array.Empty for empty byte[]
MihaZupan May 12, 2021
2931c59
Pass correct pointer to Marshal copy
MihaZupan May 13, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 43 additions & 2 deletions ...raries/System.Diagnostics.Tracing/tests/BasicEventSourceTest/TestsWriteEventToListener.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System;
using System.Collections.Generic;
using System.Linq;
using Xunit;
Expand All @@ -19,7 +20,7 @@ public partial class TestsWriteEventToListener
[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/21569", TargetFrameworkMonikers.NetFramework)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51382", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public void Test_WriteEvent_ArgsBasicTypes()
public unsafe void Test_WriteEvent_ArgsBasicTypes()
{
TestUtilities.CheckNoEventSourcesRunning("Start");

Expand Down Expand Up @@ -105,11 +106,51 @@ public void Test_WriteEvent_ArgsBasicTypes()

#region Validate byte array arguments

var rng = new Random(42);

byte[] arr = new byte[20];
rng.NextBytes(arr);
log.EventWithByteArray(arr);
Assert.Equal(52, LoudListener.t_lastEvent.EventId);
Assert.Equal(1, LoudListener.t_lastEvent.Payload.Count);
Assert.Equal(arr.Length, ((byte[])LoudListener.t_lastEvent.Payload[0]).Length);
Assert.Equal(arr, (byte[])LoudListener.t_lastEvent.Payload[0]);

log.EventWithByteArray(new byte[0]);
Assert.Equal(52, LoudListener.t_lastEvent.EventId);
Assert.Equal(1, LoudListener.t_lastEvent.Payload.Count);
Assert.Same(Array.Empty<byte>(), (byte[])LoudListener.t_lastEvent.Payload[0]);

log.EventWithByteArray(null);
Assert.Equal(52, LoudListener.t_lastEvent.EventId);
Assert.Equal(1, LoudListener.t_lastEvent.Payload.Count);
Assert.Same(Array.Empty<byte>(), (byte[])LoudListener.t_lastEvent.Payload[0]);

arr = new byte[20];
rng.NextBytes(arr);
log.EventWithByteArrayCustom(arr);
Assert.Equal(53, LoudListener.t_lastEvent.EventId);
Assert.Equal(1, LoudListener.t_lastEvent.Payload.Count);
Assert.Equal(arr, (byte[])LoudListener.t_lastEvent.Payload[0]);

log.EventWithByteArrayCustom(new byte[0]);
Assert.Equal(53, LoudListener.t_lastEvent.EventId);
Assert.Equal(1, LoudListener.t_lastEvent.Payload.Count);
Assert.Same(Array.Empty<byte>(), (byte[])LoudListener.t_lastEvent.Payload[0]);

arr = new byte[20];
rng.NextBytes(arr);
fixed (byte* arrPtr = arr)
{
log.EventWithBytePointer(arrPtr, arr.Length);
}
Assert.Equal(54, LoudListener.t_lastEvent.EventId);
Assert.Equal(1, LoudListener.t_lastEvent.Payload.Count);
Assert.Equal(arr, (byte[])LoudListener.t_lastEvent.Payload[0]);

log.EventWithBytePointer((byte*)IntPtr.Zero, 0);
Assert.Equal(54, LoudListener.t_lastEvent.EventId);
Assert.Equal(1, LoudListener.t_lastEvent.Payload.Count);
Assert.Same(Array.Empty<byte>(), (byte[])LoudListener.t_lastEvent.Payload[0]);

#endregion

Expand Down
40 changes: 40 additions & 0 deletions src/libraries/System.Diagnostics.Tracing/tests/CustomEventSources/EventSourceTest.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,46 @@ public void EventWithByteArray(byte[] arr)
this.WriteEvent(52, arr);
}

[Event(53)]
public unsafe void EventWithByteArrayCustom(byte[] arr)
{
// This implementation does not guard against passing null as DataPointer for Length == 0 case
arr ??= Array.Empty<byte>();

fixed (byte* arg1Ptr = arr)
{
int bufferLength = arr.Length;

const int NumEventDatas = 2;
var descrs = stackalloc EventData[NumEventDatas];

descrs[0] = new EventData
{
DataPointer = (IntPtr)(&bufferLength),
Size = sizeof(int)
};
descrs[1] = new EventData
{
DataPointer = (IntPtr)arg1Ptr,
Size = arr.Length
};

WriteEventCore(53, 2, descrs);
}
}

[Event(54)]
public unsafe void EventWithBytePointer(byte* ptr, int length)
{
var data = new EventData
{
DataPointer = (IntPtr)ptr,
Size = length
};

WriteEventCore(54, 1, &data);
}

#region Keywords / Tasks /Opcodes / Channels
public class Keywords
{
Expand Down
26 changes: 12 additions & 14 deletions src/libraries/System.Net.Sockets/tests/FunctionalTests/LoggingTest.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
// The .NET Foundation licenses this file to you under the MIT license.

using System.Collections.Concurrent;
using System.Diagnostics;
using System.Diagnostics.Tracing;
using Microsoft.DotNet.RemoteExecutor;
using Xunit;
Expand Down Expand Up @@ -34,32 +33,31 @@ public static void EventSource_ExistsWithCorrectId()

[OuterLoop]
[ConditionalFact(typeof(RemoteExecutor), nameof(RemoteExecutor.IsSupported))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/50639")]
public void EventSource_EventsRaisedAsExpected()
{
RemoteExecutor.Invoke(() =>
RemoteExecutor.Invoke(async () =>
{
using (var listener = new TestEventListener("Private.InternalDiagnostics.System.Net.Sockets", EventLevel.Verbose))
{
var events = new ConcurrentQueue<EventWrittenEventArgs>();
listener.RunWithCallback(events.Enqueue, () =>
await listener.RunWithCallbackAsync(events.Enqueue, async () =>
{
// Invoke several tests to execute code paths while tracing is enabled

new SendReceive_Sync(null).SendRecv_Stream_TCP(IPAddress.Loopback, false).GetAwaiter();
new SendReceive_Sync(null).SendRecv_Stream_TCP(IPAddress.Loopback, true).GetAwaiter();
await new SendReceive_Sync(null).SendRecv_Stream_TCP(IPAddress.Loopback, false);
await new SendReceive_Sync(null).SendRecv_Stream_TCP(IPAddress.Loopback, true);

new SendReceive_Task(null).SendRecv_Stream_TCP(IPAddress.Loopback, false).GetAwaiter();
new SendReceive_Task(null).SendRecv_Stream_TCP(IPAddress.Loopback, true).GetAwaiter();
await new SendReceive_Task(null).SendRecv_Stream_TCP(IPAddress.Loopback, false);
await new SendReceive_Task(null).SendRecv_Stream_TCP(IPAddress.Loopback, true);

new SendReceive_Eap(null).SendRecv_Stream_TCP(IPAddress.Loopback, false).GetAwaiter();
new SendReceive_Eap(null).SendRecv_Stream_TCP(IPAddress.Loopback, true).GetAwaiter();
await new SendReceive_Eap(null).SendRecv_Stream_TCP(IPAddress.Loopback, false);
await new SendReceive_Eap(null).SendRecv_Stream_TCP(IPAddress.Loopback, true);

new SendReceive_Apm(null).SendRecv_Stream_TCP(IPAddress.Loopback, false).GetAwaiter();
new SendReceive_Apm(null).SendRecv_Stream_TCP(IPAddress.Loopback, true).GetAwaiter();
await new SendReceive_Apm(null).SendRecv_Stream_TCP(IPAddress.Loopback, false);
await new SendReceive_Apm(null).SendRecv_Stream_TCP(IPAddress.Loopback, true);

new NetworkStreamTest().CopyToAsync_AllDataCopied(4096, true).GetAwaiter().GetResult();
new NetworkStreamTest().Timeout_Roundtrips().GetAwaiter().GetResult();
await new NetworkStreamTest().CopyToAsync_AllDataCopied(4096, true);
await new NetworkStreamTest().Timeout_Roundtrips();
});
Assert.DoesNotContain(events, ev => ev.EventId == 0); // errors from the EventSource itself
Assert.InRange(events.Count, 1, int.MaxValue);
Expand Down
14 changes: 10 additions & 4 deletions src/libraries/System.Private.CoreLib/src/System/Diagnostics/Tracing/EventSource.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1738,7 +1738,6 @@ private static unsafe void DecodeObjects(object?[] decodedObjects, Type[] parame
// byte[] are written to EventData* as an int followed by a blob
Debug.Assert(*(int*)dataPointer == (data + 1)->Size);
data++;
dataPointer = data->DataPointer;
goto BytePtr;
}
else if (IntPtr.Size == 4 && dataType == typeof(IntPtr))
Expand Down Expand Up @@ -1836,9 +1835,16 @@ private static unsafe void DecodeObjects(object?[] decodedObjects, Type[] parame
}

BytePtr:
var blob = new byte[data->Size];
Marshal.Copy(dataPointer, blob, 0, blob.Length);
decoded = blob;
if (data->Size == 0)
{
decoded = Array.Empty<byte>();
}
else
{
var blob = new byte[data->Size];
Marshal.Copy(data->DataPointer, blob, 0, blob.Length);
decoded = blob;
}
Comment on lines +1838 to +1847
Copy link
Member

@stephentoub stephentoub May 26, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be simpler and safer as:

decoded = new Span<byte>((byte*)data->DataPointer, data->Size).ToArray();

Copy link
Member Author

@MihaZupan MihaZupan May 26, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is what I had used initially, but it sadly involves duplicating the logic under #if #else as EventSource is also being built standalone (no Span)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not reference the netstandard span package from the downlevel solution?

Either way, I don't think we should penalize the latest platform because of the source sharing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@brianrob, we still maintain/ship the separate EventSource package? Who uses that?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either way, I don't think we should penalize the latest platform because of the source sharing.

100% agree with this.

Who uses that?

This periodically comes up (see: #32276 (comment)). This is building this package, which is used to ship EventSource features out-of-band for older framework releases. Download counts have been dwindling since 2015. The 5.0 version has ~320k downloads in 7 months.

Copy link
Member

@stephentoub stephentoub May 26, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which is used to ship EventSource features out-of-band for older framework releases

Are we still adding new EventSource features it's important to be able to release in this vehicle? Can we just stop shipping new versions of it? If it's about bug fixes as Brian mentions in the linked comment, can we just treat it as servicing and fix bugs out of release/5.0?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm okay with moving that to a servicing life cycle. Brian has a better sense of what impact that would have though, so I would defer to his input before ripping it out.

CC @noahfalk

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The EventSource package is really used for two things:

  1. Ship new features out-of-band for older frameworks.
  2. Ship bug fixes out-of-band for older frameworks.

The package only targets .NET Framework, so anything for .NET Core would just go through the standard servicing process. It's not clear to me that we're adding new features to EventSource at a rate that really requires the package - for me, the question is really whether or not we need a relief valve to be able to fix issues on .NET Framework. If we can do that through a servicing release, then that's fine, but I worry that .NET 5 will only be supported for so long and then we lose the infrastructure and shipping support for this. Over time, as we move folks towards .NET Core, this package becomes less and less important, but I don't think we're in a place where we'll be ready in a year or two to just stop producing the package, even through servicing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a relief valve to be able to fix issues on .NET Framework

To make sure I understand, this doesn't actually unify with what's in .NET Framework, right? Someone has to explicitly switch from using the EventSource in .NET Framework to this differently namespaced type in the OOB?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's right - it does not unify. Someone will have to explicitly consume the NuGet package. The namespace is also different - Microsoft.Diagnostics.Tracing instead of System.Diagnostics.Tracing.

goto Store;

String:
Expand Down