Substantial refinements to Network Tracing / NetBlame Plug-in

[rayfo]

• Standardize on unknown thread: tidUnknown = -1
  -Using Thread ID 0 is ambiguous.

  • NetBlame\CallStack.cs
  • NetBlame\Auxiliary\Tasks.cs
  • NetBlame\Providers\TcpIp.cs
  • NetBlame\Providers\Thread.Classic.cs
  • NetBlame\Providers\WebIO.Request.cs
  • NetBlame\Providers\WinHTTP.cs
  • NetBlame\Providers\WinINet.cs
  • NetBlame\Providers\WinsockAFD.cs
  • NetBlame\Providers\WThreadPool.Timer.cs
  • NetBlame\Tables\NetBlameTable.ThreadPool.cs
  • NetBlame\Tables\NetBlameTable.URL.cs
  • NetBlame\Tables\NetBlameTable.WinSock.cs
  • NetBlame\Tables\TableBase.cs

• Improve validity of Process ID (PID)
  -Many events have ambiguous ProcessId values, particular TcpIp (TCP/UDP) and Winsock.
  -Winsock: Use the "Location" field to determine whether the event is synchronous and the PID is reliable.
  -TcpIp: Classify the PID as confirmed when it comes from the "Process" field (not the process which emitted the event).

  • NetBlame\Providers\TcpIp.cs
  • NetBlame\Providers\WinsockAFD.cs

• Improve correlation of WebIO (WinHTTP) Requests, Connections, Sockets
  -Improve the code which finds and updates the Connection for the events: ConnectionSocketSend/Receive, ConnectionSocketConnect_Stop, ConnectionSocketClose
  -Synthesize missing Requests & Connections (near the beginning of the trace).
  -Track the validity of the Connection ID (DEBUG).

  • NetBlame\Providers\WebIO.cs
  • NetBlame\Providers\WebIO.Connection.cs
  • NetBlame\Providers\WebIO.Request.cs

• Improve correlation across event types.
  -Particularly, improve correlation of UDP events and Winsock->UDP events.

  • NetBlame\Providers\TcpIp.cs
  • NetBlame\Providers\WinsockAFD.cs

• Reconstruct Connection activity that appears near the start of the trace.

  • NetBlame\Providers\WebIO.cs

• Consolidate IP address type resolution.
  -There are ranges of IP addresses which have special functions. Use these in the GeoLocation column and elsewhere.

  • NetBlame\Auxiliary\GeoLocation.cs
  • NetBlame\Auxiliary\NetUtil.cs

• Better manage and report symbol resolution.
  -Better handle missing stackwalks.
  -Remove ambiguity when there's only one stackwalk (so it's both First & Last!?).

  • NetBlame\NetBlameDataProcessor.cs
  • NetBlame\CallStack.cs

• The rules for finding the best version of WPA, WPR, XPerf are subtly different.
  -XPerf installed with Store-WPA only works when launched as a packaged app.
  -Respect WPT_PATH when searching for XPerf.

  • Include.ps1
  • Include.WPA.ps1
  • BETA\TraceNetwork.ps1
  • BETA\GetSymbols.bat

• Confirm the above improvements against a variety of Network traces.
  -ETW events can be (seemingly) irregular. Gathering and correlating them is not straightforward: TCP/UDP <-> WinSock <-> WebIO or WinINet

  • All

• Set the version number to 1.6.0

  • NetBlame\NetBlameAddIn.csproj

---

Include.ps1

• Better algorithm for finding the best version of a requested binary.
• Handle error: 0x80071069 (aborted trace)

Include.WPA.ps1

• Find a better version of XPerf.exe to do background symbol resolution. Better output filtering.

BETA\TraceNetwork.ps1

• Better choose WPA version that works with NetBlame add-in.

BETA\GetSymbols.bat

• Use WPT_PATH to find xperf.exe, and do better output filtering.

NetBlame\NetBlameAddIn.csproj

• Add Version Number (1.6.0)

NetBlame\GatherTables.cs

• Fill in a few empty fields in the final table.

NetBlame\NetBlameDataProcessor.cs

• Better manage symbol resolution, and create better list of "target" processes for symbol resolution.

NetBlame\CallStack.cs

• Standardize on unknown thread: TID.Unknown = -1
• Better report when symbol resolution is disabled: 0% => disabled
• When there is only one stackwalk, it's the Last Stack.
• Better handle missing stackwalks.

NetBlame\Auxiliary\Extensions.cs

• SocketAddress.Equals, .IsAddrZero: 0.0.0.0 or [::]

NetBlame\Auxiliary\GeoLocation.cs

• Consolidate IP address type resolution.

NetBlame\Auxiliary\NetUtil.cs

• Consolidate IP address type resolution.
• Improve address services.

NetBlame\Auxiliary\Tasks.cs

• Standardize on unknown thread: tidUnknown = -1

NetBlame\Providers\DNSClient.cs

• Improve handling null addresses.

NetBlame\Providers\MsoIdleMan.cs

• Better assert regarding deleted task

NetBlame\Providers\TcpIp.cs

• Standardize on unknown thread: tidUnknown = -1
• Classify the PID as confirmed or not
• Improve correlation, particular with UDP events

NetBlame\Providers\Thread.Classic.cs

• Standardize on unknown thread: tidUnknown = -1

NetBlame\Providers\Thread.cs

• Handle duplicated events from a merged trace.

NetBlame\Providers\WebIO.Connection.cs

• Match the context of ConnectionSocketSend_Start/Stop and ConnectionSocketReceive_Start/Stop
• Confirm socket consistency.

NetBlame\Providers\WebIO.cs

• Match the context of ConnectionSocketSend_Start/Stop and ConnectionSocketReceive_Start/Stop
• Handle missing WebIO Request & Connection records (near trace start)
• Track the validity of the Connection ID (DEBUG).

NetBlame\Providers\WebIO.Request.cs

• Track the validity of the Connection ID (DEBUG).
• Standardize on unknown thread: tidUnknown = -1
• Improve the code which finds and updates the Connection for the events: ConnectionSocketSend/Receive, ConnectionSocketConnect_Stop, ConnectionSocketClose

NetBlame\Providers\WebIO.Socket.cs

• Remove unused inclusion

NetBlame\Providers\WinHTTP.cs

• Standardize on unknown thread: tidUnknown = -1
• Performance optimization
• Diminish an Assert.

NetBlame\Providers\WinINet.cs

• Handle a few corner cases.
• Standardize on unknown thread: tidUnknown =-1

NetBlame\Providers\WinsockAFD.cs

• Standardize on unknown thread: tidUnknown = -1
• Improve code to correlate Winsock with TCP and UDP events.
• Use "Location" field to differentiate the characteristics of each event, particularly the reliability of the Process ID (PID).
• Hash the Process Handle to a reliable PID.

NetBlame\Providers\WThreadPool.Timer.cs

• Standardize on unknown thread: tidUnknown = -1

NetBlame\Tables\NetBlameTable.ThreadPool.cs

• Standardize on unknown thread: tidUnknown = -1 => "N/A"

NetBlame\Tables\NetBlameTable.URL.cs

• Standardize on unknown thread: tidUnknown = -1 => "N/A"

NetBlame\Tables\NetBlameTable.WinSock.cs

• Standardize on unknown thread: tidUnknown = -1 => "N/A"

NetBlame\Tables\TableBase.cs

• Standardize on unknown thread: tidUnknown = -1 => "N/A"