User facing output to file diff #2749
Conversation
michaelcfanning
requested review from
suvamM,
shaopeng-gh,
scottoneil-ms,
rwoll,
yongyan-gh,
LingZhou-gh,
EasyRhinoMSFT and
cfaucon
as code owners
|
|
||
| foreach (string key in keys) | ||
| { | ||
| string actualSarifText = actualSarifTextDictionary[key]; | ||
| SarifLog actualSarifLog = JsonConvert.DeserializeObject<SarifLog>(actualSarifText); | ||
| if (actualSarifLog.Runs?[0].TryGetProperty("consoleOut", out string userFacingText) == true) |
There was a problem hiding this comment.
So here's the trick upstream users of this base class can do. If they'd like, they can populate the run with consoleOut to capture user-facing strings. We can (and should) build this deeper into the test infra but it will break everyone's tests (which will require baselining). So we're taking this spotfix for now.
|
Will the generated .txt file same when running on different machines, does it contains the part that is the absolute path of the file that is having problem which will be different on the machine |
|
The non-deterministic repo root is normalized to a variable, e.g.: $(SRC_ROOT)\src\Plugins\Tests\TestData\Example\Inputs\TST101_001.TestRule.ps1(4,43-69): warning STST101/001: '…111111' was flagged as a potential problem. In reply to: 1867931239 |
This change retrieves and persists user-facing output generated by functional tests. These strings are to be persisted alongside the generated
.sariffiles in theExpecteddirectory inTestData. We are experimenting with this capability in one SARIF Driver framework use case. Once it's proven/refined, we should push that code deeper into the test API.