Skip to content

add basic logging for test failures #21023

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

sanjays-ms
Copy link

@sanjays-ms sanjays-ms commented May 13, 2025

Context

GradleV3 Task Logging
The GradleV3 Task was printing code analysis failure messages when the pipeline was failing due to test case failures.


Task Name

GradleV3


Description

Changed output log messages to output gradle output as such.

Risk Assessment

Medium
Justification: Changing the existing logging message so customers using this task will see a change in logs in future pipeline runs in case of pipeline failures.

@mishrarish mishrarish marked this pull request as ready for review May 13, 2025 10:31
@mishrarish mishrarish requested review from tarunramsinghani and a team as code owners May 13, 2025 10:31
@sanjays-ms
Copy link
Author

@microsoft-github-policy-service agree company="Microsoft"


const errorMsg = err.toString().toLowerCase();
const isTaskNameIndicatesTest = inputTasks.some(task => task.toLowerCase().includes('test') && !task.toLowerCase().includes('setup'));
const errorMsgIndicatesTest = errorMsg.includes('test failure') || errorMsg.includes('tests failed') || errorMsg.includes('failing tests') || (errorMsg.includes('test') && errorMsg.includes('fail'));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These checks will fail in case of localization i.e. if this is running on some machine with different language. we need a better way to identify what is failing.

@sanjays-ms
Copy link
Author

/azp run

@sanjays-ms
Copy link
Author

/azp run

@sanjays-ms sanjays-ms requested a review from Copilot June 15, 2025 14:58
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enhances the GradleV3 task by capturing and surfacing Gradle output when the task fails, so that test failures are logged more clearly.

  • Bump task version (Minor: 247→257, Patch: 1→0)
  • Add an optional error field to ITaskResult and update resolveTaskResult to include full Gradle output
  • Capture Gradle’s stdout/stderr via custom streams and log the aggregated error output

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
Tasks/GradleV3/task.loc.json Updated Minor/Patch version numbers
Tasks/GradleV3/task.json Updated Minor/Patch version numbers
Tasks/GradleV3/interfaces.ts Added error?: string to ITaskResult
Tasks/GradleV3/gradletask.ts Created stdoutStream/stderrStream to collect output, passed to gradleRunner.exec, and log taskResult.error
Tasks/GradleV3/Modules/utils.ts Extended resolveTaskResult signature to accept gradleOutput and populate the error field
Comments suppressed due to low confidence (3)

Tasks/GradleV3/Modules/utils.ts:10

  • Update the JSDoc comment for resolveTaskResult to describe the new gradleOutput parameter and explain the purpose of the returned error field for clearer API docs.
export function resolveTaskResult(codeAnalysisResult: ICodeAnalysisResult, gradleOutput: string[]): ITaskResult {

Tasks/GradleV3/Modules/utils.ts:21

  • There are no unit tests covering the new error aggregation in resolveTaskResult; consider adding tests for non-zero exit codes to verify both the message and error fields.
error = gradleOutput.join('');

Tasks/GradleV3/interfaces.ts:41

  • [nitpick] Add a comment or JSDoc for the new error property in ITaskResult to describe when it will be populated and what it contains.
error?: string;

Comment on lines 143 to 147
const gradleOutput: string[] = [];
const stdoutStream = new stream.Writable({
write: (chunk, encoding, callback) => {
const output = chunk.toString();
gradleOutput.push(output);
Copy link
Preview

Copilot AI Jun 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Storing the entire Gradle output in memory may lead to high memory usage for large builds. Consider streaming to a temporary file or imposing a size limit on the buffer.

Suggested change
const gradleOutput: string[] = [];
const stdoutStream = new stream.Writable({
write: (chunk, encoding, callback) => {
const output = chunk.toString();
gradleOutput.push(output);
const fs = require('fs');
const os = require('os');
const path = require('path');
const tempFilePath = path.join(os.tmpdir(), `gradle-output-${Date.now()}.log`);
const outputStream = fs.createWriteStream(tempFilePath);
const stdoutStream = new stream.Writable({
write: (chunk, encoding, callback) => {
const output = chunk.toString();
outputStream.write(output);

Copilot uses AI. Check for mistakes.

tl.setResult(taskResult.status, taskResult.message);
if(taskResult.error !== undefined && taskResult.error !== '') {
console.log(taskResult.error);
Copy link
Preview

Copilot AI Jun 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Use console.error instead of console.log when emitting error details, to ensure these messages are clearly marked as errors in the logs.

Suggested change
console.log(taskResult.error);
console.error(taskResult.error);

Copilot uses AI. Check for mistakes.

@sanjays-ms
Copy link
Author

/azp run

@sanjays-ms
Copy link
Author

/azp run

@tarunramsinghani
Copy link
Contributor

Can you please add how the task output has changed after teh change ? both the cases i.e. a successful build and build with test failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants