Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for writing the tests output in JSON format #6010

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
69 changes: 67 additions & 2 deletions Sources/Commands/SwiftTestTool.swift
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,10 @@ struct TestToolOptions: ParsableArguments {
help: "Path where the xUnit xml file should be generated.")
var xUnitOutput: AbsolutePath?

@Option(name: .customLong("json-output"),
help: "Path where the json output file should be generated.")
var jsonOutput: AbsolutePath?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this flag should be mutually exclusive with with xUnitOutput?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would a better alternative be to introduce a single option through which users can specify test output format instead of introducing flags for every single format?

Copy link
Author

@NSCoder NSCoder Jan 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good questions, I don't see a reason why they have to be mutually exclusive besides modifying the options from the command to only allow one output.

@MaxDesiatov I was thinking about this as well, but giving that I'm not familiar with the project I decided to keep this patch it as simple as possible. Would it be all right if we try to add the output option refactor in a following up patch?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I think it is fine for now


/// Generate LinuxMain entries and exit.
@Flag(name: .customLong("testable-imports"), inversion: .prefixedEnableDisable, help: "Enable or disable testable imports. Enabled by default.")
var enableTestableImports: Bool = true
Expand Down Expand Up @@ -307,6 +311,15 @@ public struct SwiftTestTool: SwiftCommand {
try generator.generate(at: xUnitOutput)
}

// Generate json file if requested
if let jsonOutput = options.jsonOutput {
let generator = JSONGenerator(
fileSystem: swiftTool.fileSystem,
results: testResults
)
try generator.generate(at: jsonOutput)
}

// process code Coverage if request
if self.options.enableCodeCoverage {
try processCodeCoverage(testProducts, swiftTool: swiftTool)
Expand Down Expand Up @@ -578,6 +591,16 @@ struct UnitTest {
}
}

extension UnitTest: JSONSerializable {
func toJSON() -> JSON {
JSON([
"productPath": productPath,
"name": name,
"testCase": testCase
])
}
}

/// A class to run tests on a XCTest binary.
///
/// Note: Executes the XCTest with inherited environment as it is convenient to pass senstive
Expand Down Expand Up @@ -690,14 +713,34 @@ final class TestRunner {
}
}

extension DispatchTimeInterval {
/// Encode the nanoseconds dispatch time in JSON
/// - Returns: JSON instance for the nanoseconds or `.null`
public func toNanosecondsJSON() -> JSON {
guard let nanoseconds = nanoseconds() else {
return .null
}
return nanoseconds.toJSON()
}
}

/// A class to run tests in parallel.
final class ParallelTestRunner {
/// An enum representing result of a unit test execution.
struct TestResult {
struct TestResult: JSONSerializable {
NSCoder marked this conversation as resolved.
Show resolved Hide resolved
var unitTest: UnitTest
var output: String
var success: Bool
var duration: DispatchTimeInterval

func toJSON() -> TSCBasic.JSON {
JSON([
"output": output,
"success": success,
"unitTest": unitTest,
"durationInNanoseconds": duration.toNanosecondsJSON()
])
}
}

/// Path to XCTest binaries.
Expand Down Expand Up @@ -1000,11 +1043,33 @@ fileprivate extension Array where Element == UnitTest {
}
}

// JSON file generator for a swift-test run.
final class JSONGenerator {
typealias TestResult = ParallelTestRunner.TestResult

/// The file system to use.
let fileSystem: FileSystem

/// The test results.
let results: [TestResult]

init(fileSystem: FileSystem, results: [TestResult]) {
self.fileSystem = fileSystem
self.results = results
}

/// Generate the file at the given path.
func generate(at path: AbsolutePath) throws {
let json = results.toJSON()
try self.fileSystem.writeFileContents(path, bytes: json.toBytes())
}
}

/// xUnit XML file generator for a swift-test run.
final class XUnitGenerator {
typealias TestResult = ParallelTestRunner.TestResult

/// The file system to use
/// The file system to use.
let fileSystem: FileSystem

/// The test results.
Expand Down