Skip to content

[Dependency Scanning] Configure a serialized diagnostics consumer for in-memory scans #84191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 15, 2025
Merged

[Dependency Scanning] Configure a serialized diagnostics consumer for in-memory scans #84191

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c820485
[Dependency Scanning] Configure a thread-safe serialized diagnostics …
artemcm Sep 9, 2025
b6760e7
Add new-driver-only flag for dependency scanning serialized diagnosti…
artemcm Sep 9, 2025
50c0dd8
[Dependency Scanning] Refactor 'createScanQueryContext' to directly r…
artemcm Sep 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 52 additions & 13 deletions include/swift/DependencyScan/DependencyScanningTool.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,20 +15,50 @@

#include "swift-c/DependencyScan/DependencyScan.h"
#include "swift/Frontend/Frontend.h"
#include "swift/AST/DiagnosticConsumer.h"
#include "swift/AST/ModuleDependencies.h"
#include "swift/DependencyScan/ScanDependencies.h"
#include "swift/Frontend/PrintingDiagnosticConsumer.h"
#include "swift/Frontend/SerializedDiagnosticConsumer.h"
#include "llvm/Support/Error.h"
#include "llvm/Support/StringSaver.h"

namespace swift {
namespace dependencies {
class DependencyScanningTool;
class DependencyScanDiagnosticCollector;
class DepScanInMemoryDiagnosticCollector;

struct ScanQueryInstance {
struct ScanQueryContext {
/// Primary CompilerInstance configured for this scanning action
std::unique_ptr<CompilerInstance> ScanInstance;
std::shared_ptr<DependencyScanDiagnosticCollector> ScanDiagnostics;
/// An thread-safe diagnostic consumer which collects all emitted
/// diagnostics in the scan to be reporte via libSwiftScan API
std::unique_ptr<DepScanInMemoryDiagnosticCollector> InMemoryDiagnosticCollector;
/// A thread-safe serialized diagnostics consumer.
/// Note, although type-erased, this must be an instance of
/// 'ThreadSafeSerializedDiagnosticConsumer'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you need to type-erase to the base type. This can just be ThreadSafeSerializedDiagnosticConsumer

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only did so because the existing SerializedDiagnosticConsumer infrastructure, and therefore also my extension, are both defined in SerializedDiagnosticConsumer.cpp implementation file. We could hoist them up to be in a header, but I wanted to match the existing pattern.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like ThreadSafeDiagnosticConsumer can be together with DiagnosticConsumer and we can just use this type here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this comment, could you please be more specific?

ThreadSafeSerializedDiagnosticConsumer inherits from SerializedDiagnosticConsumer. The latter is implementation-only (defined in a .cpp). I modeled the former to match. What does "can be together with DiagnosticConsumer " mean?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean ThreadSafeDiagnosticConsumer that inherits from DiagnosticConsumer. We only need to make sure it is a ThreadSafeDiagnosticConsumer so we can add to scan instance. I don't see any upcast is needed to add here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you referring to ThreadSafeDiagnosticCollector? Are you suggesting that ThreadSafeSerializedDiagnosticConsumer inherit both ThreadSafeDiagnosticCollector and SerializedDiagnosticConsumer?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, ok, I don't want to have a diamond here. Maybe I will model it as ThreadSafeDiagnosticConsumer with an internal SerializedDiagnosticConsumer, but you can't finish processing like that.

I will let you decide if you want to do it this way. Otherwise LGTM.

std::unique_ptr<DiagnosticConsumer> SerializedDiagnosticConsumer;

ScanQueryContext(
std::unique_ptr<CompilerInstance> ScanInstance,
std::unique_ptr<DepScanInMemoryDiagnosticCollector>
InMemoryDiagnosticCollector,
std::unique_ptr<DiagnosticConsumer> SerializedDiagnosticConsumer)
: ScanInstance(std::move(ScanInstance)),
InMemoryDiagnosticCollector(std::move(InMemoryDiagnosticCollector)),
SerializedDiagnosticConsumer(std::move(SerializedDiagnosticConsumer)) {}

ScanQueryContext(ScanQueryContext &&other)
: ScanInstance(std::move(other.ScanInstance)),
InMemoryDiagnosticCollector(
std::move(other.InMemoryDiagnosticCollector)),
SerializedDiagnosticConsumer(
std::move(other.SerializedDiagnosticConsumer)) {}

~ScanQueryContext() {
if (SerializedDiagnosticConsumer)
SerializedDiagnosticConsumer->finishProcessing();
}
};

/// Pure virtual Diagnostic consumer intended for collecting
Expand All @@ -47,12 +77,16 @@ class ThreadSafeDiagnosticCollector : public DiagnosticConsumer {
};

/// Diagnostic consumer that simply collects the diagnostics emitted so-far
class DependencyScanDiagnosticCollector : public ThreadSafeDiagnosticCollector {
private:
/// and uses a representation agnostic from any specific CompilerInstance state
/// which may have been used to emit the diagnostic
class DepScanInMemoryDiagnosticCollector
: public ThreadSafeDiagnosticCollector {
public:
struct ScannerDiagnosticInfo {
std::string Message;
llvm::SourceMgr::DiagKind Severity;
std::optional<ScannerImportStatementInfo::ImportDiagnosticLocationInfo> ImportLocation;
std::optional<ScannerImportStatementInfo::ImportDiagnosticLocationInfo>
ImportLocation;
};
std::vector<ScannerDiagnosticInfo> Diagnostics;

Expand All @@ -61,9 +95,12 @@ class DependencyScanDiagnosticCollector : public ThreadSafeDiagnosticCollector {

public:
friend DependencyScanningTool;
DependencyScanDiagnosticCollector() {}
DepScanInMemoryDiagnosticCollector() {}
void reset() { Diagnostics.clear(); }
const std::vector<ScannerDiagnosticInfo> &getDiagnostics() const {
std::vector<ScannerDiagnosticInfo> getDiagnostics() const {
return Diagnostics;
}
const std::vector<ScannerDiagnosticInfo> &getDiagnosticsRef() const {
return Diagnostics;
}
};
Expand Down Expand Up @@ -97,10 +134,10 @@ class DependencyScanningTool {

/// Using the specified invocation command, instantiate a CompilerInstance
/// that will be used for this scan.
llvm::ErrorOr<ScanQueryInstance>
initCompilerInstanceForScan(ArrayRef<const char *> Command,
StringRef WorkingDirectory,
std::shared_ptr<DependencyScanDiagnosticCollector> scannerDiagnosticsCollector);
llvm::ErrorOr<ScanQueryContext> createScanQueryContext(
ArrayRef<const char *> Command, StringRef WorkingDirectory,
std::vector<DepScanInMemoryDiagnosticCollector::ScannerDiagnosticInfo>
&initializationDiagnostics);

private:
/// Shared cache of module dependencies, re-used by individual full-scan queries
Expand All @@ -113,7 +150,9 @@ class DependencyScanningTool {
llvm::StringSaver Saver;
};

swiftscan_diagnostic_set_t *mapCollectedDiagnosticsForOutput(const DependencyScanDiagnosticCollector *diagnosticCollector);
swiftscan_diagnostic_set_t *mapCollectedDiagnosticsForOutput(
ArrayRef<DepScanInMemoryDiagnosticCollector::ScannerDiagnosticInfo>
diagnostics);

} // end namespace dependencies
} // end namespace swift
Expand Down
9 changes: 4 additions & 5 deletions include/swift/DependencyScan/ScanDependencies.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ using ModuleDependencyIDSet =
class SwiftDependencyScanningService;

namespace dependencies {
class DependencyScanDiagnosticCollector;
struct ScanQueryContext;

using CompilerArgInstanceCacheMap =
llvm::StringMap<std::tuple<std::unique_ptr<CompilerInstance>,
Expand All @@ -62,16 +62,15 @@ bool prescanDependencies(CompilerInstance &instance);
/// Scans the dependencies of the main module of \c instance.
llvm::ErrorOr<swiftscan_dependency_graph_t>
performModuleScan(SwiftDependencyScanningService &service,
CompilerInstance &instance,
ModuleDependenciesCache &cache,
DependencyScanDiagnosticCollector *diagnostics = nullptr);
ScanQueryContext &queryContext);

/// Scans the main module of \c instance for all direct module imports
llvm::ErrorOr<swiftscan_import_set_t>
performModulePrescan(SwiftDependencyScanningService &service,
CompilerInstance &instance,
ModuleDependenciesCache &cache,
DependencyScanDiagnosticCollector *diagnostics = nullptr);
ScanQueryContext &queryContext);


namespace incremental {
/// For the given module dependency graph captured in the 'cache',
Expand Down
9 changes: 9 additions & 0 deletions include/swift/Frontend/SerializedDiagnosticConsumer.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,15 @@ namespace swift {
/// \returns A new diagnostic consumer that serializes diagnostics.
std::unique_ptr<DiagnosticConsumer>
createConsumer(llvm::StringRef outputPath, bool emitMacroExpansionFiles);

/// Create a thread-safe DiagnosticConsumer that serializes diagnostics to a file,
/// using Clang's serialized diagnostics format.
///
/// \param outputPath the file path to write the diagnostics to.
///
/// \returns A new diagnostic consumer that serializes diagnostics.
std::unique_ptr<DiagnosticConsumer>
createThreadSafeConsumer(llvm::StringRef outputPath, bool emitMacroExpansionFiles);
}
}

Expand Down
7 changes: 7 additions & 0 deletions include/swift/Option/Options.td
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2107,6 +2107,13 @@ def emit_module_serialize_diagnostics_path:
"<path>">,
MetaVarName<"<path>">;

def dependency_scan_serialize_diagnostics_path:
Separate<["-"], "dependency-scan-serialize-diagnostics-path">,
Flags<[ArgumentIsPath, SupplementaryOutput, NewDriverOnlyOption]>,
HelpText<"Emit a serialized diagnostics file for the dependency scanning task to "
"<path>">,
MetaVarName<"<path>">;

def emit_module_dependencies_path:
Separate<["-"], "emit-module-dependencies-path">,
Flags<[ArgumentIsPath, SupplementaryOutput, NewDriverOnlyOption]>,
Expand Down
Loading