Feature to opt out of tracking .swiftdoc and .swiftsourceinfo Files

[chiragramani]

This PR introduces configuration options aimed at optimizing CI build processes by excluding non-essential Swift documentation and source info files, files that don't serve as inputs to other build targets. By introducing flags to control the exclusion of these files, we aim to reduce unnecessary network I/O and improve build performance in specific CI environments. (non-dev facing only)

Features Introduced:

• swift.emit_swiftdoc

  • This feature is enabled by default.
  • Opting out of this feature will prevent .swiftdoc files from being tracked by SwiftInfo provider, though they will still be generated by the Swift compiler.
  • These documentation files are generally used for IDE features and are not required in certain CI contexts, such as non-developer-facing validation suites.

• swift.emit_swiftsourceinfo

  • This feature is enabled by default.
  • When this feature is disabled, .swiftsourceinfo files, which support source-level debugging, will also be excluded from build outputs and will not be tracked by SwiftInfo provider.

It is important to note that, by default, the Swift compiler generates .swiftdoc and .swiftsourceinfo files as part of the output when using the -emit-module flag. As of now, this behavior is consistent, given the absence of compiler options in Swift that permit controlling the generation of these files.

[brentleyjones]

Can you also update the PR description with the changes to names and behavior? It will be used for the body of the merged commit.

[chiragramani]

Can you also update the PR description with the changes to names and behavior? It will be used for the body of the merged commit.

Oh yes, updated!

[brentleyjones]

Ohh, also, do we need to adjust anything in the worker for incremental generation?

[jszumski]

we aim to reduce unnecessary network I/O and improve build performance in specific CI environments

@chiragramani what was the rough magnitude of the performance win?

[chiragramani]

Ohh, also, do we need to adjust anything in the worker for incremental generation?

I tried out the batch mode feature, and the builds worked fine! But I might be missing some scenarios. Can you suggest other tests for incremental generation? I'd like to check those out too.

we aim to reduce unnecessary network I/O and improve build performance in specific CI environments

@chiragramani what was the rough magnitude of the performance win?

I tested this for a smaller set to see the impact when there are no cache-hits.
Without excludes: ~3.6GB uploaded, ~590 seconds.
With excludes: ~3.2Gb uploaded, ~550 seconds.

Also, we are not using experimental_remote_cache_async which might explain some of the build time delta (issues with HTTP cache) but maybe this delta would be a lot smaller/negligible for environments making use of experimental_remote_cache_async.

[brentleyjones]

Can you suggest other tests for incremental generation? I'd like to check those out too.

There is the incremental_inputs stuff in output_file_map.cc, since since we are always generating these files, it probably doesn't matter. Thanks for checking.