RELNOTES: None.
PiperOrigin-RevId: 205665675

RELNOTES: None
PiperOrigin-RevId: 205668909

On two fronts: First, it should follow standard command line semantics. Second, it should work as intended: --noblock_for_lock means the client will not wait for another command to finish, but will exit eagerly. It can be useful for preventing hanging in applications that are non-interactively calling bazel.

It should have standard startup-option semantics: the default value is accepted as a no-op or can be provided to override a previous value.

The next issue involves 2 different locks - the client lock, and the server-side command lock. This duality exists because we would like, one day, to be able to run certain commands, like info or help, at the same time, so multiple commands would need specialized locks that allow some duality but blocks others. This can only be done at the server level, so as soon as the client gets the "we're connected" grpc message from the server, it releases the client lock and lets the server manage  multiple requests.

There are basically 3 possible states that are relevant to this option:

1) no other client is active, so no one holds the client lock or the command lock - the server can be used, shutdown or started as needed. - no blocking, but no need to block, either, so we're safe
2) another client (client1) holds the lock, but it is currently using a server that we want to reuse. If client1 still holds the client lock, we fail fast. Same thing if client1 is holding the server-side lock: we will exit gracefully when the BlazeCommandDispatcher responds with a failure.
3) client1 holds the lock but its server cannot be reused. (batch clients also fall into this category, as there is no server to reuse - but in this case, the client lock is still in play). However, for server mode, this is broken - the following happens:
   - Server is occupied with client1's request, holds the command lock
   - client2 wants to restart the server, so sends the old server a "shutdown" command
   - the BlazeCommandDispatcher says - nuh-uh, this is busy, and you said you didn't want to wait for the lock
   - client2 absorbs this response
   - waits (blocks...)
   - for a minute
   - then force shuts-down the old server.

So we had 2 problems - we block, and we shutdown a server that we truly intended to keep going. Now, if the server responds saying another action is using it, the client will exit correctly, and leave the old server to do its thing.

RELNOTES: None.
PiperOrigin-RevId: 205671817

PiperOrigin-RevId: 205682761

--output graph`.

Implementation:
AIUI, currently the "edges' conditions" are lost [1] when the larger graph is initially constructed. It now does a second pass over dependency subgraph to find all the conditional edges and annotates them in dot output. This can be easily extended in other forms of output, but for now it only annotates edges in dot output.

[1]: https://github.com/bazelbuild/bazel/blob/32e9fee4e2192a340d0b1823538bf8e9fdf92b65/src/main/java/com/google/devtools/build/lib/query2/output/OutputFormatter.java#L745-L770

PiperOrigin-RevId: 205685823

…ce_library is available

This way users of the Skylark API don't see this feature expanded.

RELNOTES: None.
PiperOrigin-RevId: 205704719

…ation in a very specific window of time inbetween enqueueing one top-level node for evaluation and checking if another top-level node is done. See the added unit test for details.

RELNOTES: None
PiperOrigin-RevId: 205718683

PiperOrigin-RevId: 205718853

PiperOrigin-RevId: 205729963

PiperOrigin-RevId: 205751282

…enerated source are generated.

PiperOrigin-RevId: 205756917

Each FileSystem instance has a digest function, but the getDigest and getHashDigest functions also accepted their own custom parameter functions. We only support 1 hash per filesystem instance, these parameters are redundant.

RELNOTES: None.
PiperOrigin-RevId: 205758571

The conversion approach we were previously using is not stable - the resulting
offsets can differ based on what other things are going on on the same machine
at the same time.

By using an interface and passing it to the relevant places (and only computing
the offset once), we ensure that all conversions are consistent with each other.

PiperOrigin-RevId: 205787309

Fixes bazelbuild#5599

PiperOrigin-RevId: 205794997

DeletePath and CreateJunction are now even more
tolerant with errors, particularly the class of
errors where access is denied.

Also in this change:
- remove DeletePathResult::kParentMissing, as this
  case is handled by CreateFileW's error handling
  later
- do not error-check CreateDirectoryW; if failed,
  just proceed as if the directory already existed
- print more debugging info where possible

Change-Id: I1162dae2c6b7524f14d8892047f9eb51831470dd

Closes bazelbuild#5611.

Change-Id: I78fe6aed6d0b120815339c0923c8a903990921d9
PiperOrigin-RevId: 205796307

… as we

treat java_proto_library.

RELNOTES: None
PiperOrigin-RevId: 205812269

Also simplify LoadingPhaseCompleteEvent, and SkyframeExecutor, and remove
LoadingCallback, which is unnecessary now that we only have a single
implementation (previously LoadingPhaseRunner).

This also removes some of the excessive Skyframe calls introduced by
bazelbuild@1067310, and prepares for interleaving target pattern eval and loading.

PiperOrigin-RevId: 205813197

This will be used to compute the critical path using Spawns instead of Actions,
which should be more accurate.

PiperOrigin-RevId: 205817400

3 providers had AbstractCcLinkParamsStore as a class field, now they wrap
CcLinkingInfo instead.

RELNOTES:none
PiperOrigin-RevId: 205821081

RELNOTES: None
PiperOrigin-RevId: 205825362

RELNOTES: java_common.compile creates the native headers jar accesible via JavaInfo.outputs.native_headers.

Closes bazelbuild#5662.

PiperOrigin-RevId: 205832180

… years.

Fixes bazelbuild#5592.

RELNOTES: Deleting deprecated no-op flag --show_package_location
PiperOrigin-RevId: 205834069

Comments are misleading, as discussed on bazelbuild#5305 (comment)

RELNOTES: None.
PiperOrigin-RevId: 205841782

RELNOTES: None.
PiperOrigin-RevId: 205850479

*** Reason for rollback ***

Fixed duplicate derived inputs bug. Test is in diffbase.

RELNOTES[INC]: If the same artifact is generated by two distinct but identical actions, and a downstream action has both those actions' outputs in its inputs, the artifact will now appear twice in the downstream action's inputs. If this causes problems in Skylark actions, you can use the uniquify=True argument in Args.add_args.

PiperOrigin-RevId: 205863806

RELNOTES: The JDK shipped with Bazel was updated to JDK10.
PiperOrigin-RevId: 205865966

PiperOrigin-RevId: 205876673

Truth 0.42 includes Fact which is needed by an upcoming change to
Subject

Change-Id: I791daceed18953f31bd21b184e4349baae5e30ca

*** Reason for rollback ***

Crash with NullPointerException

*** Original change description ***

C++: Remove AbstractCcLinkParamsStore from providers.

3 providers had AbstractCcLinkParamsStore as a class field, now they wrap
CcLinkingInfo instead.

RELNOTES:none
PiperOrigin-RevId: 205885939

…in one of

the declared directories. Especially when doing validation during input
discovery, the discovery's over-approximation can lead to isDeclaredIn walking
the full path to the root without finding a declared include directory.

RELNOTES: None.
PiperOrigin-RevId: 205958078

3 providers had AbstractCcLinkParamsStore as a class field, now they wrap
CcLinkingInfo instead.

This is being rolled forward, first try caused a NullPointerException in
go_wrap_cc rules that had the attribute use_default_import set to False.
GoWrapCcConfiguredTargetTest now has a go_wrap_cc rule that has this attribute
set to false and fails with the previous version of this CL.

RELNOTES:none
PiperOrigin-RevId: 205959266

Truth 0.42 adds a dependency on com.googlecode.java-diff-utils, which
we currently don't have in Bazel. We have to discuss first, whether we
actually want to add that library, too.

This reverts commit 59b044c.

RELNOTES: None.
PiperOrigin-RevId: 205975626

This makes the tests run much faster.
Fixes bazelbuild#5443

RELNOTES: None.
PiperOrigin-RevId: 205980410

For downloading output files / directories we trigger all
downloads concurrently and asynchronously in the background
and after that wait for all downloads to finish. However, if
a download failed we did not wait for the remaining downloads
to finish but immediately started deleting partial downloads
and continued with local execution of the action.

That leads to two interesting bugs:
* The cleanup procedure races with the downloads that are still
in progress. As it tries to delete files and directories, new
files and directories are created and that will often
lead to "Directory not empty" errors as seen in bazelbuild#5047.
* The clean up procedure does not detect the race, succeeds and
subsequent local execution fails because not all files have
been deleted.

The solution is to always wait for all downloads to complete
before entering the cleanup routine. Ideally we would also
cancel all outstanding downloads, however, that's not as
straightfoward as it seems. That is, the j.u.c.Future API does
not provide a way to cancel a computation and also wait for
that computation actually having determinated. So we'd need
to introduce a separate mechanism to cancel downloads.

RELNOTES: None
PiperOrigin-RevId: 205980446

PiperOrigin-RevId: 205980620

There were convoluted cases with multiple JavaInfo created from Skylark that
didn't have the JavaStrictCompilationArgsProvider. If these were to be merged
and later forwarded with exports we would miss them as dependencies.

RELNOTES: None
PiperOrigin-RevId: 205980933

…which

aren't used anymore.

RELNOTES: None.
PiperOrigin-RevId: 205984908

PiperOrigin-RevId: 205985818

The only remaining use was a testing REST backend in the LRE.

I wrote a replacement for that using netty, which we use for our network stuff in Bazel, which means we can now get rid of Hazelcast. :)

I'll remove the Hazelcast files in a separate change when this is merged.

PiperOrigin-RevId: 205985996

Instead, refactor the code to use TargetPatternPhaseValue exclusively. This
removes the need to convert from TargetPatternPhaseValue to LoadingResult, and
prepares for interleaving.

It also reduces the number of Skyframe calls which may speed up null builds a
bit, as a followup for bazelbuild@1067310.

PiperOrigin-RevId: 205989338

Closes bazelbuild#5626.

PiperOrigin-RevId: 205991094

… detected by FileSystemUtils#readWithKnownFileSize.

RELNOTES: None
PiperOrigin-RevId: 205995852

This is in preparation for deleting CcLinkParamsStore. Not all calls to
setCcLinkparamsStore have been removed in this CL.

RELNOTES:none
PiperOrigin-RevId: 205998687

When there is a hard limit on the output of the experimental UI, change
the behavior to only write to the underlying stream on flush. In this
way, we can avoid semantically incomplete writes at the moment we
run out of characters.

Change-Id: I024c776ae2139d76d380bb89d13b8fe61d6cfe51
PiperOrigin-RevId: 206000817

This avoids bazel crashes for illegally formatted strings. Previously the code would assume that a correct string was passed with only minimal validation.

RELNOTES: None
PiperOrigin-RevId: 206012819

…_javabase

to avoid confusion between the LHS and RHS host_javabases.

The LHS --host_javabase option should be considered deprecated and will
eventually be removed.

RELNOTES: Rename the startup flag --host_javabase to --server_javabase to avoid confusion with the build flag --host_javabase
PiperOrigin-RevId: 206015757

Add an option to provide a file with a resolved value, that will be
used to verify that the repositories mentioned in this file produce
a correct directory tree.

RELNOTES: newly added options --experimental_repository_hash_file and
  --experimental_verify_repository_rules allow to verify for repositories
  the directory generated against pre-recorded hashes. See documentation
  for those options.

Work towards bazelbuild#5660.

Change-Id: I2d8becb188d0fa51e890fb8f6139f321cca14b7b
PiperOrigin-RevId: 206016792

This feature sets `-fdebug_prefix_map=$PWD=.` as an argument to Clang
in order to standardize the paths to debug symbols (by making them
relative to the exec root).

Note that this needs to be implemented inside wrapped_clang instead
of as an argument to the action itself for caching purposes, as PWD
can change based on the action's executor.

PiperOrigin-RevId: 206019016

RELNOTES: None

PiperOrigin-RevId: 206021118

PiperOrigin-RevId: 206025645

RELNOTES: none.
PiperOrigin-RevId: 206035420

…k rule.

RELNOTES: none.
PiperOrigin-RevId: 206035881

The cookbook was not maintained and was replaced by https://github.com/bazelbuild/examples

RELNOTES: None.
PiperOrigin-RevId: 206046316

… struct.to_json.

Dictionaries are frequently used for generic configuration descriptions especially
given that `struct` is not allowed in build files.

In order to be JSON-compatible, only dictionaries with string keys are allowed.
Technically Python also allows integers and booleans by automatically converting
them to strings, but this is confusing and not necessarily a good thing to do.

Fixes bazelbuild#5542

Closes bazelbuild#5543.

PiperOrigin-RevId: 206049754

`keyWordArgs` is already `ImmutableMap<String, Object>` and `copyOf` contains enough
overhead to show up on parse profiles.

Closes bazelbuild#5651.

PiperOrigin-RevId: 206051798

*** Reason for rollback ***

Crash (see linked bug).

PiperOrigin-RevId: 206060905

RELNOTES: None
PiperOrigin-RevId: 206063489

… of Param annotations only once.

This change has been manually verified to greatly reduce analysis time (~50%) on very large builds, as it mitigates a previous regression brought on by the migration of @SkylarkSignature to @SkylarkCallable.

RELNOTES: None.
PiperOrigin-RevId: 206063684

PiperOrigin-RevId: 206085005

RELNOTES: Removed the gen_jars output group
PiperOrigin-RevId: 206092599

PiperOrigin-RevId: 206102499

…figuration

This allows loading data inputs to annotation processors and Error Prone plugins.
Previously the only supported way to do this was using Java resources, but in
the case of per-package configuration that ties any changes to the resources
to JavaBuilder's release process.

PiperOrigin-RevId: 206106802

This is in preparation for deleting CcLinkParamsStore. Not all calls to
setCcLinkparamsStore have been removed in this CL.

Roll forward with bzl change in separate CL (unknown commit) and giving a proper error in Skylark instead of a crash when CcLinkingInfo is not built correctly.

RELNOTES:none
PiperOrigin-RevId: 206122870

PiperOrigin-RevId: 206124597

This is in preparation for deleting CcLinkParamsStore. All remaining calls
to the setCcLinkparamsStore method of the CcLinkingInfo builder have been removed.

RELNOTES:none
PiperOrigin-RevId: 206128439

PiperOrigin-RevId: 206130934

This significantly reduces the size of the resulting profiles. Do we need the
events that I remove? What do we do with the data? Who knows?

PiperOrigin-RevId: 206153368

Also update the Java Tutorial page to say you have
to install the JDK and set JAVA_HOME.

See bazelbuild#5208

Change-Id: I14b792589c39f03572518aaff6f6f8e5ca8a65c8

Closes bazelbuild#5617.

Change-Id: I45c841189b0c220365ab9aeeb67f26b32059be4a
PiperOrigin-RevId: 206156529

RELNOTES: None.
PiperOrigin-RevId: 206156886

This is accomplished by:

- Setting the APPLE_CROSSTOOL configuration distinguisher in AppleCrosstoolTransition
- Doing nothing in AppleCrosstoolTransition in case we are already in an Apple configuration so that funky use cases like a binary having a dependency on a swift_library both directly and through an objc_library work
- Adding the "apl-" prefix to the output directory name in APPLE_CROSSTOOL configurations

Plus a few minor cleanups:

- Removed some unused methods
- Nopped out --enable_apple_crosstool_transition if the new flag is set
- Nopped out --target_uses_apple_crosstool if the new flag is set

These latter reduce the possible space of Apple configurations, thus making the code a bit more comprehensible.

RELNOTES: None.
PiperOrigin-RevId: 206157413

RELNOTES: None
PiperOrigin-RevId: 206157591

Instead of using the default thread pool size of 200, use the number set for
the loading phase. This is in preparation for interleaving the loading and
target pattern eval phases.

PiperOrigin-RevId: 206172915

PiperOrigin-RevId: 206182289

Truth 0.42 includes Fact which is needed by an upcoming change to
Subject

Adds diffutils which is needed by truth >= 0.40

Changed filename of diffutils jar to include version number

Diffutils jar from http://repo1.maven.org/maven2/com/googlecode/java-diff-utils/diffutils/1.3.0/

See also https://code.google.com/archive/p/java-diff-utils/

Change-Id: Iecd60c921c2562750d1075e0afbb7777e6b86ed2

Similar to the sequence number, the timestamp should be immutable and computed when the event is enqueued (not at serialization time).

PiperOrigin-RevId: 206186036

RELNOTES: None.
PiperOrigin-RevId: 206195789

PiperOrigin-RevId: 206195967

PiperOrigin-RevId: 206203357

Add a missing indefinite article

Closes bazelbuild#5673.

PiperOrigin-RevId: 206203662

…ommand

as a list, rather than as a single line (i.e., newline delimited rather than
space delimited).

Before:

SUBCOMMAND: # //src/main/java/com/google/devtools/build/lib:string_util [action 'Building src/main/java/com/google/devtools/build/lib/libstring_util.jar (2 source files) [for host]']
(cd /tmp/devbazel_output_base/execroot/io_bazel && \
  exec env - \
    LC_CTYPE=en_US.UTF-8 \
  external/embedded_jdk/bin/java -XX:+UseParallelOldGC -XX:-CompactStrings '--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED' '--add-opens=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED' '--patch-module=java.compiler=external/bazel_tools/third_party/java/jdk/langtools/java_compiler.jar' '--patch-module=jdk.compiler=external/bazel_tools/third_party/java/jdk/langtools/jdk_compiler.jar' '--add-opens=java.base/java.nio=ALL-UNNAMED' -jar external/bazel_tools/tools/jdk/JavaBuilder_deploy.jar @bazel-out/host/bin/src/main/java/com/google/devtools/build/lib/libstring_util.jar-2.params)

After:

SUBCOMMAND: # //src/main/java/com/google/devtools/build/lib:string_util [action 'Building src/main/java/com/google/devtools/build/lib/libstring_util.jar (2 source files) [for host]']
(cd /tmp/devbazel_output_base/execroot/io_bazel && \
  exec env - \
    LC_CTYPE=en_US.UTF-8 \
  external/embedded_jdk/bin/java \
    -XX:+UseParallelOldGC \
    -XX:-CompactStrings \
    '--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED' \
    '--add-exports=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED' \
    '--add-exports=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED' \
    '--add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED' \
    '--add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED' \
    '--add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED' \
    '--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED' \
    '--add-opens=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED' \
    '--patch-module=java.compiler=external/bazel_tools/third_party/java/jdk/langtools/java_compiler.jar' \
    '--patch-module=jdk.compiler=external/bazel_tools/third_party/java/jdk/langtools/jdk_compiler.jar' \
    '--add-opens=java.base/java.nio=ALL-UNNAMED' \
    -jar \
    external/bazel_tools/tools/jdk/JavaBuilder_deploy.jar \
    @bazel-out/host/bin/src/main/java/com/google/devtools/build/lib/libstring_util.jar-2.params)

RELNOTES: --subcommands can now take a "pretty_print" value ("--subcommands=pretty_print") to print the
arguments of subcommands as a list for easier reading.
PiperOrigin-RevId: 206213009

This is just a style fix so it should have no functional changes.

RELNOTES: None.
PiperOrigin-RevId: 206220348

This forces the ex-default to be explicit in a lot of tests, but I'd rather that than have the risk of implicit md5-use in production code.

To keep this CL smaller, do not remove the default from UnixFS quite yet.

RELNOTES: None.
PiperOrigin-RevId: 206223521

RELNOTES: None.
PiperOrigin-RevId: 206224046

… been used at the start of execution.

PiperOrigin-RevId: 206230049

Previously, only trivial relative paths (within the same package) were handled correctly. Now paths such as ":foo/bar.bzl" are handled appropriately.

RELNOTES: None.
PiperOrigin-RevId: 206237161

…classpath is

non-empty but there are no annotation processors to run.

PiperOrigin-RevId: 206244327

…utput

and just skip the diagnostics at the end if strict deps errors are disabled.

This is necessary to javac-turbine, where we don't report strict deps errors
(we rely on JavaBuilder for enforcement) but where we still want to emit
jdeps.

See also bazelbuild/rules_scala#559

PiperOrigin-RevId: 206257304

Reporting strict deps errors in addition to attribution errors is helpful
particularly with code-generating annotation processors that may fail due
to missing transitive deps. Currently those failures prevent add_dep
suggestions from being emitted, making the problem harder to diagnose.

The presence of other errors may prevent some SJD issues from being detected,
but in theory should never lead to spurious SJD errors from being reported,
so the additional errors should generally be helpful.

PiperOrigin-RevId: 206261677

This was missed in earlier patch.

RELNOTES: None.
PiperOrigin-RevId: 206276771

RELNOTES:none
PiperOrigin-RevId: 206287557

…tatus

When the experimental UI is asked to limit its overall output, reserve a certain
fraction for output to be generated after the build is complete. In this way,
at least a final status, or a bit of the test summary, is shown.

Also, to help staying within the limit also for small limits, from a certain
threshold onwards, more severely restrict the output we allow for each individual
action.

Change-Id: I912aec0dd17639d9864133a10359f93537b473ad
PiperOrigin-RevId: 206288651