They have apparently deleted the SNAPSHOT jar and now builds are failing.

The only way it's safe to split a compressed file is if the file is not compressed. This can
only happen when the source itself is splittable, and that in turn will result in the inner
source's reader being returned. A CompressedReader will only be created in the event that
the file is NOT splittable. So remove all the logic handling splittable compressed readers,
and instead go with the logic when we know/assume the file is compressed.

* TextIO: test compression with larger files

It is important for correctness that we test with large files
because otherwise the compressed file may be larger than the
uncompressed file, which could mask bugs

* TextIOTest: flesh out more

* TextIOTest: add large uncompressed file

If the test hangs due to bugs, the infrastructure should kill it.

If provided with an Unbounded PCollection, Write will fail due to
restriction of calling finalize only once. This error message fails in a
deep stack trace based on it not being possible to apply a GroupByKey.
Instead, throw immediately on application with a specific error message.

These writes should be forbidden based on the boundedness of the input
PCollection. As Write explicitly forbids the application of the
transform to an Unbounded PCollection, this will be equivalent in most
cases; In cases where the input PCollection is Bounded, due to an
UnboundedReadFromBoundedSource, the write will function as expected and
does not need to be forbidden.

Aggregator is the model level concept. Counter was specific to the
Dataflow Runner, and is now not needed as part of Beam.

This cleans up any state stored within the Transform Evaluator Factory.

This is a more focused interface that interacts with a DoFn before it
is available for use and after it has completed and the reference is
lost. It is required to properly support setup and teardown, as the
fields in a ThreadLocal cannot all be cleaned up without additional
tracking.

Part of BEAM-452.

Methods annotated with these annotations are used to perform expensive
setup work and clean up a DoFn after another method throws an exception
or the DoFn is discarded.

These tests are not yet functional in all runners, and this makes them
easier to ignore.

Until we implement it for Dataflow runner.

Previously, guava-testlib guava-testlib was being relocated as part of
the shading process, but test-scope dependencies aren't bundled in the
uber-jar. As a result, the output JAR was unusable without recreating the
same shading rules in a consuming project.

It's the only unsynchronized function, and it could potentially produce bad data.

DoFnSignature encapsulates type information about a DoFn,
in particular which arguments/features its methods
actually use.

Before this commit, DoFnReflector would parse/verify/generate
code in one go; after this commit, these stages are separated:
DoFnSignature encapsulates all information needed to generate
the code.

Additionally, removes the unnecessary genericity in the
implementation of DoFnReflector's code generation for the
very different methods processElement and start/finishBundle.
The code is simpler if decomposed into utility functions,
rather than attempting a uniform representation for different
methods.

There are a number of places in the Java SDK where we use
ParDo.of(DoFn) when MapElements or other higher-level
composites are applicable and readable. This change
alters a number of those.

This is for use in asserting the contents of speculative panes in the
global window.

This is a source suitable for use with tests that have interesting
triggering behavior. It is an Unbounded source that emits elements in
bundles, and advances the watermark and processing time appropriately.

This generalizes Write to Write and Delete cleanly.

This corrects the double-checked locking and lazy intialization

…ies.

Also support excluding runners by disabling the include-runners profile.

Tested ITs
BQ and WordCount pass on Dataflow
WordCount passes on Flink and Spark

These timers should not be fired - the windows should be expired via the
GC timer, and any elements should be emitted if neccessary.

This reenables the lifecycle tests now that they are properly supported.

Update the container image.

This also replaces the custom Flink StateInternals by proper Flink
Partitioned StateInternals.

This adds a generic SideInputHandler in runners-core that is only used
by the Flink runner right now but can be used by other runner
implementations.

These tests were not written in such a way as to succeed if the trigger
fired multiple times.

This ensures that tests with triggering in the global window which assert
on the entire PCollection (not a singleton iterable) will succeed over the
entire PCollection

BigQuery has changed their total size quota to 12 TiB.
https://cloud.google.com/bigquery/quota-policy#import

In the worst case scenario for random key assignment in
Write.ApplyShardingKey, the chance of the number of records per output
shard was too high. This makes the test significantly less likely to
flake.

Apparently the IllegalImport check only blocks packages,
so we had to move to Regexp to get individual classes.

As a bonus, this enforcement let us remove two bogus dependencies.
Smaller JARs for the win!

…rtBundle/finishBundle

Will still update releases with the --update-snapshots version (I know, bad name)

Fixes an invalid cast that breaks some PCollectionList-related tests.

This test exercises the PTransforms that make up the LeaderBoard
example. This includes speculative and late trigger firings to produce
team and individual scores on a global and fixed window basis.

Refactor LeaderBoard to expose the team and user score calculations as
composite PTransforms to enable this testing.

IDE over-eagerly replaced some occurrences of createAggregator with
createAggregatorForDoFn. This corrects that.

Add KeyedResourcePool

This interface represents some shared pool of values that may be used by
at most one caller at a time.

Add LockedKeyedResourcePool which has at most one value per key and
at most one user per value at a time.

Use KeyedResourcePool in TestStream