Skip to content

Conversation

@yanghua
Copy link
Contributor

@yanghua yanghua commented May 28, 2018

What is the purpose of the change

This pull request fixed flink 1.5 release document version title

Brief change log

  • changed version title from 1.5-SNAPSHOT to 1.5

Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (yes / no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
  • The serializers: (yes / no / don't know)
  • The runtime per-record code paths (performance sensitive): (yes / no / don't know)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know)
  • The S3 file system connector: (yes / no / don't know)

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

Nico Kruber and others added 30 commits April 19, 2018 10:28
…ould always be performed in cleanup.

This step should be independent from the fact if the backend is still registered with the closeable registry.

(cherry picked from commit 777cc1a)
…to avoid problem with concurrent directory creation.

(cherry picked from commit a455d6a)
…nting from 1 for index 0

(cherry picked from commit 1bfe87f)
…k to prevent data loss

This change replaces Hadoop's LocalFileSystem (which is a checksumming filesystem) with the RawFileSystem implementation. For performing checksums the default filesystem only flushes in 512 byte intervals which might lead to data loss during checkpointing. In order to guarantee exact results we skip the checksum computation and perform a raw flush.

Negative effect: Existing checksums are not maintained anymore and thus become invalid.

This closes apache#5861.
…opy of default/registered serializer instances.

This method did create deep copies of registered or default serializer instances and
as a result those serializer instances can accidentally be shared across different threads.

This closes apache#5880.

(cherry picked from commit 7d0bfd5)
Gets rid of unnecessary boxing and multiplications/divisions that Duration does internally.
Prevents the class from dropping the TypeInformation, adds a convenience constructor.
…extract generic types.

This results in better error messages.
…partitionTest#testConsumeSpilledPartitionSpilledBeforeAdd

This closes apache#5882
Releasing an operator chain's outputs will call RecordWriter#clearBuffers() and
this may not be run in parallel with RecordWriter#broadcastEvent() which the
asynchronous checkpoint barrier trigger inside Task may run via
StreamTask#triggerCheckpoint(). Now, during the cleanup of StreamTask#invoke(),
StreamTask's asynchronous services are shut down but not those of the Task and
also, operatorChain.releaseOutputs() was not put under the checkpoint lock.

This commit adds the checkpoint lock here and we should be safe to do so since
we already closed all of StreamTask's asynchronous executors and also disposed
the operators and hence nothing should be blocking the cleanup by holding the
checkpoint lock.

This closes apache#5748
tweise and others added 26 commits May 23, 2018 00:59
… region or endpoint.

Fix validation logic to allow either region or endpoint, but not both.

This closes apache#6045.
This commit changes the behaviour of the JM to always try to reconnect to the latest known RM address.

This closes apache#6056.
Use equals instead of referential equality in JobMaster#requestPartitionState
when comparing the producerExecution attempt id with the result partition
producer id.

This closes apache#6057.
…es from LICENSE

The jline-reader and jline-terminal are dependencies against which we link.
They don't need to be included in LICENSE, because LICENSE only contains code
which is contained in the source release of Flink.
…ryAndLoopFetching test

This commit improves the said test to fail consistently without fixing
concurrent modifications on the partition states list.

This closes apache#6040.
…ched write performance in RocksDB backend.

This closes apache#5650.

(cherry picked from commit 1c7341a)
…anceTest.benchMark().

This closes apache#6063.

(cherry picked from commit b485f8c)
Remove the generation of MD5 checksum files and create sha checksum
files with a sha512 file ending.

This closes apache#6061.
…Executor

This commit fixes a race condition between the TaskExecutor and the ResourceManager. Before,
it could happen that the ResourceManager sends requestSlots message before the TaskExecutor
registration was completed. Due to this, the TaskExecutor did not have all information it needed
to accept task submissions.

The problem was that the TaskExecutor sent the SlotReport at registration time. Due to this, t
he SlotManager could already assign these slots to pending slot requests. With this commit, the
registration protocol changes such that the TaskExecutor first registers at the ResourceManager
and only after completing this step, it will announce the available slots to the SlotManager.

This closes apache#6067.
…rminal state

This commit lets the Dispatcher remove the RunningJobsRegistry entry for a completed job
when it is removed from the Dispatcher.

This closes apache#6068.
This commit changes the RestClusterClient calls such that they are all retriable wrt
to connection errors and if the service is currently unavailable (return code 503).

Moreover, it changes the retry behaviour for polling the JobResult such that it fails
now if the cluster returns a NOT_FOUND code.

This closes apache#6069.
…roblems

See FLINK-9437 for more information.
…ure mode"

This reverts commit e7e1bac.

Netty's SslHandler v4.0.27 requires to allocate heap buffers when using a GCM enabled
cypher suite. Since we don't allow this, we have to use the old cypher algorithm
TLS_RSA_WITH_AES_128_CBC_SHA.
@yanghua yanghua changed the title [Flink-9452] Flink 1.5 document version title shows snapshot [FLINK-9452] Flink 1.5 document version title shows snapshot May 28, 2018
@yanghua yanghua closed this May 28, 2018
@yanghua yanghua deleted the FLINK-9452 branch May 28, 2018 05:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.