[FLINK-1679] rename degree of parallelism to parallelism & extend documentation about parallelism

[mxm]

https://issues.apache.org/jira/browse/FLINK-1679

[uce]

Very nice and exhaustive change.

One question though: do we want to just rename configuration parameters and methods like that or should we keep the old config keys (parallelization.degree.default) and the old config methods (setDegreeOfParallelism) as well and deprecate them first?

I think I am in favour of option 2.

[StephanEwen]

I think this is very API breaking - in my opinion, we should keep the old methods where it breaks the API, or at least keep them for one more version (deprecated).

That does not prevent us from merging the fixes to comments and docs.

[mxm]

@uce @StephanEwen I agree with you two that it's better to keep the old API methods and mark them as deprecated while introducing the new ones. When the API has been stabilized and 1.0 comes up, we can remove the old methods.

[mxm]

I've updated the pull request to include deprecation of the setters/getters and the config parameter.

[rmetzger]

Thank you for the huge change! Our users will appreciate the more concise naming and improved documentation.

+1 to merge

[mxm]

Any objections against merging this?

[StephanEwen]

Are we sure that this is breaking no existing API. No method on DataSet or any descendant of it has been renamed? Also not on other user facing types?

[StephanEwen]

With 236 files, that is hard to assess...
That is a point, in general, with humongous pull requests that touch 10 projects at the same time.

I would personally like another +1, from someone who verified that this is not API breaking. But again, PRs of this size are not easily reviewed...

[mxm]

Let me explain how I made out the relevant API facing methods to deprecate. Since I changed all calls to set the parallelism to getParallelism or setParallelism I did not have to change classes which already used these methods. To find the remaining which used setDegreeOfParalleism or getDegreeOfParalleism method, I did a regexp search in the Java and Scala classes:

Java: public .* (get|set)DegreeOfParallelism

ExecutionConfig
Operator
CollectionEnvironment
ExecutionEnvironment
StreamExecutionEnvironment
SpargelIteration

Scala: def (get|set)DegreeOfParallelism

ExecutionEnvironment
StreamExecutionEnvironment

In these classes I deprecated the methods, and made them call the new methods getParallelism and setParallelism. That's how I verified this pull request is not API breaking. For you inspection, I've created a separate commit with these changes. The deprecated methods are also covered in the documentation about how to change the parallelism: http://ci.apache.org/projects/flink/flink-docs-master/programming_guide.html#parallel-execution

[mxm]

Any other opinions on the API breaking nature of these changes?

[StephanEwen]

I think all changes that are in flink-core, flink-java, flink-scala, flink-streaming, flink-spargel are critical. For all those changes, there should be an additional method and the current one should be deprecated.

Other changes (Runtime, Optimizer, tests) can be made without any problem.

NOTE: You are also breaking the system configuration. We should change it to use the new parameter, but we should still accept the config values of the old parameter.

[mxm]

@StephanEwen That's exactly what I did.

[mxm]

Merged in master with 126f9f7