[FLINK-20659][yarn][tests] Add flink-yarn-test README

Author: XComp

flink-yarn-tests/README.md

@@ -0,0 +1,17 @@
+# Flink YARN tests
+
+`flink-yarn-test` collects test cases which are deployed to a local Apache Hadoop YARN cluster. 
+There are several things to consider when running these tests locally:
+
+* `YarnTestBase` spins up a `MiniYARNCluster`. This cluster spawns processes outside of the IDE's JVM 
+  to run the workers on. `JAVA_HOME` needs to be set to make this work.
+* The Flink cluster within each test is deployed using the `flink-dist` binaries. Any changes made 
+  to the code will only take effect after rebuilding the `flink-dist` module.
+* Each `YARN*ITCase` will have a local working directory for resources like logs to be stored. These 
+  working directories are located in `flink-yarn-tests/target/` (see 
+  `find flink-yarn-tests/target -name "*.err" -or -name "*.out"` for the test's output).
+* There is a known problem causing test instabilities due to our usage of Hadoop 2.8.3 executing the 
+  tests. This is caused by a bug [YARN-7007](https://issues.apache.org/jira/browse/YARN-7007) that 
+  got fixed in [Hadoop 2.8.6](https://issues.apache.org/jira/projects/YARN/versions/12344056). See 
+  [FLINK-15534](https://issues.apache.org/jira/browse/FLINK-15534) for further details on the 
+  related discussion.

flink-yarn-tests/src/test/java/org/apache/flink/yarn/YARNSessionCapacitySchedulerITCase.java

@@ -232,6 +232,10 @@ public void perJobYarnCluster() throws Exception {
      * <p>This ensures that with (any) pre-allocated off-heap memory by us, there is some off-heap
      * memory remaining for Flink's libraries. Creating task managers will thus fail if no off-heap
      * memory remains.
+     *
+     * @throws NullPointerException There is a known Hadoop bug (YARN-7007) that got fixed in Hadoop
+     *     2.8.6 but might cause test instabilities. See FLINK-20659/FLINK-15534 for further
+     *     information.
      */
     @Test
     public void perJobYarnClusterOffHeap() throws Exception {
@@ -289,6 +293,10 @@ public void perJobYarnClusterOffHeap() throws Exception {
      *
      * <p><b>Hint: </b> If you think it is a good idea to add more assertions to this test, think
      * again!
+     *
+     * @throws NullPointerException There is a known Hadoop bug (YARN-7007) that got fixed in Hadoop
+     *     2.8.6 but might cause test instabilities. See FLINK-13009/FLINK-15534 for further
+     *     information.
      */
     @Test
     public void
@@ -451,6 +459,9 @@ private static Map<String, String> getFlinkConfig(final String host, final int p
      * Test deployment to non-existing queue & ensure that the system logs a WARN message for the
      * user. (Users had unexpected behavior of Flink on YARN because they mistyped the target queue.
      * With an error message, we can help users identifying the issue)
+     *
+     * @throws NullPointerException There is a known Hadoop bug (YARN-7007) that got fixed in Hadoop
+     *     2.8.6 but might cause test instabilities. See FLINK-15534 for further information.
      */
     @Test
     public void testNonexistingQueueWARNmessage() throws Exception {
@@ -493,6 +504,9 @@ public void testNonexistingQueueWARNmessage() throws Exception {
     /**
      * Test per-job yarn cluster with the parallelism set at the CliFrontend instead of the YARN
      * client.
+     *
+     * @throws NullPointerException There is a known Hadoop bug (YARN-7007) that got fixed in Hadoop
+     *     2.8.6 but might cause test instabilities. See FLINK-15534 for further information.
      */
     @Test
     public void perJobYarnClusterWithParallelism() throws Exception {