Successful tests marked as "terminated" in IntelliJ

[colin-williams]

Hi,

I'm puzzled why I'm getting Tests passed: 0 but not explicitly failing from my tests when running via Idea.

I made a simple pom and I seem to pass the examples in another project, but my custom tests aren't passing. It seems the assertions are as expected and the tables are created. Then for some reason some part of the test setup or annotations is not passing my tests. Can someone get me back on track?

Thanks

package com.test.scno.hiveunit.location;

import com.google.common.collect.Sets;
import com.klarna.hiverunner.HiveShell;
import com.klarna.hiverunner.StandaloneHiveRunner;
import com.klarna.hiverunner.annotations.HiveProperties;
import com.klarna.hiverunner.annotations.HiveResource;
import com.klarna.hiverunner.annotations.HiveRunnerSetup;
import com.klarna.hiverunner.annotations.HiveSQL;
import com.klarna.hiverunner.annotations.HiveSetupScript;
import com.klarna.hiverunner.config.HiveRunnerConfig;
import org.apache.commons.collections.MapUtils;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;

import java.io.File;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;


@RunWith(StandaloneHiveRunner.class)
public class SomeTableTest {

    @HiveRunnerSetup
    public final HiveRunnerConfig CONFIG = new HiveRunnerConfig(){{
        setHiveExecutionEngine("tez");
    }};


    @HiveProperties
    public Map<String, String> hiveProperties = MapUtils.putAll(new HashMap(), new Object[]{
            "MY.HDFS.DIR", "${hadoop.tmp.dir}",
            "my.schema", "SOME_DB",
            "my.schema2", "ANOTHER_DB"
    });

    @HiveSetupScript
    private String createSchemaScript = "create schema ${hiveconf:my.schema}"+";" +"create schema ${hiveconf:my.schema2}";

    @HiveResource(targetFile = "${hiveconf:MY.HDFS.DIR}/data/in/data.gz")
    private File dataFromFile = new File(ClassLoader.getSystemResource("data/location/SOME_DB/SOME_TABLE/data.gz").getPath());


    @HiveSQL(files = {"schemas/hive/ANOTHER_DB/SOME_TABLE.sql"}, encoding = "UTF-8")
    private HiveShell hiveShell;

    @Test
    public void testTablesContainsItems() {
        List<String> sourceTableResult = hiveShell.executeQuery("SELECT * FROM SOME_DB.SOME_TABLE LIMIT 10");
        List<String> targetTableResult = hiveShell.executeQuery("SELECT * FROM ANOTHER_DB.ANOTHER_TABLE LIMIT 10");
        System.out.println(sourceTableResult.size());
        System.out.println(targetTableResult.size());
        Assert.assertTrue(sourceTableResult.size()==10);
        Assert.assertTrue(targetTableResult.size()==10);
    }
}

<?xml version="1.0"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.test.scno</groupId>
  <artifactId>scno-hive-unit</artifactId>
  <version>1.0.0</version>
  <name>${project.artifactId}</name>
  <properties>
   <hiverunner.version>4.0.0</hiverunner.version>
    <encoding>UTF-8</encoding>
    <junit.version>4.12</junit.version>
    <!-- Plugins -->
    <surefire.plugin.version>2.21.0</surefire.plugin.version>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
  </properties>
  <dependencies>
    <dependency>
      <groupId>com.klarna</groupId>
      <artifactId>hiverunner</artifactId>
      <version>${hiverunner.version}</version>
     <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>${junit.version}</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.21.0</version>
        <configuration>
          <includes>
          </includes>
          <forkCount>1C</forkCount>
          <reuseForks>true</reuseForks>
          <argLine>-Xmx2048m -XX:MaxPermSize=512m</argLine>
          <systemProperties>
            <hiveconf_hive.execution.engine>tez</hiveconf_hive.execution.engine>
            <hiveconf_hive.exec.counters.pull.interval>1000</hiveconf_hive.exec.counters.pull.interval>
            <commandShellEmulator>BEELINE</commandShellEmulator>
            <enableTimeout>false</enableTimeout>
            <timeoutSeconds>30</timeoutSeconds>
            <timeoutRetries>2</timeoutRetries>
          </systemProperties>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.7.0</version>
        <configuration>
          <source>${maven.compiler.source}</source>
          <target>${maven.compiler.target}</target>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

[colin-williams]

My SOME_TABLE.sql definition looks like:

CREATE EXTERNAL TABLE SOME_DB.SOME_TABLE
     (
      E1 VARCHAR(5),
      E2 VARCHAR(100),
      E3 VARCHAR(50),
      E4 DATE,
      E5 TIMESTAMP,
      E6 TIMESTAMP
      )
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\037'
STORED AS TEXTFILE
LOCATION "${hiveconf:hadoop.tmp.dir}/path/to/data.gz";

CREATE EXTERNAL TABLE ANOTHER_DB.ANOTHER_TABLE
     (
      E1 VARCHAR(5),
      E2 VARCHAR(100),
      E3 VARCHAR(50),
      E4 DATE,
      E5 TIMESTAMP,
      E6 TIMESTAMP
      )
STORED AS PARQUET
LOCATION "${hiveconf:hadoop.tmp.dir}/data/out/";

INSERT INTO ANOTHER_DB.ANOTHER_TABLE SELECT * from SOME_DB.SOME_TABLE

[colin-williams]

Test failure but values are returned as expected.

[massdosage]

Where is the test failure in the above log? I don't see any output from JUnit indicating the test failed? Do you get some feedback in the IDE or somewhere else that says this?

[colin-williams]

Hi, there's a clear green check mark on pass but not sure what that symbol is. I will take a screenshot of a passing test. If you missed it look at the top left. Tests passed 0 of 1 test. I will try to give more details and run through the debugger and see if I can figure out anything else.

[colin-williams]

Here's examples of the tests passing. I'm going to create a project which reproduces the issue so it's easy to reproduce on clone.

[massdosage]

A separate project to reproduce would be nice. I'm a bit confused, I thought the original issue you raised was that the assertions pass, i.e. your tests pass but that HiveRunner wasn't indicating that the tests pass. The above screenshot seems to indicate that all the tests are passing so this is good isn't it?

…

On 25 July 2018 at 16:26, colin-williams ***@***.***> wrote: Here's examples of the tests passing. I'm going to create a project which reproduces the failures so it's easy to reproduce on clone. [image: pass] <https://user-images.githubusercontent.com/28717206/43210544-434c4f0e-8fe4-11e8-834e-58a048ba2db5.png> — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#94 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABzEbXojJB6dMSAga4Ht966draS-DESks5uKI4WgaJpZM4VblrZ> .

[colin-williams]

I took a screenshot of passing tests from a different example to juxtaposition the differences between a passing test and my issue. I tried to reproduce but there's some subtle issue I am having problems with. Then I will update when I can reproduce or if I can find the issue.

[massdosage]

OK, good luck tracking it down!

…

On 25 July 2018 at 17:29, colin-williams ***@***.***> wrote: I took a screenshot of passing tests from a different example to juxtaposition the differences between a passing test and my issue. I tried to reproduce but there's some subtle issue I am having problems with. Then I will update when I can reproduce or if I can find the issue. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#94 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABzEYUvOp8aVJXgBJIEE9VgZJX2XJUfks5uKJzagaJpZM4VblrZ> .

[PelleUllberg]

We're having the same issue:

[PelleUllberg]

Posting from one of our engineers:

Could perhaps be other issues but what I have identified is that when there is any set statement in a hive sql we get this behaviour.

Have debugged the code all the way down to the nitty-gritty in the hive code and can see that the set works as expected without throwing any errors as far as I could tell. Can also see that hiverunner notifies IntelliJ that the test has finished and without reporting any failures.

Of course I must be missing something somewhere but no solution in sight so far.

Get the same behaviour when running tests in the hiverunner project from github for example:
MultipleExecutionEnginesTest
OrcSnappyTest

The good news is I found the commit in hiverunner where this behaviour started:
80d990c

The bad news is that the commit contain 79 changed files and it is the commit where hive.version was bumped from 1.2.1 to 2.3.3.
Actually the behaviour disappear if hive.version is flipped back to 1.2.1(and making some small changes to make the code compile with that version).

Behaviour appear when moving from 1.2.2 to 2.0.0

[labbedaine]

Any resolution on this issue? I have the same behaviour using IJ, but it works fine in Eclipse.

[massdosage]

Can you compare the IJ and Eclipse classpaths? It might be due to some classpath ordering differences.

[labbedaine]

@PelleUllberg described the problem very well, I am experiencing the same. Looks like when a Hive query has an explicit "SET" it forces the unit test to exit.

`
USE ${hivevar:db};

SET hive.exec.dynamic.partition.mode=nonstrict;
SET hive.exec.dynamic.partition=true;

SELECT * FROM mytable;
`

This test ^^^ will be interrupted and marked as "Test terminated". If you remove the "SET", it works.

As per the IJ Test Runner tab documentation (https://www.jetbrains.com/help/idea/test-runner-tab.html)

Test terminated. This status is assigned to tests that were cancelled by clicking the Stop button. If at least one test receives this status, then all unfinished tests and their parents are marked as terminated.

I setup a test project on my Github page (https://github.com/labbedaine/HiveRunnerAssertionIssue). If you have a minute, import the project in Eclipse and then IJ and see how it behaves.

I will definitely look into the class path and see if I have differences between Eclipse and IJ.

Thank you for the follow up!

[shermosa]

@labbedaine @PelleUllberg @colin-williams @philltomlinson

Hey everyone, I'm trying to fix this issue and I was wondering if any of you figured anything else that could be useful?
I've been trying things with the https://github.com/labbedaine/HiveRunnerAssertionIssue project but I still haven't figured anything out.

Any help is appreciated :) !

[massdosage]

It looks like the root cause of this is a problem in how IntelliJ mishandles things when the code under test calls close() on System.out. More information at https://youtrack.jetbrains.com/issue/IDEA-120628 (a bug that's been open with them for 7 years!).

Herewith a summary of the events that lead to the problem:

• When Hive runs a SQLOperation it sets up the session state's output stream to a PrintStream that uses System.out, this is done in a private method that doesn't allow you to specify any other behaviour: https://github.com/apache/hive/blob/rel/release-2.3.7/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java#L145
• When running code that contains a "SET" this causes Hive to run a HiveCommandOperation which in turn runs the SetProcessor.
• Later on, when tearing things down after a run, HiveRunner calls client.closeSession(sessionHandle) (see https://github.com/klarna/HiveRunner/blob/hive-2.x/src/main/java/com/klarna/hiverunner/HiveServerContainer.java#L177).
• The above call goes into Hive's internals which in turn gets to HiveCommandOperation which then uses ServiceUtils to close the session state's output stream (which wraps System.out as shown above): https://github.com/apache/hive/blob/rel/release-2.3.7/service/src/java/org/apache/hive/service/cli/operation/HiveCommandOperation.java#L103
• ServiceUtils then calls close() on the output stream which results in a call to System.out.close(): https://github.com/apache/hive/blob/rel/release-2.3.7/service/src/java/org/apache/hive/service/ServiceUtils.java#L61
• From this point on IntelliJ is unable to correctly interpret the JUnit test results (due to the above bug) so it incorrectly marks the test as terminated.

We'll look into adding a workaround for this in HiveRunner as it seems unlikely that it will be fixed in IntelliJ any time soon if they've left the issue open for 7 years.