random port not working for TestNG tests

[wimdeblauwe]

Use Spring Boot 1.2.0, selecting a random port for integration tests does not work for TestNG based tests. I have converted the SpringApplicationIntegrationTestTests.java from JUnit to TestNG. This test fails:

import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.context.embedded.EmbeddedServletContainerFactory;
import org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainerFactory;
import org.springframework.boot.test.IntegrationTest;
import org.springframework.boot.test.SpringApplicationConfiguration;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.support.PropertySourcesPlaceholderConfigurer;
import org.springframework.test.context.testng.AbstractTestNGSpringContextTests;
import org.springframework.test.context.web.WebAppConfiguration;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.client.RestTemplate;
import org.springframework.web.servlet.DispatcherServlet;
import org.springframework.web.servlet.config.annotation.EnableWebMvc;
import org.testng.annotations.Test;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotEquals;

@SpringApplicationConfiguration(classes = SpringApplicationIntegrationTestTestsWithTestNG.Config.class)
@WebAppConfiguration
@IntegrationTest({"server.port=0", "value=123"})
public class SpringApplicationIntegrationTestTestsWithTestNG extends AbstractTestNGSpringContextTests
{

    @Value("${local.server.port}")
    private int port = 0;

    @Value("${value}")
    private int value = 0;

    @Test
    public void runAndTestHttpEndpoint()
    {
        assertNotEquals( 8080, this.port );
        assertNotEquals( 0, this.port );
        String body = new RestTemplate().getForObject( "http://localhost:" + this.port
                                                               + "/", String.class );
        assertEquals( "Hello World", body );
    }

    @Test
    public void annotationAttributesOverridePropertiesFile() throws Exception
    {
        assertEquals( 123, this.value );
    }

    @Configuration
    @EnableWebMvc
    @RestController
    protected static class Config
    {

        @Value("${server.port:8080}")
        private int port = 8080;

        @Bean
        public DispatcherServlet dispatcherServlet()
        {
            return new DispatcherServlet();
        }

        @Bean
        public EmbeddedServletContainerFactory embeddedServletContainer()
        {
            TomcatEmbeddedServletContainerFactory factory = new TomcatEmbeddedServletContainerFactory();
            factory.setPort( this.port );
            return factory;
        }

        @Bean
        public static PropertySourcesPlaceholderConfigurer propertyPlaceholder()
        {
            return new PropertySourcesPlaceholderConfigurer();
        }

        @RequestMapping("/")
        public String home()
        {
            return "Hello World";
        }

    }
}

The only way to fix it is to explicitly add the IntegrationTestPropertiesListener

@TestExecutionListeners(inheritListeners = false,
        listeners = {
                    IntegrationTestPropertiesListener.class,
                DependencyInjectionTestExecutionListener.class,
                DirtiesContextTestExecutionListener.class
        })

The problem is that the class is package protected, so I cannot reference it. I have tested it with copying the code of the listener and referencing that, and then the unit test is ok.

AbstractTestNGSpringContextTests has the following @TestExecutionListeners:

@TestExecutionListeners({ServletTestExecutionListener.class, DependencyInjectionTestExecutionListener.class, DirtiesContextTestExecutionListener.class})

So, or this has to change, or somehow these listeners defined in AbstractTestNGSpringContextTests have to be merged with those of @IntegrationTest or at the very least IntegrationTestPropertiesListener should be made public. In that last case, you might want to add a warning in the docs for TestNG users.

[philwebb]

Did this previously work with Spring Boot 1.1.x?

[wimdeblauwe]

I can confirm that it works with 1.1.8

[sbrannen]

Out of curiosity, can you try extending from AbstractJUnit4SpringContextTests, as I imagine it would fail in that case, too?

Actually, now that I think about it, I imagine the use of @IntegrationTest should fail every time that a superclass declares @TestExecutionListeners explicitly. The reason is that IntegrationTestPropertiesListener is intended to be executed before any other listeners, but as soon as a superclass declares listeners, those listeners will be executed before IntegrationTestPropertiesListener. In the case of AbstractTestNGSpringContextTests as a superclass, both the ServletTestExecutionListener and DependencyInjectionTestExecutionListener access the application context, thereby rendering the entire goal of IntegrationTestPropertiesListener useless.

FYI: see the discussion in #1956. As I mentioned there, IntegrationTestPropertiesListener should be an automatically registered default listener (with highest priority), or better yet, the code should be moved to a boot strapper.

Regards,

Sam

[philwebb]

I can't find a way to make things 100% compatible with Spring Boot 1.1.x and I'm not keen to make IntegrationTestPropertiesListener public since we are likely to remove it when #1956 is fixed.

Instead I've added a new @WebIntegrationTest annotation which is equivalent to @IntegrationTest + @WebAppConfiguration but also works with TestNG. I've also added an example test which runs as part of the build.

I'm sorry we couldn't find a way to make things "just work", but hopefully you can migrate to the new annotation without too much pain.

[sbrannen]

@philwebb, I haven't looked at everything in detail yet, but shouldn't @WebIntegrationTest be annotated with @WebAppConfiguration as a meta-annotation?

That's at least what the Javadoc implies, as far as I understood it.

[sbrannen]

Also, it appears you need to update the Javadoc for SpringApplicationContextLoader regarding @WebAppConfiguration @IntegrationTest.

[philwebb]

I think if it does that we will be back to not knowing which @BootstrapWith will be used. The WebAppIntegrationTestContextBootstrapper triggers the WebDelegatingSmartContextLoader in the same way as WebTestContextBootstrapper.

We also have a different meaning for value in @WebIntegrationTest vs @WebAppConfiguration.