MutationObserver not triggering?

[skyhirider]

I've ran into a case where the mutation observer is not triggering, not sure if its a configuration issue or if the observer is busted. My test case:

package mains;

import com.gargoylesoftware.htmlunit.BrowserVersion;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.WebClientOptions;
import com.gargoylesoftware.htmlunit.javascript.SilentJavaScriptErrorListener;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxDriverLogLevel;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.htmlunit.HtmlUnitDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public class MutationTest {

    private WebDriver ffDriver;
    private WebDriver huDriver;

    private static final String TRIGGER_OBSERVER = "document.text=\"Text1\";\n" +
            "const targetNode = document.getElementById('code-tab');\n" +
            "const config = { attributes: true, childList: true, subtree: true };\n" +
            "window.count = 0;\n" +
            "document.count = 0;\n" +
            "const callback = function(mutationsList, observer) {\n" +
            "    console.log('derp');\n" +
            "    window.count++;\n" +
            "    document.count++;\n" +
            "};\n" +
            "const observer = new MutationObserver(callback);\n" +
            "observer.observe(targetNode, config);";

    @BeforeEach
    void initDrivers() {
        System.setProperty("webdriver.gecko.driver", "c:\\...\\geckodriver.exe");
        FirefoxOptions options = new FirefoxOptions();
        options.setLogLevel(FirefoxDriverLogLevel.FATAL);
        ffDriver = new FirefoxDriver(options);
        huDriver = new HtmlUnitDriver(BrowserVersion.FIREFOX, true) {
            @Override
            protected WebClient modifyWebClient(WebClient client) {
                final WebClient webClient = super.modifyWebClient(client);
                WebClientOptions options = webClient.getOptions();
                options.setCssEnabled(true);
                options.setThrowExceptionOnScriptError(false);
                webClient.setJavaScriptErrorListener(new SilentJavaScriptErrorListener());
                return webClient;
            }
        };
    }

    @AfterEach
    void closeDrivers(){
        ffDriver.close();
        huDriver.close();
    }

    @Test
    void givenJavascriptPage_whenModifyingElement_triggerMutationObserver(){
        Assertions.assertEquals(mutate(ffDriver),mutate(huDriver));
    }

    private Long mutate(WebDriver driver) {
        driver.get("https://github.com/HtmlUnit/htmlunit");
        JavascriptExecutor js = (JavascriptExecutor) driver;
        js.executeScript(TRIGGER_OBSERVER);
        WebDriverWait wait = new WebDriverWait(driver,10);
        By codeTab = By.cssSelector("#code-tab");
        wait.until(ExpectedConditions.elementToBeClickable(codeTab));
        driver.findElement(codeTab).click();
        return (Long) js.executeScript("return document.count") + (Long) js.executeScript("return window.count");
    }

}

Firefox returns a number, Htmlunit returns null.

[rbri]

Did some testing with this - when i run this i got a NPE because both counters are null (undefined).

Looks like driver.findElement(codeTab).click(); drops the variables.

I fear it will be a nightmare to debug the github js code to figure out where the error in HtmlUnit is. Do you have any simpler example?

[skyhirider]

I've tested it on another site and its failing, thou differently (expected is 6, returned is 8)

public class MutationTest {

    private WebDriver ffDriver;
    private WebDriver huDriver;

    private static final String TRIGGER_OBSERVER = "document.text=\"Text1\";\n" +
            "const targetNode = document.getElementsByClassName('site-wrapper  site-wrapper--home  js-site-wrapper')[0];\n" +
            "const config = { attributes: true, childList: true, subtree: true };\n" +
            "window.count = 0;\n" +
            "document.count = 0;\n" +
            "const callback = function(mutationsList, observer) {\n" +
            "    console.log('derp');\n" +
            "    window.count++;\n" +
            "    document.count++;\n" +
            "};\n" +
            "const observer = new MutationObserver(callback);\n" +
            "observer.observe(targetNode, config);";

    @BeforeEach
    void initDrivers() {
        System.setProperty("webdriver.gecko.driver", "c:\\geckodriver.exe");
        FirefoxOptions options = new FirefoxOptions();
        options.setLogLevel(FirefoxDriverLogLevel.FATAL);
        ffDriver = new FirefoxDriver(options);
        huDriver = new HtmlUnitDriver(BrowserVersion.FIREFOX, true) {
            @Override
            protected WebClient modifyWebClient(WebClient client) {
                final WebClient webClient = super.modifyWebClient(client);
                WebClientOptions options = webClient.getOptions();
                options.setCssEnabled(true);
                options.setThrowExceptionOnScriptError(false);
                webClient.setJavaScriptErrorListener(new SilentJavaScriptErrorListener());
                return webClient;
            }
        };
    }

    @AfterEach
    void closeDrivers(){
        ffDriver.close();
        huDriver.close();
    }

    @Test
    void givenJavascriptPage_whenModifyingElement_triggerMutationObserver(){
        Assertions.assertEquals(mutate(ffDriver),mutate(huDriver));
    }

    private Long mutate(WebDriver driver) {
        driver.get("https://duckduckgo.com/");
        JavascriptExecutor js = (JavascriptExecutor) driver;
        WebDriverWait wait = new WebDriverWait(driver,10);
        By codeTab = By.xpath("//div[@class='site-wrapper  site-wrapper--home  js-site-wrapper']");
        wait.until(ExpectedConditions.presenceOfAllElementsLocatedBy(codeTab));
        js.executeScript(TRIGGER_OBSERVER);
        driver.findElement(codeTab).click();
        return (Long) js.executeScript("return document.count") + (Long) js.executeScript("return window.count");
    }

}

[rbri]

@skyhirider do you see any chance to have this kind of test for simple pages - at best under our control.
This will make it much more easy for me to write a isolated test and fix it.

[skyhirider]

Frankensteined this test with a previous one I posted for another issue way back, no external site dependency now, even if it is a bit messier :)

public class MutationTest {

    private WebDriver ffDriver;
    private WebDriver huDriver;

    private final static String buttonPage = "<button id=\"testButton\">Click me</button>\n" +
            "<div id=\"logArea\"></div>\n" +
            "<script>\n" +
            "    const button = document.getElementById('testButton');\n" +
            "    const div = document.getElementById('logArea');\n" +
            "\n" +
            "    button.addEventListener('mousedown', e => {\n" +
            "        console.log('up');\n" +
            "        div.innerHTML += \"up \";\n" +
            "    });\n" +
            "    button.addEventListener('mouseup', e => {\n" +
            "        console.log('down');\n" +
            "        div.innerHTML += \"down \";\n" +
            "    });\n" +
            "    button.addEventListener('click', e => {\n" +
            "        console.log('click');\n" +
            "        div.innerHTML += \"click \";\n" +
            "    });\n" +
            "\n" +
            "\n" +
            "</script>";

    private static final String TRIGGER_OBSERVER = "document.text=\"Text1\";\n" +
            "const targetNode = document.getElementById('testButton');\n" +
            "const config = { attributes: true, childList: true, subtree: true };\n" +
            "window.count = 0;\n" +
            "document.count = 0;\n" +
            "const callback = function(mutationsList, observer) {\n" +
            "    console.log('derp');\n" +
            "    window.count++;\n" +
            "    document.count++;\n" +
            "};\n" +
            "const observer = new MutationObserver(callback);\n" +
            "observer.observe(targetNode, config);";

    @BeforeEach
    void initDrivers() {
        System.setProperty("webdriver.gecko.driver", "c:\\geckodriver.exe");
        FirefoxOptions options = new FirefoxOptions();
        options.setLogLevel(FirefoxDriverLogLevel.FATAL);
        ffDriver = new FirefoxDriver(options);
        huDriver = new HtmlUnitDriver(BrowserVersion.FIREFOX, true) {
            @Override
            protected WebClient modifyWebClient(WebClient client) {
                final WebClient webClient = super.modifyWebClient(client);
                WebClientOptions options = webClient.getOptions();
                options.setCssEnabled(true);
                options.setThrowExceptionOnScriptError(false);
                webClient.setJavaScriptErrorListener(new SilentJavaScriptErrorListener());
                return webClient;
            }
        };
    }

    @AfterEach
    void closeDrivers(){
        ffDriver.close();
        huDriver.close();
    }

    @Test
    void givenJavascriptPage_whenModifyingElement_triggerMutationObserver(){
        Assertions.assertEquals(mutate(ffDriver),mutate(huDriver));
    }

    private Long mutate(WebDriver driver) {
        driver.get("data:text/html;charset=utf-8," + buttonPage);
        JavascriptExecutor js = (JavascriptExecutor) driver;
        WebDriverWait wait = new WebDriverWait(driver,10);
        By codeTab = By.cssSelector("#testButton");
        wait.until(ExpectedConditions.presenceOfAllElementsLocatedBy(codeTab));
        js.executeScript(TRIGGER_OBSERVER);
        driver.findElement(codeTab).click();
        return (Long) js.executeScript("return document.count") + (Long) js.executeScript("return window.count");
    }

}

[rbri]

Great, thanks this is a great starting point, i fear there are many holes in the current impl.

[skyhirider]

Seems so, but its still a great library and the only way to run a driver purely in java. I didn't experience any major issues apart from the few cases that I have reported.

I wonder if you could re-use test cases from other driver libraries by swapping where they are ran and see how well HtmlUnit holds.

The HtmlUnit driver + browser is an adapter so I presume the port (interface) should have test cases on it that should be applicable to all, but haven't really checked the Selenium code or any others.

[rbri]

Hi @skyhirider,
another really interesting riddle!

Base on your code a made this simple HtmlPage

<html>
<body>
<button id="testButton">Click me</button>
<div id="logArea"></div>

<script>
const button = document.getElementById('testButton');
const div = document.getElementById('logArea');
button.addEventListener('click', e => {
	div.innerHTML += "click ";    });
	const targetNode = document.getElementById('testButton');
	const config = { attributes: true, childList: true, subtree: true };
	document.count = 0;
	const callback = function(mutationsList, observer) {
		mutationsList.forEach(function(mutation) {
			div.innerHTML += mutation.type + "; ";
			div.innerHTML += mutation.attributeName;
		});
	document.count++;
};

const observer = new MutationObserver(callback);
observer.observe(targetNode, config);
</script>
</body></html>

When running this page with Selenium and clicking the button i got this output

attributes; style; attributes; style; attributes; style; attributes; style; click

Doing the same with HtmlUnit (WebDriver) or even with real browsers does only produce

click

What do you think - might this be a selenium bug to trigger some style changes when clicking the button?
Confused....
RBRi

[skyhirider]

Huh, interesting. And confusing. We could report it as a gecko driver bug, as I've tested htmlunit against the chrome driver and those produce the same result.

class HtmlUnitClickMutatorTest {

    private WebDriver ffDriver;
    private WebDriver huDriver;
    private final static String buttonPage = "<html>\n" +
            "<body>\n" +
            "<button id=\"testButton\">Click me</button>\n" +
            "<div id=\"logArea\"></div>\n" +
            "\n" +
            "<script>\n" +
            "const button = document.getElementById('testButton');\n" +
            "const div = document.getElementById('logArea');\n" +
            "button.addEventListener('click', e => {\n" +
            "\tdiv.innerHTML += \"click \";    });\n" +
            "\tconst targetNode = document.getElementById('testButton');\n" +
            "\tconst config = { attributes: true, childList: true, subtree: true };\n" +
            "\tdocument.count = 0;\n" +
            "\tconst callback = function(mutationsList, observer) {\n" +
            "\t\tmutationsList.forEach(function(mutation) {\n" +
            "\t\t\tdiv.innerHTML += mutation.type + \"; \";\n" +
            "\t\t\tdiv.innerHTML += mutation.attributeName;\n" +
            "\t\t});\n" +
            "\tdocument.count++;\n" +
            "};\n" +
            "\n" +
            "const observer = new MutationObserver(callback);\n" +
            "observer.observe(targetNode, config);\n" +
            "</script>\n" +
            "</body></html>";
    private WebDriver chromeDriver;


    @BeforeEach
    void initDrivers() {
        System.setProperty("webdriver.gecko.driver", "C:\\geckodriver.exe");
        System.setProperty("webdriver.chrome.driver", "C:\\chromedriver.exe");
        FirefoxOptions options = new FirefoxOptions();
        options.setLogLevel(FirefoxDriverLogLevel.FATAL);
        ffDriver = new FirefoxDriver(options);
        chromeDriver = new ChromeDriver();
        huDriver = new HtmlUnitDriver(BrowserVersion.FIREFOX, true) {
            @Override
            protected WebClient modifyWebClient(WebClient client) {
                final WebClient webClient = super.modifyWebClient(client);
                WebClientOptions options = webClient.getOptions();
                options.setCssEnabled(true);
                options.setThrowExceptionOnScriptError(false);
                webClient.setJavaScriptErrorListener(new SilentJavaScriptErrorListener());

                return webClient;
            }
        };
    }

    @AfterEach
    void closeDrivers(){
        ffDriver.close();
        huDriver.close();
        chromeDriver.close();
    }

    @Test
    void givenFFandHtmlUnit_whenClicking_registerThreeEvents() {
        Assertions.assertEquals(testClicks(huDriver), testClicks(chromeDriver));
        Assertions.assertEquals(testClicks(ffDriver), testClicks(chromeDriver));
        Assertions.assertEquals(testClicks(ffDriver), testClicks(huDriver));
    }

    private String testClicks(WebDriver driver) {
        driver.get("data:text/html;charset=utf-8," + buttonPage);
        driver.findElement(By.id("testButton")).click();
        return driver.findElement(By.id("logArea")).getText();
    }

}

[skyhirider]

I am also wondering if the original bug I reported was the same as the old test code throws a nullpointer. So the test cases may be different.

[rbri]

Seems so, but its still a great library and the only way to run a driver purely in java. I didn't experience any major issues apart from the few cases that I have reported.

Great

I wonder if you could re-use test cases from other driver libraries by swapping where they are ran and see how well HtmlUnit holds.

In fact that is the only way to do this project - 90% of all testcases can run with HtmlUnit (for every supported browser) and using the WebDriver to run exactly the same test against the real counterpart to validate the expected results. Every time a browser gets updated i have to rerun the test suite here to get an idea about changes.
On the other hand i have to write detailed single function test cases for every functionality including all the border cases.

And i use many of the test suites from other libs (jQuery, htmx, prototype) and run them as part of the test suite.

If you like to get an impression have a look at https://jenkins.wetator.org/view/HtmlUnit/

[rbri]

Will try to report the second problem as selenium bug and then come back to the first one (NPE).

[rbri]

SeleniumHQ/selenium#10520

[rbri]

mozilla/geckodriver#2002

[rbri]

Ok, i found the reason for the NPE - looks like clicking the button changes the current page in HtmlUnitDriver and based on this the property are back to undefined.
Have to dig deeper to find the reason for this.

[skyhirider]

Glad to hear, managed to hunt it down in the end? It did seem like an odd bug.