XMLHttpRequest + Access-Control-Allow-Origin = ScriptException

[lilo-san]

I tested that my code works perfectly in Chrome, but fails on HtmlUnit. All my code is pretty basic old style Javascript that I use to test a JSON-RPC2 API.

Steps to reproduce.

1. Send an XMLHttpRequest To a server using the next method.

var sendHttpPost = function(url, payload) {
        console.log('url: ' + url['@value']);
        console.log('payload: ' + payload);

        var request = new XMLHttpRequest();
        var method = 'POST';
        var shouldBeAsync = false;
        request.open(method, url['@value'], shouldBeAsync);
        request.send(payload);
        if (request.status === 200) {
            return request.responseText;
        } else {
            throw new Error('failed call to server - status: ' + request.status);
        }
    }

2. For the first request the server returns with the response the header "Access-Control-Allow-Origin = *", this is expected, is an API usable from any domain.

3. Now send a second request from the client, for some reason the client throws an exception: No permitted "Access-Control-Allow-Origin" header. The request never reaches the server. And as shown on the code above the request doesn't include that header, so why the client complains like is there.

My wild guess is that now the client is trying to put that header because it received it from the server but is not a valid request header. I have total control of the client and server running on my unit tests. In case I can try anything let me know.

This is all my HtmlUnit code:

@Test
    public void javascriptAPITest() throws Exception {
        try (final WebClient webClient = new WebClient(BrowserVersion.BEST_SUPPORTED)) {
            String path = IOUtils.getPath(new File("").getCanonicalPath(), "src/main/javascript/test.html");
            if (IOUtils.exists(path)) {
                URL url = new File(path).toURI().toURL();
                HtmlPage page = webClient.getPage(url);
            }
        }
    }

Stack-trace below:

======= EXCEPTION START ========
Exception class=[net.sourceforge.htmlunit.corejs.javascript.WrappedException]
com.gargoylesoftware.htmlunit.ScriptException: Wrapped java.io.IOException: No permitted "Access-Control-Allow-Origin" header. (file:/Users/lilo-san/Documents/code/repository/objectall-dev-support/src/main/javascript/objectall-jsonrpc2-api-client.js#160)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:891)
	at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:624)
	at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:537)
	at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.callSecured(HtmlUnitContextFactory.java:354)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:771)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:747)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:738)
	at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScript(HtmlPage.java:937)
	at com.gargoylesoftware.htmlunit.html.HtmlScript.executeInlineScriptIfNeeded(HtmlScript.java:305)
	at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:395)
	at com.gargoylesoftware.htmlunit.html.HtmlScript$2.execute(HtmlScript.java:234)
	at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:256)
	at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoDOMBuilder.endElement(HtmlUnitNekoDOMBuilder.java:560)
	at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
	at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoDOMBuilder.endElement(HtmlUnitNekoDOMBuilder.java:514)
	at net.sourceforge.htmlunit.cyberneko.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1192)
	at net.sourceforge.htmlunit.cyberneko.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1132)
	at net.sourceforge.htmlunit.cyberneko.filters.DefaultFilter.endElement(DefaultFilter.java:219)
	at net.sourceforge.htmlunit.cyberneko.filters.NamespaceBinder.endElement(NamespaceBinder.java:312)
	at net.sourceforge.htmlunit.cyberneko.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3189)
	at net.sourceforge.htmlunit.cyberneko.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2114)
	at net.sourceforge.htmlunit.cyberneko.HTMLScanner.scanDocument(HTMLScanner.java:937)
	at net.sourceforge.htmlunit.cyberneko.HTMLConfiguration.parse(HTMLConfiguration.java:443)
	at net.sourceforge.htmlunit.cyberneko.HTMLConfiguration.parse(HTMLConfiguration.java:394)
	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoDOMBuilder.parse(HtmlUnitNekoDOMBuilder.java:760)
	at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoHtmlParser.parse(HtmlUnitNekoHtmlParser.java:236)
	at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoHtmlParser.parseHtml(HtmlUnitNekoHtmlParser.java:179)
	at com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:280)
	at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:163)
	at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:638)
	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:451)
	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:368)
	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:520)
	at com.translator.javascript.APITest.javascriptAPITest(APITest.java:42)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: net.sourceforge.htmlunit.corejs.javascript.WrappedException: Wrapped java.io.IOException: No permitted "Access-Control-Allow-Origin" header. (file:/Users/juanf/Documents/code/repository/objectall-dev-support/src/main/javascript/objectall-jsonrpc2-api-client.js#160)
	at net.sourceforge.htmlunit.corejs.javascript.Context.throwAsScriptRuntimeEx(Context.java:2003)
	at com.gargoylesoftware.htmlunit.javascript.host.xml.XMLHttpRequest.doSend(XMLHttpRequest.java:825)
	at com.gargoylesoftware.htmlunit.javascript.host.xml.XMLHttpRequest.send(XMLHttpRequest.java:611)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:188)
	at net.sourceforge.htmlunit.corejs.javascript.FunctionObject.call(FunctionObject.java:457)
	at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1674)
	at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:1009)
	at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:111)
	at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:427)
	at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:340)
	at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3607)
	at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.exec(InterpretedFunction.java:123)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$2.doRun(JavaScriptEngine.java:762)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:876)
	... 58 more
Caused by: java.io.IOException: No permitted "Access-Control-Allow-Origin" header.
	at com.gargoylesoftware.htmlunit.javascript.host.xml.XMLHttpRequest.doSend(XMLHttpRequest.java:811)
	... 74 more
WrappedException: net.sourceforge.htmlunit.corejs.javascript.WrappedException: Wrapped java.io.IOException: No permitted "Access-Control-Allow-Origin" header. (file:/Users/juanf/Documents/code/repository/objectall-dev-support/src/main/javascript/objectall-jsonrpc2-api-client.js#160)
	at net.sourceforge.htmlunit.corejs.javascript.Context.throwAsScriptRuntimeEx(Context.java:2003)
	at com.gargoylesoftware.htmlunit.javascript.host.xml.XMLHttpRequest.doSend(XMLHttpRequest.java:825)
	at com.gargoylesoftware.htmlunit.javascript.host.xml.XMLHttpRequest.send(XMLHttpRequest.java:611)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:188)
	at net.sourceforge.htmlunit.corejs.javascript.FunctionObject.call(FunctionObject.java:457)
	at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1674)
	at script(file:/Users/juanf/Documents/code/repository/objectall-dev-support/src/main/javascript/objectall-jsonrpc2-api-client.js:160)
	at script(file:/Users/juanf/Documents/code/repository/objectall-dev-support/src/main/javascript/objectall-jsonrpc2-api-client.js:140)
	at script(file:/Users/juanf/Documents/code/repository/objectall-dev-support/src/main/javascript/objectall-jsonrpc2-api-client.js:106)
	at script(file:/Users/juanf/Documents/code/repository/objectall-dev-support/src/main/javascript/objectall-jsonrpc2-api-client.js:249)
	at script(script in file:/Users/juanf/Documents/code/repository/objectall-dev-support/src/main/javascript/test.html from (6, 40) to (31, 18):26)
	at script(script in file:/Users/juanf/Documents/code/repository/objectall-dev-support/src/main/javascript/test.html from (6, 40) to (31, 18):30)
	at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:1009)
	at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:111)
	at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:427)
	at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:340)
	at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3607)
	at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.exec(InterpretedFunction.java:123)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$2.doRun(JavaScriptEngine.java:762)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:876)
	at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:624)
	at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:537)
	at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.callSecured(HtmlUnitContextFactory.java:354)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:771)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:747)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:738)
	at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScript(HtmlPage.java:937)
	at com.gargoylesoftware.htmlunit.html.HtmlScript.executeInlineScriptIfNeeded(HtmlScript.java:305)
	at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:395)
	at com.gargoylesoftware.htmlunit.html.HtmlScript$2.execute(HtmlScript.java:234)
	at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:256)
	at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoDOMBuilder.endElement(HtmlUnitNekoDOMBuilder.java:560)
	at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
	at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoDOMBuilder.endElement(HtmlUnitNekoDOMBuilder.java:514)
	at net.sourceforge.htmlunit.cyberneko.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1192)
	at net.sourceforge.htmlunit.cyberneko.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1132)
	at net.sourceforge.htmlunit.cyberneko.filters.DefaultFilter.endElement(DefaultFilter.java:219)
	at net.sourceforge.htmlunit.cyberneko.filters.NamespaceBinder.endElement(NamespaceBinder.java:312)
	at net.sourceforge.htmlunit.cyberneko.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3189)
	at net.sourceforge.htmlunit.cyberneko.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2114)
	at net.sourceforge.htmlunit.cyberneko.HTMLScanner.scanDocument(HTMLScanner.java:937)
	at net.sourceforge.htmlunit.cyberneko.HTMLConfiguration.parse(HTMLConfiguration.java:443)
	at net.sourceforge.htmlunit.cyberneko.HTMLConfiguration.parse(HTMLConfiguration.java:394)
	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoDOMBuilder.parse(HtmlUnitNekoDOMBuilder.java:760)
	at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoHtmlParser.parse(HtmlUnitNekoHtmlParser.java:236)
	at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoHtmlParser.parseHtml(HtmlUnitNekoHtmlParser.java:179)
	at com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:280)
	at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:163)
	at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:638)
	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:451)
	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:368)
	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:520)
	at com.translator.javascript.APITest.javascriptAPITest(APITest.java:42)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.io.IOException: No permitted "Access-Control-Allow-Origin" header.
	at com.gargoylesoftware.htmlunit.javascript.host.xml.XMLHttpRequest.doSend(XMLHttpRequest.java:811)
	... 74 more
Inside wrapped exception:
java.io.IOException: No permitted "Access-Control-Allow-Origin" header.
	at com.gargoylesoftware.htmlunit.javascript.host.xml.XMLHttpRequest.doSend(XMLHttpRequest.java:811)
	at com.gargoylesoftware.htmlunit.javascript.host.xml.XMLHttpRequest.send(XMLHttpRequest.java:611)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:188)
	at net.sourceforge.htmlunit.corejs.javascript.FunctionObject.call(FunctionObject.java:457)
	at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1674)
	at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:1009)
	at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:111)
	at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:427)
	at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:340)
	at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3607)
	at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.exec(InterpretedFunction.java:123)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$2.doRun(JavaScriptEngine.java:762)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:876)
	at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:624)
	at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:537)
	at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.callSecured(HtmlUnitContextFactory.java:354)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:771)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:747)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:738)
	at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScript(HtmlPage.java:937)
	at com.gargoylesoftware.htmlunit.html.HtmlScript.executeInlineScriptIfNeeded(HtmlScript.java:305)
	at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:395)
	at com.gargoylesoftware.htmlunit.html.HtmlScript$2.execute(HtmlScript.java:234)
	at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:256)
	at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoDOMBuilder.endElement(HtmlUnitNekoDOMBuilder.java:560)
	at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
	at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoDOMBuilder.endElement(HtmlUnitNekoDOMBuilder.java:514)
	at net.sourceforge.htmlunit.cyberneko.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1192)
	at net.sourceforge.htmlunit.cyberneko.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1132)
	at net.sourceforge.htmlunit.cyberneko.filters.DefaultFilter.endElement(DefaultFilter.java:219)
	at net.sourceforge.htmlunit.cyberneko.filters.NamespaceBinder.endElement(NamespaceBinder.java:312)
	at net.sourceforge.htmlunit.cyberneko.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3189)
	at net.sourceforge.htmlunit.cyberneko.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2114)
	at net.sourceforge.htmlunit.cyberneko.HTMLScanner.scanDocument(HTMLScanner.java:937)
	at net.sourceforge.htmlunit.cyberneko.HTMLConfiguration.parse(HTMLConfiguration.java:443)
	at net.sourceforge.htmlunit.cyberneko.HTMLConfiguration.parse(HTMLConfiguration.java:394)
	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoDOMBuilder.parse(HtmlUnitNekoDOMBuilder.java:760)
	at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoHtmlParser.parse(HtmlUnitNekoHtmlParser.java:236)
	at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoHtmlParser.parseHtml(HtmlUnitNekoHtmlParser.java:179)
	at com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:280)
	at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:163)
	at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:638)
	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:451)
	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:368)
	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:520)
	at com.translator.javascript.APITest.javascriptAPITest(APITest.java:42)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
======= EXCEPTION END ========

[rbri]

Thanks for your error report - to deeper analyze the problem i need some help

• do you use the latest version?
• can you please capture the communication (using Charles WebProxy)
  ** for one real browser
  ** for HtmlUnit

Looks like a problem with the preflight requests but i need more details....

[lilo-san]

Hi @rbri,

I'm indeed using version 2.43.0. There is no preflight request, maybe that is the problem?

This server will always response with the header: Access-Control-Allow-Origin = *

Why? Well, is purely an API server that initially everybody is allowed to call, and not necessarily a browser, it has its own Authentication/Authorisation on the JSONRPC2 Protocol.

This works perfectly on browsers. But well, a lot of stuff works in browsers that it shouldn't.

It's a missing preflight request against the specification?

Here is the Charles Proxy Export for the Safari Browser: https://www.dropbox.com/s/6kwzm4wv3ylp4fp/Working-Safari.chls?dl=0

[lilo-san]

Sadly I have been unable to record HTMLUnit until now even if is the same server and I'm reaching it using the same URL.

Is there any particular trick/guide to set HTMLUnit to go though Charles? I will continue looking at it.

[rbri]

define charles as proxy when setting up you web client

try (final WebClient webClient = new WebClient(BrowserVersion.BEST_SUPPORTED, "localhost", 8888)) {

[rbri]

do you have a chance to debug into the htmlunit code?

[rbri]

Does your code sets the withCredentials property of the request for the second call?

[lilo-san]

I have just tried with:

try (final WebClient webClient = new WebClient(BrowserVersion.BEST_SUPPORTED, "localhost", 8888)) {

To my amusement, using the proxy I can not only record it, but it works, making this futile if what we want is to get the stack trace.

https://www.dropbox.com/s/ahm1fqddjo0dc48/Working-HTMLUnit.chls?dl=0

My code doesn't change anything on the Java side, no withCredentials, no anything, is just as plain as shown above.

I have not tried to debug HtmlUnit, would you like to get the state of any particular class?

[rbri]

Can you please check if it is related to the browser version or to charles Am 3. September 2020 20:44:46 MESZ schrieb lilo-san <notifications@github.com>:

…

I have just tried with: try (final WebClient webClient = new WebClient(BrowserVersion.BEST_SUPPORTED, "localhost", 8888)) { To my amusement, using the proxy I can not only record it, but it works, making this futile if what we want is to get the stack trace. https://www.dropbox.com/s/ahm1fqddjo0dc48/Working-HTMLUnit.chls?dl=0 My code doesn't change anything on the Java side, no withCredentials, no anything, is just as plain as shown above. I have not tried to debug HtmlUnit, would you like to get the state of any particular class? -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #222 (comment)

-- Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

[rbri]

Another reason might be the server adds some whitespace to the star in the header (cleaned by charles) Am 3. September 2020 20:44:46 MESZ schrieb lilo-san <notifications@github.com>:

…

I have just tried with: try (final WebClient webClient = new WebClient(BrowserVersion.BEST_SUPPORTED, "localhost", 8888)) { To my amusement, using the proxy I can not only record it, but it works, making this futile if what we want is to get the stack trace. https://www.dropbox.com/s/ahm1fqddjo0dc48/Working-HTMLUnit.chls?dl=0 My code doesn't change anything on the Java side, no withCredentials, no anything, is just as plain as shown above. I have not tried to debug HtmlUnit, would you like to get the state of any particular class? -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #222 (comment)

-- Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

[lilo-san]

Debugging the HTMLUnit code, it seems this is the code throwing the exception:

           if (!allowOriginResponse) {
                if (LOG.isDebugEnabled()) {
                    LOG.debug("No permitted \"Access-Control-Allow-Origin\" header for URL " + this.webRequest_.getUrl());
                }

                throw new IOException("No permitted \"Access-Control-Allow-Origin\" header.");
            }

What doesn't make sense, if we look at the Charles output because the header is there.

I also noticed that now debugging step by step, for some reason I can't explain yet, it works.

[lilo-san]

The server doesn't add any whitespaces, just checked.

Are synchronous XMLHttpRequest well supported?

[rbri]

Please try with a breakpoint at the throw statement maybe this gives a hint Am 3. September 2020 20:55:41 MESZ schrieb lilo-san <notifications@github.com>:

…

Debugging the HTMLUnit code, it seems this is the code throwing the exception: ``` if (!allowOriginResponse) { if (LOG.isDebugEnabled()) { LOG.debug("No permitted \"Access-Control-Allow-Origin\" header for URL " + this.webRequest_.getUrl()); } throw new IOException("No permitted \"Access-Control-Allow-Origin\" header."); } ``` What doesn't make sense, if we look at the Charles output because the header is there. I also noticed that now debugging step by step, for some reason I can't explain yet, it works. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #222 (comment)

-- Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

[rbri]

Do you reuse the same xmlrequest instance ? Am 3. September 2020 20:55:41 MESZ schrieb lilo-san <notifications@github.com>:

…

Debugging the HTMLUnit code, it seems this is the code throwing the exception: ``` if (!allowOriginResponse) { if (LOG.isDebugEnabled()) { LOG.debug("No permitted \"Access-Control-Allow-Origin\" header for URL " + this.webRequest_.getUrl()); } throw new IOException("No permitted \"Access-Control-Allow-Origin\" header."); } ``` What doesn't make sense, if we look at the Charles output because the header is there. I also noticed that now debugging step by step, for some reason I can't explain yet, it works. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #222 (comment)

-- Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

[lilo-san]

I don't reuse it in Javascript, this is everything, a new call to sendHttpPost is used every time.

var sendHttpPost = function(url, payload) {
        var request = new XMLHttpRequest();
        var method = 'POST';
        var shouldBeAsync = false;
        request.open(method, url['@value'], shouldBeAsync);
        request.send(payload);
        if (request.status === 200) {
            return request.responseText;
        } else {
            throw new Error('failed call to server - status: ' + request.status);
        }
    }

Looking that doing a set by step works but putting a breakpoint on the throw I can see the state when is not working I'm looking for race conditions or some oversight when doing synchronous requests.

[rbri]

I cross my finger and hop you can find the reason Am 3. September 2020 21:15:53 MESZ schrieb lilo-san <notifications@github.com>:

…

I don't reuse it in Javascript, this is everything, a new call to sendHttpPost is used every time. var sendHttpPost = function(url, payload) { var request = new XMLHttpRequest(); var method = 'POST'; var shouldBeAsync = false; request.open(method, ***@***.***'], shouldBeAsync); request.send(payload); if (request.status === 200) { return request.responseText; } else { throw new Error('failed call to server - status: ' + request.status); } } Looking that doing a set by step works but putting a breakpoint on the throw I can see the state when is not working I'm looking for race conditions or some oversight when doing synchronous requests. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #222 (comment)

-- Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.