[BUG] Playwright cannot download the browsers automatically

[vidi42]

I'm trying to use Playwright in a Java EE environment with Zulu 11 and I always get this error messaged when launching chromium.

com.microsoft.playwright.PlaywrightException: Error {
  message='Failed to launch chromium because executable doesn't exist at C:\Users\Alex\AppData\Local\ms-playwright\chromium-857950\chrome-win\chrome.exe
Try re-installing playwright with "npm install playwright"
  name='Error
  stack='Error: Failed to launch chromium because executable doesn't exist at C:\Users\Alex\AppData\Local\ms-playwright\chromium-857950\chrome-win\chrome.exe
Try re-installing playwright with "npm install playwright"
    at Chromium._launchProcess (C:\Users\Alex\AppData\Local\Temp\playwright-java-12834975463618432611\package\lib\server\browserType.js:174:19)
    at async Chromium._innerLaunch (C:\Users\Alex\AppData\Local\Temp\playwright-java-12834975463618432611\package\lib\server\browserType.js:110:62)
    at async ProgressController.run (C:\Users\Alex\AppData\Local\Temp\playwright-java-12834975463618432611\package\lib\server\progress.js:79:28)
    at async Chromium.launch (C:\Users\Alex\AppData\Local\Temp\playwright-java-12834975463618432611\package\lib\server\browserType.js:78:25)
    at async BrowserTypeDispatcher.launch (C:\Users\Alex\AppData\Local\Temp\playwright-java-12834975463618432611\package\lib\dispatchers\browserTypeDispatcher.js:30:25)
    at async DispatcherConnection.dispatch (C:\Users\Alex\AppData\Local\Temp\playwright-java-12834975463618432611\package\lib\dispatchers\dispatcher.js:181:28)
}
    at com.microsoft.playwright.impl.WaitableResult.get(WaitableResult.java:54)
    at com.microsoft.playwright.impl.ChannelOwner.runUntil(ChannelOwner.java:88)
    at com.microsoft.playwright.impl.Connection.sendMessage(Connection.java:96)
    at com.microsoft.playwright.impl.ChannelOwner.sendMessage(ChannelOwner.java:79)
    at com.microsoft.playwright.impl.BrowserTypeImpl.launchImpl(BrowserTypeImpl.java:49)
    at com.microsoft.playwright.impl.BrowserTypeImpl.lambda$launch$0(BrowserTypeImpl.java:41)
    at com.microsoft.playwright.impl.LoggingSupport.withLogging(LoggingSupport.java:47)
    at com.microsoft.playwright.impl.BrowserTypeImpl.launch(BrowserTypeImpl.java:41)
    at com.microsoft.playwright.impl.BrowserTypeImpl.launch(BrowserTypeImpl.java:34)
    at com.microsoft.playwright.BrowserType.launch(BrowserType.java:601)
    at de.symeda.sormas.ui.labmessage.LabMessageController.convertToPdf(LabMessageController.java:965)
    at de.symeda.sormas.ui.labmessage.LabMessageController.downloadLabMessage(LabMessageController.java:941)
    at de.symeda.sormas.ui.labmessage.LabMessageGrid.lambda$buildDownloadButton$c7abbaac$1(LabMessageGrid.java:177)
    at com.vaadin.server.StreamResource.getStream(StreamResource.java:143)
    at com.vaadin.server.FileDownloader.handleConnectorRequest(FileDownloader.java:167)
    at com.vaadin.server.ConnectorResourceHandler.handleRequest(ConnectorResourceHandler.java:90)
    at com.vaadin.server.VaadinService.handleRequest(VaadinService.java:1608)
    at com.vaadin.server.VaadinServlet.service(VaadinServlet.java:448)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
    at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1628)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:339)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:209)
    at de.symeda.sormas.ui.SessionFilter.lambda$doFilter$2(SessionFilter.java:72)
    at de.symeda.sormas.ui.SessionFilterBean.doFilter(SessionFilterBean.java:37)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:588)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:408)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4791)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:665)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:836)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:615)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
    at jdk.internal.reflect.GeneratedMethodAccessor1707.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:895)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:835)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:615)
    at org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:72)
    at org.jboss.weld.module.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
    at jdk.internal.reflect.GeneratedMethodAccessor1687.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:895)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:835)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:374)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4763)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4751)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:90)
    at com.sun.proxy.$Proxy2418.doFilter(Unknown Source)
    at de.symeda.sormas.ui.__EJB31_Generated__SessionFilterBean__Intf____Bean__.doFilter(Unknown Source)
    at de.symeda.sormas.ui.SessionFilter.doFilter(SessionFilter.java:65)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:251)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:209)
    at de.symeda.sormas.ui.Slf4jMdcFilter.doFilter(Slf4jMdcFilter.java:29)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:251)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:209)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:256)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:160)
    at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:755)
    at org.apache.catalina.core.StandardPipeline.doChainInvoke(StandardPipeline.java:579)
    at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:97)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:159)
    at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:371)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:238)
    at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
    at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
    at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:182)
    at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:156)
    at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:218)
    at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
    at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
    at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:524)
    at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
    at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
    at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: com.microsoft.playwright.impl.DriverException: Error {
  message='Failed to launch chromium because executable doesn't exist at C:\Users\Alex\AppData\Local\ms-playwright\chromium-857950\chrome-win\chrome.exe
Try re-installing playwright with "npm install playwright"
  name='Error
  stack='Error: Failed to launch chromium because executable doesn't exist at C:\Users\Alex\AppData\Local\ms-playwright\chromium-857950\chrome-win\chrome.exe
Try re-installing playwright with "npm install playwright"
    at Chromium._launchProcess (C:\Users\Alex\AppData\Local\Temp\playwright-java-12834975463618432611\package\lib\server\browserType.js:174:19)
    at async Chromium._innerLaunch (C:\Users\Alex\AppData\Local\Temp\playwright-java-12834975463618432611\package\lib\server\browserType.js:110:62)
    at async ProgressController.run (C:\Users\Alex\AppData\Local\Temp\playwright-java-12834975463618432611\package\lib\server\progress.js:79:28)
    at async Chromium.launch (C:\Users\Alex\AppData\Local\Temp\playwright-java-12834975463618432611\package\lib\server\browserType.js:78:25)
    at async BrowserTypeDispatcher.launch (C:\Users\Alex\AppData\Local\Temp\playwright-java-12834975463618432611\package\lib\dispatchers\browserTypeDispatcher.js:30:25)
    at async DispatcherConnection.dispatch (C:\Users\Alex\AppData\Local\Temp\playwright-java-12834975463618432611\package\lib\dispatchers\dispatcher.js:181:28)
}
    at com.microsoft.playwright.impl.Connection.dispatch(Connection.java:209)
    at com.microsoft.playwright.impl.Connection.processOneMessage(Connection.java:189)
    at com.microsoft.playwright.impl.ChannelOwner.runUntil(ChannelOwner.java:86)
    ... 87 more
]]

I tried changing the path for browser download using the Environment Variable PLAYWRIGHT_BROWSERS_PATH but still the script was unable to download them.

Any idea what might be wrong? Or any suggestion how can I debug the download script?

Environment Specifications

OS: Windows 10
JDK: Zulu 11
Server: Payara 5

[yury-s]

Which Playwright version are you using?

Any idea what might be wrong?

Could it be that you have PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD in your environment?

Or any suggestion how can I debug the download script?

First time you launch given version of Playwright you should see lines like this in stdout:

chromium v869727 downloaded to /home/yurys/.cache/ms-playwright/chromium-869727
firefox v1245 downloaded to /home/yurys/.cache/ms-playwright/firefox-1245
webkit v1463 downloaded to /home/yurys/.cache/ms-playwright/webkit-1463
ffmpeg v1005 downloaded to /home/yurys/.cache/ms-playwright/ffmpeg-1005

Can you confirm if you see them? If the browsers are not in the cache playwright will try to download them on every launch and if that step failed it would not continue. Your code runs after that step which means that either the browsers were downloaded and then removed while the program was still running or the download step was skipped because of the env variable above.

[vidi42]

Thanks for the response @yury-s

I'm using 1.10.0.

The PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD variable is not set on my environment.
Also I don't see the lines in the logs of the application.

My first assumption was that maybe the script downloading the drivers doesn't have enough permissions so I tried changing it to download in the context of the server, however still the same result.

The issue seems to be located in the playwright.cmd script when it's called from com.microsoft.playwright.impl.DriverJar#installBrowsers. Anything up to that point is working correctly.

In the end I just end up with a folder which should contain the files you mentioned above, but it's just a .links folder in it with files containing the path to the playwright.cmd script.

A few things worth mentioning maybe is that I am using maven to build the application which is using Playwright.

[yury-s]

In the end I just end up with a folder which should contain the files you mentioned above, but it's just a .links folder in it with files containing the path to the playwright.cmd script.

This is really odd, .links is updated by the same code that downloads browsers. Presence of .links suggests that download should have run and if it failed you should have seen an error printed to the console. Can you delete C:\Users\Alex\AppData\Local\ms-playwright\ altogether, run playwright again and see what's inside?

A few things worth mentioning maybe is that I am using maven to build the application which is using Playwright.

You just add playwright dependency to your pom.xml (i.e. you don't build it from sources), correct?

[yury-s]

Another idea, can you run the following in terminal:

mvn exec:java -e -Dexec.mainClass=com.microsoft.playwright.CLI -Dexec.args="install chromium"

It should download chromium to the cache folder (it should print something like Downloading chromium v854489 - 124.7 Mb [====================] 100% 0.0s), chrome.exe should be present thereafter.

[vidi42]

Can you delete C:\Users\Alex\AppData\Local\ms-playwright\ altogether, run playwright again and see what's inside?

Did that and still the same result. Just the .links directory was created.

You just add playwright dependency to your pom.xml (i.e. you don't build it from sources), correct?

Exactly

Another idea, can you run the following in terminal:

Running the command works and it downloads the browsers.

---

The issue seems to be related to how calling the install process works in a Java Enterprise environment.

Is seems that the download fails in the com.microsoft.playwright.impl.DriverJar#installBrowsers method when the external process is called

 ProcessBuilder pb = new ProcessBuilder(driver.toString(), "install");
 pb.redirectError(ProcessBuilder.Redirect.INHERIT);
 pb.redirectOutput(ProcessBuilder.Redirect.INHERIT);
 Process p = pb.start();
 boolean result = p.waitFor(10, TimeUnit.MINUTES);

Using the ProcessBuilder.Redirect.INHERIT reuses the server's logging system. However on Windows at least I suspect that the log files are locked by the master process so the child process (playwright.cmd install) fails when trying to access those files.

I validated this assumption by running during debug the following line

pb.redirectError(File.createTempFile("playwright",".error"))

thus changing the stderr for the external process which then worked and the browsers were downloaded. The process also created an empty playwright161449457654957299.error file (sometimes with a few empty lines in it).

I'm now trying to figure out how to pass from our code to the external process a different stderr path which it can inherit so it works.

[yury-s]

Good finding! Thanks for the detailed explanation too, this is very helpful!

thus changing the stderr for the external process which then worked and the browsers were downloaded. The process also created an empty playwright161449457654957299.error file (sometimes with a few empty lines in it).

I believe what we can do to work around this is to manually read stderr from the child process into a buffer and then when the process exits dump all the logs using System.err.print in the playwright process. I assume this should work nicely with the bizarre logging environment in Java Enterprise.

[yury-s]

@vidi42 can you run with your fix and DEBUG=pw:protocol and see if it is working? I'd like to understand if we also need to update the other place where we call redirectError(ProcessBuilder.Redirect.INHERIT); on the driver process.

I've added a new separate thread that pumps messages from the child process manually (see #426). Can you run your test locally with that change applied and see if it helps. If it doesn't help we may redirect stderr to a temp file and read from the file after the process finishes but I'd rather avoid temp files if possible.

Could you also tell me how to setup enterprise environment similar to yours so that I could reproduce this locally?

[yury-s]

@vidi42 did you have a chance to validate if #426 fixed your problem? Unfortunately, we have to revert that change due to #439 but it'd be nice if you could give it a try with v1.11.0, if it does fix the problem we could think about similar fix.

[yury-s]

Closing as the discussion has stalled.