Error initializing SparkContext.

[drjavaxml]

I would have sent this as a private email if I had an email address.
I am from SNOMED and do most of the demo sessions for our data analysis tool.
Our new tool will be based on Pathling, and the data will be generated by Synthea.
I can now generate scenario data on Synthea and wanted to install Pathling.
I have Docker, Postman Java etc installed. When I run (as described in your documentation)

docker run --rm -p 8080:8080 aehrc/pathling

I get an error about initializing Spark which I will paste below. Basically not enough memory initializing Spark.
I am looking for help in just getting this up and running so I can start the actual work of loading and analyzing data.

This is on Windows. I tried on Linux but then ran into a different problem about not finding shared 64 bit library files. So I'm back to Windows which seems the easier problem.

Here is the part of the long stack trace where it first goes wrong. I have not separately installed Spark.

20:56:19.802 [main] [] INFO a.c.pathling.fhir.AnalyticsServer - Initializing Spark session
20:56:20.621 [main] [] ERROR org.apache.spark.SparkContext - Error initializing SparkContext.
java.lang.IllegalArgumentException: System memory 464519168 must be at least 471859200. Please increase heap size using the --driver-memory option or spark.driver.memory in Spark configuration.
at org.apache.spark.memory.UnifiedMemoryManager$.getMaxMemory(UnifiedMemoryManager.scala:217)
at org.apache.spark.memory.UnifiedMemoryManager$.apply(UnifiedMemoryManager.scala:199)

Thanks
My email is phen@snomed.org Happy to take this thread off of github as it may not be an "issue" but it does prevent the "getting started" instructions from working.

[johngrimes]

If you are using Docker for Windows, try clicking on the tray icon, going to "Settings", then "Resources".

Set the memory slider to 3GB or greater, apply and restart, then try running the Pathling container again.

Note that the application does not necessarily need this much memory to run, it just comes down to the default JVM Xmx setting. It will go for a quarter of the available memory by default, which in this case is less than the absolute minimum needed for Apache Spark to run, which is 471.9MB.

What we probably should do is create an environment variable to control this setting, for those who need control of it. I will create a separate issue for this.

I will also create an issue to update the "Getting Started" section of the documentation.

[johngrimes]

Could you provide any further information about the issue you had when running on Linux?

Were you also using Docker on Linux?

[johngrimes]

@drjavaxml Just wanted to check back to see if we either fixed all of your issues, or at least captured them.

[johngrimes]

Closing this now, feel free to get back in touch if you have any more problems.