New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
scala-scraper's implementation of HtmlUnit doesn't have .close() #35
Comments
Hi @piercelamb! Sure, this can be easily done. However, calling something like A question about your problem, are you reusing the same |
@ruippeixotog I'm actually not too sure. I'm using Play Framework 2.5.9 so everything is dependency injected. In order to inject HtmlUnitBrowser i had to make this:
Because it doesnt have a parameterless constructor. That then gets injected like this:
And passed to an Actor like this:
How would I access that .close() method? |
Hmm, I'm not sure I understand one thing in your code: why do you add Either way, if you are using class HtmlUnitBrowserFactory {
def newBrowser() = new HtmlUnitBrowser
} And create a new browser instance each time you run a job. |
@ruippeixotog Great point on the constructor. Major oversight. My issue is that browser.close() does not compile, e.g.
|
Oh, I guess I wasn't clear. The method still doesn't exist, I'm suggesting implementing it as a possible solution to your use case :) I'll get to it this weekend. |
@ruippeixotog awesome! I'll test it out as soon as you have it ready. Thank you |
@piercelamb Not really on the topic, but I am curious to know why/how you are using akka actors for screen scraping? Can you mention a bit what is the use case for akka actors? Thanks. |
I have an akka Actor that starts up every 3 hours to scrape some content using HtmlUnitBrowser (have to use this because of JS execution). Everything works fine except memory usage jumps every time it starts and stays constant at that new level. So eventually I run out of memory. I'm not 100% sure HtmlUnit is the issue but they do have a FAQ question about it specifically:
http://htmlunit.sourceforge.net/faq.html#MemoryLeak
As such, I'd like to test closing the browser after its used in the akka actor every time. However, I don't see a .close() method on HtmlUnitBrowser.
Please advise.
The text was updated successfully, but these errors were encountered: