-Added support for html frame tags.
-Included support for inspecting code within frame tags.
-Added support for crawling meta refresh tags.
-Added a new optional specification to carry out deeper analyses.
-Fixed bug causing crawlax not to update candidates after running the
-Created a new test class that targets the wivet benchmark .
For detailed information refer to the patch files.
We used a set of frameworks to validate the results of our
improvements. WIVET is the most noticeable one, it is widely used to
test crawlers. If you run the WivetTest class, you will see if crawls
up to a 74% of the site. In the trunk version of crawljax it is only
able to crawl betwen 0% and 10% depending on the targeted node (due to
the issues fixed in our patches).