Fragmentation-based Crawling

[amesbah]

Integrate fragment-based crawling and test generation from the following paper: Fragment-Based Test Generation For Web Apps (FragGen)

How it works

In order to determine similarity between two web pages, FragGen segments the page into smaller fragments and compares the similarity between fragments. Fragment similarity is established through visual and DOM structure comparison.

FragGen also uses fragments to identify similar actions and prioritizes exploration of diverse actions. Fragment-based comparison also is used in the generation of test oracles to improve fragility of test cases in regression testing scenarios.

More details regarding the technique can be found in the paper

Limitations

• Does not support crawling iframes/frames
• Memory/CPU usage is not optimized and can make the crawler inefficient for web applications with large DOMs.
• Has not been tested extensively for browsers other than Google Chrome

Usage

In order to activate FragGen, use FragGenStateVertexFactory as the state abstraction. An example usage can be found in FragCrawlExample