Repository navigation
In this release, you will find the necessary code to:
- (Training folder) Train models for application-level, window-level UI group, container-level UI group, general UI element, and text UI element for the detection of UI components in desktop interfaces.
- (Predict folder) Perform predictions and assess how the models detect UI components and extract the UI hierarchy, in SOM format, from an image. For this purpose, the file Models.zip is attached, containing the already trained models available for making predictions.
- (Comparison Experiment folder) Launch experiments to compare the results of UI component detections and the extraction of the UI hierarchy, in SOM format, from our proposal Screen2SOM with other frameworks like YOLO, MobileSAM, Moran, and UIED.