21 . 09 . 2021 to 24 . 09 . 2021
Bucharest, Romania

Detection and Recognition of Hand Drawn Website UIs Task at ImageCLEF 2021


On September 21-24, 2021, AI4Media will co-organise the workshop "Detection and recognition of Hand Drawn Website UIs Task" within the CLEF2021 - Conference and Labs of the Evaluation Forum in Bucharest.

The increasing importance of User Interfaces (UIs) for companies highlights the need for novel ways of creating them. Currently, this activity can be slow and error-prone due to the constant communication between the specialists involved in this field, e.g., designers and developers. The use of machine learning and automation could speed up this process and ease access to the digital space for companies who could not afford it with today’s tools. A first step to build a bridge between developers and designers is to infer the intent from a hand-drawn UI (wireframe) or from a web screenshot. This is done by detecting atomic UI elements, such as images, paragraphs, containers, or buttons.



In this edition, two tasks are proposed to the participants, both requiring them to detect rectangular bounding boxes corresponding to the UI elements from the images. The first task, wireframes annotation, is a continuation of the previous edition, where about 1,300 more wireframes are added to the existing 3,000 images of the previous data set. These new images will contain a bigger proportion of the rare classes to tackle the long tail problem found in the previous edition. For the second task, we present the new challenge of screenshot annotation, where 9,276 screenshots of real websites were compiled into a data set by utilizing an in-house parser. Due to the nature of the web, the data set is noisy, e.g., some of the annotations correspond to invisible elements, while other elements have missing annotations. To deal with this dirty dataset, part of the images will be cleaned manually. The development set will contain 6,555 images uncleaned and 903 images cleaned and the test set will contain 1,818 clean images.

More information about the event HERE



AI4Media's involvement:



Co-organisers: Mihai Dogariu, Liviu Daniel Ștefan, Mihai Gabriel Constantin, Bogdan Ionescu, AI Multimedia Lab, and AI Multimedia Lab participating team, UPB.