Benchmarking represents a vital tool for the development of new technologies as it allows and eases a fair process for comparing the performance of different AI algorithms on common grounds, e.g., data, training, and metrics. The dedicated AI4Media open Benchmarking Platform is in its Prototype phase at this moment, providing such capabilities.
It was developed on the CodaLab framework. A testing benchmark is also provided as an example, namely the novel “late fusion” benchmark (ImageCLEFfusion 2022 task). The platform allows the users to create benchmarking tasks, create cloud-based repositories, manage participants and submitted data, as well as API integration.
It brings several advantages:
Access the prototype source code HERE
Author: Bogdan Ionescu, (Politehnica University of Bucharest)