project coordinator
academia university
research center
10 projects funded under Open Call #1 >>
Application track
| Research track |
The AI4Media - Open Call #1 aimed to engage entrepreneurs, companies (e.g., SMEs, mid-caps) and researchers that regularly develop and integrate applied research in the field of AI, to develop new research and applications for AI, and contribute to the enrichment of the pool of technological tools of the AI4Media platform. 7 specific challenges and 2 open challenges across the Research and Application tracks were addressed by the Open Call.
In total, 60 proposals from applicants in 22 countries were submitted to the open call.
These are the winners:
Musicube
(Germany)
Track & Challenge addressed:
Application track - Navigating multi perspectivity in media heritage collections
Summary of the project:
Music streaming services give access to more than 60 million songs, but the biggest part remains undiscovered because search functions are limited, playlists are predefined and recommendation engines work on an “other users have also heard” basis. Thus, users are locked in filter bubbles. That is why we need music discovery tools that both work with musical and sound features but adapt to the personal musical taste of users. These tools can help them escape their musical filter bubble and support content creators to become discoverable.
Musicube is an SME that builds AI software for audio and especially music. Musicube developed neural networks that process audio files and automatically tag them with musical features, sound features and emotions. The business model is B2B, so that companies from the media sector use Musicube’s Auto-Tagging Software to enrich their metadata or semantic music search to find music by its features (e.g. “fast and happy songs with acoustic guitar and female vocals”).
In the AI Empathic DJ project, we intend to further develop this software to adapt to listener’s perspectives on music. For example, when searching for “relaxing songs” the musicube engine currently suggests typically ambient relaxing music. However, different people find different styles relaxing, as for example Jazz and Heavy Metal fans. We want to further develop our AI to learn musical tastes and adapt to it. The AI can make personalised suggestions based on a small sample and become the person’s own perfect empathic DJ. Technically, the “empathy” for a user’s perspective on music is achieved by a reduction of the neural network’s (AI’s) own view on music. Musicube’s current neural net produces a Euclidean space of about 500 dimensions and localises all songs in that space. A specific musical taste can be defined as a subspace of this total knowledge. Starting from points inside this user’s subspace (also known as taste or perspective), semantic music search or recommendations can be made that are outside a user’s scope but inside his or her own view on music. This way death metal fans get other search results for “aggressive” music than HipHop fans, just to name an example.
The AI Empathic DJ will be a module that can be integrated in musicube’s existing services, but also in third party software like streaming services and the AI4Media ecosystem.
AdVerif.ai
(Israel)
Track & Challenge addressed:
Application track - Evidence Collection in Digital Media Authentication
Summary of the project:
Propelled by the latest advancements in open-source intelligence (OSINT), our team of data scientists and veteran fact-checking experts will develop a next-generation intelligence platform to make collaborative collection of evidence for media authentication easier and faster. The platform will adopt cutting-edge AI methods from cyber-security to the media domain, empowering fact-checkers, investigators and journalists to be more effective. It will power more effective, transparent, explainable and reproducible verification processes, across all media formats - videos, audios, images and text - at scale.
Today, cyber-security researchers and forensic investigators have a variety of verification capabilities at their disposal, yet the existing workflows are fragmented across many tools with individualized and unstructured processes. Leveraging the latest advancements in NLP and Computer Vision, the team will develop an innovative platform which addresses these challenges and provides a centralized, collaborative and structured workflow to make evidence collection and preservation quick and simple.
The platform will provide a centralized User Interface with integrations to the most prominent and effective OSINT Tools, into one platform. It will have built-in standardized processes with checklists and best practices for verification. The technology is building upon an existing prototype we developed for fact-checking and media authentication on Twitter, already tested by leading fact-checkers. The Twitter messaging UI empowers users to exchange information and promotes collaborative investigations. Moreover, the system will be integrated with an innovative e-signature time-stamped solution for evidence storage that goes beyond the existing tools.
We will deliver a fully working system that will help save 20%-40% of the time needed for verification and make fact-checkers more effective, while advancing state-of-the-art in media authentication and contributing to the AI4Media ecosystem. To promote the project, we will publish whitepapers and case studies to demonstrate the effectiveness and value of the system and promote it via social media and webinars. Also, the team will leverage partnerships with Nielsen, Twitter and the International Fact Checking Network (IFCN) to raise awareness and position the solution as a standard for media authentication.
AdVerif.AI is an AI start-up backed by Nielsen and a global leader in disinformation defence with a social-impact mission to keep users safe from harmful content while building trust into digital media and advertising networks. It is a member of the Microsoft AI for Good program and was named by CB insights among 36 global game changers: start-ups with potential to transform society and economy for the better.
IN2 Digital Innovations GmbH
(Germany)
Track & Challenge addressed:
Application track - Navigating multiperspectivity in media heritage collections
Summary of the project:
The project CUHE aims to develop and demonstrate a web-based application based on AI recommendations that will allow cultural heritage professionals (e.g. museum curators, archivists) as well as (humanities) researchers to explore existing media and cultural heritage digital collections in a more holistic way and allow them to curate new galleries or create digital stories and exhibitions which can showcase and share the new insights gained.
The project will target a key infrastructure for researchers and heritage professionals: Europeana. Users can not only find over 50 million records but also explore over 60 curated digital exhibitions, countless galleries and blog posts.
CUHE plans to improve and make more scalable the process of curating digital exhibitions and galleries by providing a facility for related content exploration based on an AI recommendation system that exploits the metadata available for the content. Resulting recommendations would be at the level of records related to a given record, records related to a given curated collection (e.g. gallery, exhibition,etc), and collections related to a given collection. The work in the project will not only focus on the recommendation algorithm themselves, but most importantly, will aim through co-creation to come up with a user interface that allows the users (who are often not ICT specialists) to understand what data dimensionalities resulted in the relation between the records or collections presented and give the option to adjust these parameters with several options. The recommendations could be used for both curating new collections (to help find interesting, diverse and related content that will enable a holistic documentation) and for contextualising and exploring connections between existing collections, allowing the audience to examine the presented topic from multiple perspectives.
JOT Internet Media
(Spain)
Track & Challenge addressed:
Application track - Evidence Collection in Digital Media Authentication
Summary of the project:
During the last years, video marketing has been proved as a highly efficient media to engage targeted audience at reasonable prices and good performance indicators. These aspects motivatesthat the global video ad investment forecast in this content media shows a continuous growth of 12% till 2025, reaching $148 B.
This huge market size has associated an enormous amount of web traffic being redirected to the companies ́ websites. However, based on the information reported by the video ad platforms in JOT video campaigns, between 20-30% of this traffic comes from inauthentic users, programmed to spend the companies budget clicking in the ads with no post conversion, limiting marketing campaigns impact and reducing the return of investment by wasting companies’ budget.
In this scenario, JOT, as a performance-based marketing company that manages 200,000 ad impressions and 10,000 clicks per day, will develop an innovative AI based system to: (i) identify the main behavioural patterns of inauthentic users to predict their actions and limit their impact in the video marketing campaigns and (ii) model the quality score associated to a campaign. These data services will be developed thanks to the existing database describing the video web traffic (YouTube) of more than 12 landing during more than a year. This ensures that algorithms will be trained, tested, and validated with real data to reach the required quality and accuracy to be used in production conditions for decision making. To reach such objectives, JOT team is formed by complementary skills combining, video marketing campaigns management and optimization, data science, big data processing and cloud architecture deployment. The team has been working in data collection and development of predictive services at small scale to validate to data accessibility and algorithms implementation as realistic proof of concept. Thanks to the AI4Media support, JOT will be able to develop a system capable of identifying unauthentic/authentic web traffic for video marketing campaigns to increase trust in this media as a tool to reach the right audience at the right time, exploiting all the features and benefits of this media content ad platform.Varia UG
(Germany)
Track & Challenge addressed:
Application track - Leveraging the power of media archives through Artificial Intelligence
Summary of the project:
Varia Research is a project that aims to put journalists, the heavy lifters of the industry, in its focus, by providing an AI powered research solution.
For too long, the efforts in terms of digitalization and tooling have been focusing on content monetization and distribution. While these two tasks are undoubtedly important, the emphasis on content creation was neglected. At the same time, budget cuts of the recent years have led to an increasing pressure on journalists, who have to produce more content with less capacity – while adhering to constant quality standards.
Varia Research is here to change that. The goal of this project is to provide a research solution for journalists, that will help them structure their research much more efficiently and partially automate their research by leveraging machine learning insights. All with the ultimate goal to enable journalist, freelancing or employed, to produce more and better content. The outcome should be a SaaS application that is easy to use and flexible enough to fit into individual and newsroom workflows.
With this approach, Varia Research appeals to the AI4Media use case 2, which is looking for projects in the “Smart News Assistant” field, that aims to find solutions that help journalists to better cope with the ever increasing amount of news that they are confronted with – and supports them in the content creation process. Varia Research further answers the challenge 7 of the AI4Media Application Track, by leveraging publisher archives for efficient content production.
lnternational Hellenic University
(Greece)
Track & Challenge addressed:
Research track - Combining deep learning-based computer vision and classic path-planning/ control for autonomous UAV cinematography tasks
Summary of the project:
The project aims to deliver a complete framework for moving people and objects detection and tracking in order to extract evidence data (e.g. photos and videos from specific events) at realtime (when the event occurs), like cinematography tasks, though a reactive Unmanned Aerial Vehicle (UAV). To this end, edgeAI4UAV project will implement an edge computation node for UAVs. The node will be equipped with a stereoscopic camera, which will provide lightweight stereoscopic depth information to be utilized for the evidence detection and UAV locomotion.
To achieve this, the edge node of the UAV will be endorsed with an embedded processor to allow edge capabilities to the UAVs, since the information will be processed locally. The edgeAI4UAV project will develop lightweight computer vision and AI (deep learning) algorithms capable of detecting and tracking moving objects, while at the same time will ensure robust UAV localization and reactive navigation behaviour. The results of the algorithms will be exploited by an embedded decision-making module (edge computation), which will accomplish dedicated navigation missions, like follow a specific moving object (e.g. specific actor, animal, etc), turn at a specific angle of view (e.g. side face, front face, etc.), getting closer or farther to it, etc. Thus, the embedded decision-making module will have the ability to define the navigation behaviour of the UAV in a dynamic manner at real-time, in order to accomplish the envisioned tasks. The UAV will be programmed with a main mission (e.g. to follow the main actor), which can be temporarily interfered by the subordinate missions (e.g. a closer photograph from the front or side face of the main moving object), and then the main mission will be dynamically re-adjusted in order to smoothly continue its main navigation plan without disturbances, offering smooth data continuation.
Furthermore, the UAV will be equipped with a WiFi module, providing the ability to send specific photographs to a server during the flight. Thus, photographs will be sent to a centralized platform at real-time, without need to land the UAV in order to upload the photographs and/or video to a remote data storage (like usb stick, memory card, etc.), connect the USB stick (or memory card) to a PC and upload them to the server.
The project will fuse the already mature technology of industrial UAV applications with the edge computing advantages to extract scene semantic information through reactive mission planning to be researched and implemented as an adaptive decision making system, constituting the UAV’s cognitive functionalities.
Irida Labs
(Greece)
Track & Challenge addressed:
Research track - Bio-inspired deep learning
Summary of the project:
The goal of NeurAdapt project is to explore a new path in the design of deep Convolutional Neural Networks(CNNs), which could enable a new family of more efficient and adaptive models for any application that rely on the predictive capabilities of deep learning. Inspired by recent advances in the field of biological Interneurons that highlight the importance of inhibition and random connectivity to the encoding efficiency of neuronal circuits, we aim to investigate the mechanisms that could impart similar qualities to artificial CNNs.
Established techniques such as Channel Gating, Channel Attention and calibrated dropout, each proposed independently and with different objectives, can offer tools to formulate a novel building block for CNN models that expands the functional diversity of the standard convolutional layer. By formulating a differentiable convolutional operator with additional mechanisms for competitive inhibition/excitation and stochastic activation with tunable probability, we pursue the hypothesis that the optimization of the learning tasks will drive the model to create modes that are information-rich, in a process like the one observed in biological neural networks.
Furthermore, the stochastic nature of neuronal activity if appropriately modelled and augmented with sparsity-inducing mechanisms, has the potential to enable the training of models with parametrized levels of sparsity, offering the capacity to control inference/complexity tradeoff on-the-fly, without any need for additional finetuning. Achieving a functionality like this can impart new capabilities to information processing systems which are based on CNNs. Inference with adjustable level of thoroughness/speed can enable applications such as rapid content search in large media databases, energy-aware decision-making etc.
The main outcome of the proposed project will be a new methodology for designing efficient deep CNN architectures regardless of the specific task and target domain. Furthermore, NeurAdapt aims to create new knowledge regarding the dynamic behavior of the excitation- inhibition mechanisms in feed-forward DNNs, their capabilities for further development and the respective limitations.
Institut Jozef Stefan
(Slovenia)
Track & Challenge addressed:
Research track - Open Challenge
Summary of the project:
One of the primary concerns of the news media industry is how to manage the comments that readers post on news articles. Most online news publishers provide content in a form that allows readers not only to access it, but to post their own comments: for readers, this is valuable in allowing them to express their opinions and interact with each other; for the publishers, it is valuable in that it provides a way to understand their audience, and increase reader engagement. However, the ability to comment is often mis-used, with comments used to advertise, to abuse others, to spread misinformation and to post illegal content. In many countries, publishers are legally accountable for the content that is posted. Publishers therefore usually employ some form of moderation: human moderators will scan the comments posted, and apply some moderation policy to block those that should not appear, and in severe cases perhaps ban the users from posting again.
This job is not easy: decisions can be subjective and hard to make consistently; it can be easy to miss comments that need blocking; and when high volumes of comments are coming in (peak volumes of many thousands of comments per hour are not unusual during events of note) it can be difficult to keep up. There has therefore been great interest in recent years in AI tools to assist moderators: tools to analyse the content of comments using natural language processing (NLP) methods and help flag those which should or should not be blocked, helping speed up the moderators’ work and produce consistent results. Recent research shows impressive accuracies.
However, transferring these AI methods from research to practical industry use is not straightforward. Tools must usually be trained on large volumes of data labelled with the correct expected output decisions: this data must be in the domain, style and language that will be seen in use, so must generally be produced from scratch for any new publisher, newspaper or topic. This process is expensive and needs expertise in NLP and AI methods.
This project seeks to develop new methods to bypass this problem and make the initial implementation process easy and fast. We will develop methods for semi-automatic annotation of data, including new variants of active learning in which the AI tools can quickly select the data they need to be labelled. We will build on recent progress in topic-dependent comment filtering to build tools that can take the context of the associated news article into account, reducing the new data needed. Finally, we will use recent progress in transfer learning to allow tools to be initialised from existing labelled data in other domains and languages, reducing the amount of data required.
The result will be a suite of tools to enable easy, fast, practical implementation of accurate, robust comment filtering methods for use in the news media industry.
University of Manchester
(United Kingdom)
Track & Challenge addressed:
Research track - Human-centred interactive explainable AI
Summary of the project:
Assessing the veracity of claims is a vital capability in the modern world, but it is a task that the public is often ill-equipped to do. This is evidenced, for example, in debates around human contribution to climate change, public health policies and political elections. As a result of targeted disinformation campaigns, many users are inadvertently spreading misinformation, without critically reflecting about its sources, as the information is often presented without further context. As experts cannot provide contextualising explanations about the validity of a claim instantaneously, this gives the opportunity to research into automated claim and fact verification approaches.
State-of-the-art research on fact verification mostly focuses on the capability to identify misleading claims. However, for end-users, it is important to provide explanations why exactly a claim was identified as wrong. These explanations serve both as a context for the claim and as an insight into the reasoning process that led to the veracity decision. Existing fact verification approaches rely on deep learning-based models optimised on large static datasets to automatically classify whether a claim is true based on retrieved supporting evidence. It is however unclear, whether end-users will accept these verdicts without further context. This is further problematic, as these models have been shown to exhibit biases inferred from the datasets they were optimised upon, for example relying on the appearance of specific keywords.
Instead, we take a novel approach to developing AI-driven fact verification systems, with explainability at their core. We will leverage the large availability of high-quality human-written claim verification articles from specialised journalist outlets such as Full Fact to create a dataset for the task of explainable fact verification. We will use the data to develop a fact verification system underpinned by deep learning based, generative language models that will generate human-understandable explanations that contextualise their verdicts.
The dataset collection will follow the paradigm of iterative crowdsourcing, where annotators not only annotate data to train neural networks but also rate the quality of the predictions of models optimised on the previous iteration. Thus, we will shift the focus to an evaluation methodology that revolves around the explainability requirements of humans. This will allow us to iteratively refine the developed tool and subsequently to learn to judge the quality of explanations from human annotations.
Sofia University
“St. Kliment Ohridski” GATE Institute (Bulgaria)
Track & Challenge addressed:
Research track - Innovative solutions for fake content detection in line with fundamental rights and the developing EU regulations
Summary of the project:
In the last few years, the problem of fake content and disinformation spread worldwide and across Europe has dramatically increased (especially in social media). Even if there is a large body of research, there are countries which still lag (e.g. have few or no fact-checkers, no tools and resources. This is particularly challenging when detecting deep fakes. Although there is some research on fake news detection for low-resourced languages (e.g. Romanian, Bangla, Tagalog), there are no guidelines on how to solve this problem for a new language. Also, even if disinformation is defined as an intentional spread of fake information, it is currently addressed by considering only its fakeness and harm, but not its intent. This is technically wrong and can prevent distinguishing between misinformation and disinformation.
TRACES is addressing these problems, by finding solutions and developing new methods for disinformation detection in low-resourced languages. The innovativeness of TRACES is in detecting both human and deep fakes disinformation, recognizing disinformation by its intent, the interdisciplinary mix of solutions, and creating a package of methods, datasets, and guidelines for creating such methods and resources for other low-resourced languages. The Use Case of TRACES is Bulgarian, the national language of a European Union (EU) country with a very low level of media literacy, problematic media freedom and a geopolitically strategic position at the border of the EU. Bulgaria has a high number of self-taught advanced computer hackers, which makes it highly plausible (but not researched) the existence of deepfakes in Bulgarian social media. Detecting and signalling fake content and especially disinformation is thus a critical need for Bulgaria, but there is only one independent fact-checker and very little NLP research on the topic. Another challenge is that Bulgarian is a low-resourced language, with very few NLP tools and datasets, and almost none for processing social media texts.
The proposed research is very well aligned with AI4media Open Call 1, Use Case 1 and Challenge C4-Rt:
1) It fills a significant gap in the EU’s existing AI research and technologies on disinformation detection;
2) It applies AI methods and tools to support journalists and fact-checking experts in digital content verification and disinformation detection;
3) Its results can be integrated into AI4Media, Truly Media and TruthNest; and
4) it is in line with EU’s fundamental rights and all relevant EU regulations (the EU Charter of Fundamental Rights, GDPR, the AI ACT, the Digital Services Act, and the EU Code of Practice on Disinformation) - no personal information will be collected, nor any content removed.
The outcomes of the project will provide crucial insights for other EU countries and will consist of language resources, annotated disinformation datasets, machine learning methods, a tool for potential integration into Truly Media to support the discovery of disinformation by journalists and new language adaptation guidelines.
Cookie Settings
AI4Media may use cookies to store your login data, collect statistics to optimize the website’s functionality and to perform marketing actions based on your interests. You can personalize your cookies in .
COOKIES