Soccernet Brings AI To The Game

About

In October 2022, at the FIFA headquarters in Zurich — the international “Home of Football” — Anthony Cioppa gave a presentation on the rapidly developing artificial intelligence (AI) project SoccerNet.

The presentation was the culmination of five years of work with fellow football fan and the original creator of SoccerNet, Silvio Giancola. From the kernel of an idea in 2018, the duo has created a unique open platform for AI-driven sports analysis backed by a global community of more than 500 researchers who participate in annual competitions and challenges.

“I met Silvio at a computer vision conference in 2018 where we both had a paper,” says Cioppa. “My paper was a follow-up to my master’s thesis on an AI system that could recognize the different phases in a soccer game, and Silvio’s was on the first version of SoccerNet, which I immediately thought was an incredible idea, and so we got to discussing it.”

“A year later, while at Aalborg University, I started working on SoccerNet and contacted Silvio to collaborate. It was from here that we proposed the first context-aware deep-learning method to spot actions in soccer broadcast video.”

Shared passion

Cioppa is a visiting postdoctoral researcher at KAUST where he works with Giancola on SoccerNet in the research group led by Bernard Ghanem.

“My lab focuses on several research themes in the fields of computer vision and machine learning,” says Ghanem. “I am particularly interested in next-generation AI methods for analyzing long-form videos.”

Cioppa is a postdoctoral researcher on computer vision and AI at the University of Liège in Belgium, but it is his interest in soccer that led him to KAUST. “The first AI system I developed for my thesis was actually a project from my internship at EVS Broadcast Equipment, one of the world leaders in live broadcast solutions and now one of our collaborators on SoccerNet.”

Giancola previously studied at the Institut National des Sciences Appliquées in France and Politecnico di Milano in Italy before coming to KAUST in 2017.

“I am passionate about sports, so combining my passion with my work motivated me more than anything else,” says Giancola. “With the rise of AI and deep-learning data-based approaches, I was interested in embedding the knowledge we have when watching sports into intelligent machines that will understand what happens in a game.”

A community of fans

One thing that makes SoccerNet unique is that it uses a very large reference set of soccer game recordings — 500 games and 850 hours of video — providing a platform for research. Prior to SoccerNet, sports AI research used small in-house datasets, making it difficult to compare performance among different AI approaches.

To make the video library useful for AI, the videos need to be labeled by humans to annotate events and actions, which is then used to train the AI.

“Annotating data takes a lot of time and money,” says Cioppa. “We had to find funds from various sources and create a unified annotation format and tool that would allow annotators and users to work efficiently and consistently.”

Giancola annotated the videos in-house for the first version of SoccerNet in 2018, labelling 6,000 basic events. With Cioppa, the pair then recruited teams of hundreds of students and collaborators to ultimately record 5.5 million annotations, including 17 classes of action, camera shot segments, streams and replays. They even annotated player and ball tracking — a task that took many months in campaigns over a number of years and is still in progress.

“We worked with experts in annotations to define the annotation format and guidelines and developed a custom annotation tool that was user-friendly and efficient,” says Giancola. “Our amazing team of passionate paid collaborators then helped us to annotate the thousands of meta-data entries for each game.”

An annual AI World Cup

Giancola and Cioppa have organized annual global SoccerNet challenges since 2021. The results of the 2022 challenge were presented at the 5th International ACM Workshop on Multimedia Content Analysis in Sports.

“SoccerNet is the largest and most comprehensive open-source dataset for video analysis in sports, and we also organize annual open challenges for researchers from around the world, including our industry partners Sportradar, EVS Broadcast Equipment and Baidu Research,” says Cioppa. “These challenges create a great dynamic in our dataset and support its use by the research community.”

There are now 19 researchers and industry groups involved in organizing SoccerNet projects, with Cioppa and Giancola being the lead organizers of SoccerNet research and annual open challenges.

“We have organized three challenges so far,” says Giancola. “Our 2023 challenge has just started, but we are already planning the 2024 challenge and tasks. Our goal is to make SoccerNet the go-to resource for researchers and industry looking to develop new computer vision techniques and applications for sports. KAUST has an amazing infrastructure that has helped us to share our data worldwide as a unique publicly available collaboration platform.”