Skip to content
AI for E-sports

Real-time object detection for online gaming tournaments

Real-time object detection for online gaming tournaments

Abios Gaming is a startup that focuses on tracking real-time events in online gaming tournament video streams. They asked us to improve detection performance and make previously impossible detections, possible.

Abios Gaming provides an API for live information on games, teams, and players in esports tournaments. To strengthen Abios Gaming’s offer Modulai joined forces with their tech team to build an object detection and OCR solution for extracting information on the fly from real-time video streams of gaming tournaments. 

Among the objects to be tracked were various weapons and tools the players use in-game, as well as certain interactions. The system detects and tracks timestamped events in correspondence with the objects the players possess, or use, to provide a structured stream of events that captures essential information about the evolution of the tournament. In the absence of large sets of annotated image data, the core of the project was to create a robust pipeline for generating synthetic datasets with images and annotations (location and class of the objects). As well as train a neural network-based algorithm for improved detection, localization, and semantic segmentation.

We developed a pipeline for synthetic data generation capable of solving the object detection task. Several object detection implementations in PyTorch and Tensorflow were tested and tuned. The final solution was based on a Mask R-CNN that improved the existing classical computer vision solution substantially, enabling the client to generalize the solutions to new titles, simpler and faster. We worked in close collaboration with the team on the client’s side and handed over the solution for the client to use in new titles.

Wanna talk more real-time object detection?