The team was tasked to aid a startup, focusing on tracking real-time events from online gaming tournament video streams, improving the detection performance, and making previously impossible detections possible.
Among the objects to be tracked were the various weapons and tools the players use in the game, as well as certain types of interactions. The system detects and tracks timestamped events corresponding to objects the players possess or use, in order to provide a structured stream of events that captures essential information about the evolution of the tournament.
In the absence of large sets of annotated image data, the core of the project was to create a robust pipeline for generating synthetic datasets with images and annotations (location and class of the objects), as well as training a state-of-the-art neural network-based algorithm for detection, localization, and semantic segmentation.
This client is a hot and upcoming startup that provides real-time data about ongoing online gaming tournaments to its customers. They track tournaments for a large set of the most popular titles, including World of Warcraft and Counter-Strike Go. They track everything from the composition of the teams to various events and interactions between players.
How Modulai did it
A pipeline for synthetic data generation, capable of solving the object detection task was developed. Several state-of-the-art object detection implementations in PyTorch and Tensorflow were tested and tuned. The final solution was based on a Mask R-CNN and improved the existing classical computer vision solution substantially, enabling the client to generalize the solutions to new titles (games) simpler and faster.
The Modulai team worked in close collaboration with the team at client’s side, and handed over the solution for the client to roll out for new titles.