Shot speed prediction using object tracking for Passion Football

The ICONS app by Passion Football lets football players of all levels develop their skills and challenge others. We developed a solution that uses the smartphone camera to track and accurately estimate a players shot speed.

Background
With the app, users can access a variety of exercises that measure essential football skills. These exercises are designed to be both fun and interactive, while also providing analytical insights to the player and suggesting exercises that target and improve their weak areas.

One such exercise allows players to measure and improve their shooting power. By recording a video of the player kicking the ball as hard as they can against a goal, the app is able to analyze the video and determine the speed of the player’s shot.

Solution
An object detector locates the bounding boxes of all footballs and goals in the video. An object tracker refines the detections from the object detector to reflect real-world constraints, such as the fact that footballs follow a certain trajectory, and accomplishes this by applying a constant velocity model and a 2D nearest-neighbor filter.

Finally, the known size of the football is used in conjunction with trigonometry to translate the speed of the ball inside of the video to real-world speed. The app communicates via a REST-api with a cloud solution that performs the inference and performs all necessary calculations.

Tools/Tech
The codebase was developed in Python, with the knowledge that our model could be exported to run on a mobile via the ONNX project. DVC was used for data and model version control and the object detection data was annotated with a SaaS solution. We used Pytorch/Pytorch lightning to train a suitable object detection model. For better results, the data was augmented. The object tracker uses Kalman filters and was developed in-house.