Modulai trained multiple machine learning models for On Device Research to streamline their business operations, including fraud detection classifiers, a survey ranking system, and user segmentation, improving security and enhancing user experience across their online survey platform.
Stats
35M+
Surveys delivered
3.2M
Survey completes for training
2009
Mobile surveys since
Challenge
On Device Research is a UK-based tech company with a platform that allows users to complete online surveys in exchange for a monetary reward. Like similar businesses, ODR can attract fraudulent users trying to cheat the system. It is essential to protect the business from such users while not punishing good users, and to maintain a strong user experience to attract and retain customers and survey suppliers.
Solution
In close collaboration with the ODR team, Modulai trained multiple models per market to streamline operations. Classifiers were trained to predict the likelihood of a survey being completed with bad intent, using features describing the user, context, and survey. A ranking system was developed to sort available surveys by relevance for each user. Finally, model results were aggregated across tasks and used for user segmentation, enabling personalization of application logic based on segments.
Tools
Feature engineering was performed in relational databases. Gradient boosting models were trained using Dask to handle the large dataset size, and the trained models were deployed in AWS using various managed services. Training and validation pipelines were developed in Python with DVC for versioning.
Value created
The models let ODR catch surveys completed with bad intent without penalizing genuine users, protecting the rewards business from fraud while keeping the experience strong for the legitimate users and survey suppliers it depends on. Trained per market and combined with relevance ranking and user segmentation, the system also lets ODR tailor how the platform behaves for different users, rather than treating everyone the same.
Learn more
