Customer request:Testing a new discount system to increase profitability. The goal was to provide small discounts (<$20) on flights to Skyscanner users, elevating their offers in search results. While this approach might decrease immediate profit, it offered potential for crossselling other services like insurance, cars, and hotels.Project goal:Our mission was to predict potential profit, determine optimal discount allocation, and assess the probability of converting discounted buyers into long-term clients.Solution and Technologies:The solution involved two Apache Kafka topics with 1 TB data/hour throughput. Data was joined using Spark Structured Streaming, transformed and applied to an ML model. The project was focused on building en efficient data pipeline to apply ML models to predict user revenue.The results were then converted to rules and submitted to the rule engine
 on AWS. A/B testing was conducted to compare the results of the solution with the base, and real-time statistics were monitored with Grafana/Graphite dashboard.Conclusions on the ProjectOver the 6-month duration, we successfully determined costeffective discounts, identified the target segment for the discounts, and calculated the conversion rate of discounted customers into loyal clients. By comparing statistics between the pipeline with discounts and the one without, we observed a significant increase in Expedia’s profit in thr ticketing sector on the Skyscanner by 32%.